AI Misjudgment Brings U.S. Military to Brink of Operation Against Chinese Vessel; Error Found Just Before Execution

[Financial News] It has belatedly emerged that the U.S. military prepared a military operation against a Chinese vessel based on incorrect information generated by artificial intelligence (AI), but discovered the error just before carrying it out and called off the operation.
According to CNN on the 18th local time, an intelligence report circulated within the U.S. military this spring, while the war involving Iran was underway. It claimed that a Chinese vessel in the Middle East was carrying components that could be used in a nuclear weapons program.
After the report was circulated, the U.S. military began preparing an operation to interdict the vessel. Armed personnel had reportedly prepared to board it, and military aircraft had even been scrambled.
However, officials discovered a problem while reviewing the report shortly before the operation was to be carried out. The report had been written by an analyst with United States Special Operations Command (USSOCOM) using AI. During the analysis, a chatbot had mistakenly identified the Chinese vessel's cargo as items related to a nuclear weapons program.
A source told CNN that the intelligence was "entirely false" and that "a war almost started" because of it. If the U.S. military had actually used force against the Chinese vessel, military tensions between the United States and China could have escalated sharply.
The error began when an analyst entered information about a Chinese vessel's cargo manifest, obtained by U.S. Special Operations Command Pacific, into a chatbot. After combining and analyzing publicly available information with classified signals intelligence held by the United States government, the chatbot produced the incorrect conclusion that the vessel's cargo was linked to a nuclear weapons program.
The analyst entered the result back into the AI system and had it formatted as a standard intelligence report used by the U.S. military before distributing it internally. Because AI was used not only to analyze the information but also to draft the report, the erroneous assessment circulated in the form of an official intelligence document.
CNN noted that the case shows how failures to establish adequate systems for filtering out AI errors, as the DoD and intelligence agencies accelerate their adoption of AI, could lead to actual military misjudgments.
U.S. Defense Secretary Pete Hegseth announced the "AI Acceleration Strategy" in January and has been expanding the use of AI across defense operations, including intelligence analysis and target selection. However, each U.S. government agency reportedly uses different AI tools and safety standards, and no unified guidelines have been established for determining how and according to what standards AI-generated information should be verified.
The growing use of AI in the target-selection process, which determines whether an actual attack will take place, is particularly concerning. One source explained, "The use of AI in target selection is clearly on the rise," while noting that specific guidelines to prevent civilian casualties and mistaken attacks on friendly forces remain inadequate. 

[email protected] Yoon Hong-jip Reporter