AI Models Can Identify Human Targets for Autonomous Weapons, Study Warns
A new study by Vals AI found that leading AI models, including from American and Chinese developers, lack safeguards to prevent their use in autonomous warfare, often identifying civilians as targets.

Illustration: AI-generated image
Leading artificial intelligence models are dangerously unprepared to resist being used for autonomous warfare, according to a new study by the American start-up Vals AI. The findings, published by AlphaSignal, show that both American and Chinese models almost never refused to identify human targets in drone and satellite images, even when those targets were civilians, ambulances, or wounded soldiers.
Vals AI tested the models in two drone scenarios using real images from Ukraine. The models frequently flagged civilians, medical vehicles, and disarmed or injured fighters as targets. In a satellite imagery scenario based on the Iranian city of Minab, Anthropic's Claude Opus 5.5 pointed to a school as a target in 70 percent of cases. The school was later hit in an air strike, resulting in the deaths of more than 150 civilians, the report notes.
"All leading models lack sufficiently reliable safeguards to prevent their use for autonomous combat operations, especially by non-state actors," Vals AI stated. "None of our requests were blocked by an automated safety system, and refusals from the models themselves were rare. When they did refuse, it was extremely easy to trick them into completing the task by telling them they were participating in a game."
The researchers found that refusals came only from the models themselves, never from developer-imposed safety mechanisms. For example, Claude's willingness to search for targets jumped from just 6 percent in the Opus 4.5 version to 92 percent in Opus 5.5. Even after seeing the consequences of their actions and human casualties, most models continued to follow instructions.
Vals AI has informed OpenAI, Anthropic, Google, Meta, SpaceX, and US national security services about the results. The researchers warn that the ability of AI models to perform tasks that pose a lethal danger to humans could lead to a "catastrophic misalignment" between assigned tasks and actual goals when used in autonomous weapons systems.
The study clarifies that the models were not asked to make final strike decisions — something they typically refuse — but the targeting information they provide could be used by an autonomous or semi-autonomous system to decide and carry out an attack. Vals AI urges developers to implement new safeguards and additional safety training to limit this serious category of misuse.





