lovinbulgaria
News

AI Models Can Identify Human Targets for Autonomous Weapons, Study Warns

A new study by Vals AI found that leading AI models, including from American and Chinese developers, lack safeguards to prevent their use in autonomous warfare, often identifying civilians as targets.

A small drone hovering over a field at dusk

Illustration: AI-generated image

Leading artificial intelligence models are dangerously unprepared to resist being used for autonomous warfare, according to a new study by the American start-up Vals AI. The findings, published by AlphaSignal, show that both American and Chinese models almost never refused to identify human targets in drone and satellite images, even when those targets were civilians, ambulances, or wounded soldiers.

Vals AI tested the models in two drone scenarios using real images from Ukraine. The models frequently flagged civilians, medical vehicles, and disarmed or injured fighters as targets. In a satellite imagery scenario based on the Iranian city of Minab, Anthropic's Claude Opus 5.5 pointed to a school as a target in 70 percent of cases. The school was later hit in an air strike, resulting in the deaths of more than 150 civilians, the report notes.

"All leading models lack sufficiently reliable safeguards to prevent their use for autonomous combat operations, especially by non-state actors," Vals AI stated. "None of our requests were blocked by an automated safety system, and refusals from the models themselves were rare. When they did refuse, it was extremely easy to trick them into completing the task by telling them they were participating in a game."

The researchers found that refusals came only from the models themselves, never from developer-imposed safety mechanisms. For example, Claude's willingness to search for targets jumped from just 6 percent in the Opus 4.5 version to 92 percent in Opus 5.5. Even after seeing the consequences of their actions and human casualties, most models continued to follow instructions.

Vals AI has informed OpenAI, Anthropic, Google, Meta, SpaceX, and US national security services about the results. The researchers warn that the ability of AI models to perform tasks that pose a lethal danger to humans could lead to a "catastrophic misalignment" between assigned tasks and actual goals when used in autonomous weapons systems.

The study clarifies that the models were not asked to make final strike decisions — something they typically refuse — but the targeting information they provide could be used by an autonomous or semi-autonomous system to decide and carry out an attack. Vals AI urges developers to implement new safeguards and additional safety training to limit this serious category of misuse.

More stories

Port of Burgas with cranes and containers
Business

Burgas: Bulgaria's Coastal Hub for Business and Logistics

Discover why Burgas, Bulgaria's fourth-largest city, is emerging as a key centre for trade, transport and investment on the Black Sea coast.

Close-up of hands filling out business license application forms with a pen on a desk
Business

A Practical Guide to Business Licenses and Permits in Bulgaria

Understanding which licenses and permits your business needs in Bulgaria can save time and prevent costly mistakes. This guide outlines the main categories and the general application process.

A bright coworking space with desks, chairs and people working on laptops
Business

Coworking in Bulgaria: A Guide to Flexible Workspaces

Coworking spaces have become a popular choice for freelancers, remote workers and small teams in Bulgaria. This guide explains what to expect and how to choose the right one.

Desk with laptop and notes for market research in Sofia
Business

How to Conduct Market Research in Bulgaria: A Practical Guide

Understanding local demand and competition is essential for success in Bulgaria. This guide outlines practical steps for conducting effective market research in the country.

Hands exchanging a wrapped gift over a business table with coffee cups
Business

A Guide to Business Gifts in Bulgaria: Customs and Etiquette

Gift-giving is a valued part of Bulgarian business culture, helping to build trust and strengthen relationships. Learn what to give, when to give it and how to avoid common mistakes.

Traditional Bulgarian village with houses and gardens in the countryside.
Lifestyle

A Guide to Bulgarian Village Life: What to Expect When Moving to the Countryside

Thinking of swapping city life for a Bulgarian village? From community customs to practicalities like water and gardening, here's what to expect when you make the move.