Government experts are meeting in Geneva, Switzerland, to work towards a possible negotiating mandate on autonomous weapons systems (AWS). These are military systems that can select and engage targets without human intervention.

This major international gathering follows a recent call by the UN and the International Committee of the Red Cross (ICRC) for urgent controls on these weapons, sometimes dubbed “killer robots” in descriptions by the media.

Recent events in Ukraine have further consolidated concerns that a slide towards weapons with full autonomy (and without meaningful human control) is well underway. According to the New York Times, a Russian drone attacked a petrol station in Zaporizhzhia on July 6, killing three civilians. What marked it out was the drone’s use of AI to decide where and when to strike.

The drone aircraft had been programmed to fly towards the petrol station. But on its approach, an AI system autonomously chose the precise target – probably propane tanks – based on its ability to recognise them.

On August 25, Mirjana Spoljaric, the president of the ICRC, and António Guterres, the secretary-general of the United Nations, issued their joint appeal to the international community to agree on legally binding rules regulating the development and use of AWS.

New technologies are making it cheaper and easier for nations to put together these weapons. Thus, the UN and ICRC appeal for binding rules could not be more urgent. The international community is ideally placed to start negotiations soon, building on many years of diplomatic talks.

For the past decade, governments have debated how AWS challenge existing international law and what should be done under the auspices of the convention on certain conventional weapons, a UN arms control framework. Since 2017, the group of governmental experts (GGE) on lethal autonomous weapons systems (LAWS) – which is part of the convention on certain conventional weapons – has convened typically twice a year.

The experts are meeting again this week in Geneva. The aim is to agree a governance framework and language for a rolling text that could be the foundation for negotiations. This text provides a baseline for continued discussions in November.

While this process has been rife with political disagreements, most experts agree on the need to retain some level of meaningful human control over the use of autonomous systems.

Moral significance

This demonstrates a shared sense by states participating in the process that human judgment is pivotal for the laws governing warfare. It also shows that the moral significance of warfare underlines the importance of human responsibility.

Human judgment is also embedded in international humanitarian law. Targeting processes require context-specific judgments about the targets themselves, potential harm to civilians from striking the target and other precautions. These tasks cannot be delegated to machines.

AI-enabled systems show how technology already shapes human perceptions of targets and risk. With autonomous weapons, the system shifts from influencing human judgement to exercising the targeting function itself.

Any AI system is only as good as the data it has been trained on, or that it works with. For armed conflicts, this data is difficult to come by and is almost always insufficient for dynamic and variable human contexts. This means many systems are likely to be technically inadequate.

The problem becomes quite visible with rules around assessing proportionality, for example, which is a context specific judgement about anticipated civilian harm versus military advantage.

Militaries may try to quantify targeting decisions in a way that creates the appearance of precision. However, the underlying legal judgment remains qualitative regardless. Technical accuracy, therefore, is not the same as lawfulness or moral consideration.

Only humans can understand the significance of respecting the inherent rights and dignity of other humans. Decisions to end human life must therefore be made qualitatively, by humans, in order to be morally and legally justifiable. Without this provision, and clear guidelines around it, we argue that governments risk running into a legal vacuum and a steep moral abyss.

AWS, especially those that prioritise speed and scale of action though AI, carry a significant risk of enabling an uncontrolled and rapid escalation. Escalation from crisis to war, or from conflict to a higher level of violence, could come about by erroneous indications of attack or by a simple sensor or computer error.

Unpredictable systems could give leaders false impressions of the capabilities of those systems. This could lead to overconfidence or encourage preemptive attacks.

Inevitable outcome?

One argument raised by academic researchers in the context of AWS is that they become necessary because they are inevitable. The risk is that more countries opt for full autonomy as components become cheaper and readily available, exacerbating an arms race.

However, an adversary’s conduct does not release a state from its own obligations under international humanitarian law. Following those rules is also part of what makes military force legitimate, including through civilian protection, precautions and accountability. We might turn the argument around and say: legally binding rules become inevitable, because they are absolutely necessary.

Governments and advocacy groups have worked hard over the past decade to put together a lucid proposal for the regulation of AWS. Some efforts are intended to prohibit those systems that target humans directly, or that are unpredictable and therefore cannot be meaningfully controlled by a human. All other AWS must be strictly regulated.

There are signs of political momentum. In November 2025, 156 states voted in favour of a UN General Assembly resolution that “draws attention to the serious challenges and concerns” that autonomous weapons raise. A legally binding instrument is possible, even if some states resist it.

The time is now. Clear, binding rules would help ensure that increasingly autonomous warfare does not make violence easier to unleash and harder to control.