From August 31 to September 4, the United Nations (UN) office in Geneva hosted a meeting on the use of lethal autonomous weapons systems (LAWS). The contents of the meeting were unknown to the public until September 26, when the Washington Post released an exclusive detailing the closed-door negotiations. The newspaper revealed that the meeting collapsed, failing to produce a binding document, due to objections from the US and Russian delegations, whose extensive legal teams spent fifteen hours removing various provisions that would have limited the use of artificial intelligence in weapons. The Washington Post, citing three anonymous sources, disclosed to the public that the initial requirements for systems to be “predictable” and “reliable” were removed from the document. Additionally, they eliminated the requirement for humans to review military targets created by AI prior to striking. Despite the objection from these two powerful nations, the meeting left the door open for further discussion in November, which could yet lead to a legally binding agreement.
A few weeks after the meeting’s tense end, in a speech at the UN General Assembly, President Trump said the US “totally rejects any attempt to construct a globalist scheme to control for the artificial intelligence being spoken of so much now.” He then said that the AI would be renamed to “super intelligence.” This comment comes in the wake of an essay released by Dario Amodei, the co-founder of Anthropic. Dr. Amodei wrote that because of the high risk of the technology, “we must slow the pace at which we improve the capabilities of AI models.” A New York Times article noted that some argue his plea to slow development stems from a desire among leading AI companies to create AI regulations that would prevent smaller firms from competing in the market. Nonetheless, the essay has brought significant attention to the issue, with Trump, not long after, saying he would not “stifle growth of something that will be bigger than the industrial revolution.” To put it mildly, AI is presently an extremely contentious topic on the world stage.
It is clear the US and Russian governments have resisted international restrictions on autonomous weapons. But does existing international law already regulate the use of AI in warfare?
Where Does International Law Stand?
On October 10, 1980, the UN adopted the Convention on Certain Conventional Weapons (CCW). It entered into force three years later. The recent negotiations defined LAWS as “a combination of one or more weapons and technological components that, operating together, can identify, select, and engage a target with force, without intervention by a human being operating the system.” In other words, it is a killer robot.
Under existing international humanitarian law, the use of AI in warfare is governed by two main principles: “Distinction” requires that military actors distinguish between civilians and military objects, while “proportionality” says, “the use of force must never exceed what is deemed necessary to achieve the legitimate military objective.” People must calculate the expected harm of their use of force using the principles of distinction and proportionality.
However, while AI is regulated when it causes expected harm, a legal gap is created when these systems cause unexpected harm. If LAWS were to malfunction unpredictably, resulting in civilian casualties, there would be no person to hold accountable. Essentially, it is impossible to blame someone for a mistake made by AI if they had no way of knowing it would happen. International Humanitarian Law (IHL) does not have a way to establish accountability for most unintentional civilian harm in situations of armed conflict. As AI continues to evolve, it has the potential to develop on its own through a process called “recursive self-improvement.” This could result in the technology making decisions that affect lives in ways humans could not predict. Under current international law, there would be no one to blame for the impacts of AI if it makes decisions completely on its own. It is for this reason that the regulation of LAWS could have a significant impact and why the removal of the “predictable and reliable” provisions from the talks in Geneva poses such a great threat. Without the need to be predictable and reliable, it will be legal for LAWS to cause unexpected harm, for which no one can be blamed. Additionally, because AI technology is still so new, there is little jurisprudence. With few previous cases to rely on, even where IHL standards do exist, it can be difficult to know how to apply them.
What’s Next?
Even if a binding agreement is reached regarding the use of LAWS, a legal gap will remain for non-lethal AI systems. This past spring, a US analyst used AI to assess a Chinese ship and subsequently create a standard report based on that assessment. The AI-generated report stated that the vessel contained components for a nuclear weapons program, leading the military to plan an interception. If the interception had been completed, the AI’s inaccurate assessment would not necessarily be regulated under IHL. Because this report was not a lethal autonomous weapon system, meaning it did not have the capability to kill someone directly, it would also not be regulated under any international agreement specifically targeting LAWS.
The risks to AI-powered military operations, especially in the form of LAWS, are severe. So severe that one Anthropic researcher claimed there is a more than 10% chance the technology could kill all humans within 10 years. The mere suggestion of this is chilling, to say the least.
During the Cold War, as the US and USSR competed to see who could stockpile more arms, they knew the risks of nuclear weapons. The potential aftermath of AI use in the military, despite the warnings, remains murky. And, while many states have pledged to be nuclear-free, some states still possess nuclear weapons. The U.S.’s foreign policy operates under the assumption that possessing nuclear weapons will prevent anyone from using them because their use would result in “mutual assured destruction.” Just as the US has with nuclear weaponry, AI companies are racing one another for the greatest AI developments. It is hard to say whether or how states will choose to regulate AI. They have not even wholly regulated nuclear weapons, despite clear knowledge of nuclear risks. Perhaps states will choose to build AI technology in hopes of deterring its use. But unlike launching a nuclear attack, it may not be clear when the line has been crossed with AI. Even if legally binding international legislation is created surrounding LAWS, there will still exist a gap in governing other types of AI systems. The situation is precarious, and whether international law can close the legal gaps may depend on only two key players: the U.S. and Russia.
Edited by Gita Kerwin.
This is an article written by a Staff Writer. Catalyst is a student-led platform that fosters engagement with global issues from a learning perspective. The opinions expressed above do not necessarily reflect the views of the publication.
Kendall Fagan is a third-year student at McGill University currently pursuing a BA Honours in International Development and a minor in Economics.