Quick Buzz Feed
As states race to embed algorithms in systems of reconnaissance, targeting, air defence, and military decision-making, a new layer of risk is taking shape
The danger is immediate: humans make momentous decisions in an AI-shaped environment, often with seconds to accept or reject system proposals. This assumes humans are even making the decisions. Israel’s war on Gaza exemplifies how AI systems like Habsora (The Gospel) and Lavender are used for target identification, military coordination, and accelerated decision-making, processing vast information to generate targets. An investigation revealed that Lavender identified potential militants without a thorough check of the machine's choices or underlying intelligence data. While humans may formally remain 'in the decision loop,' this doesn't guarantee meaningful oversight. If a system classifies a target, assigns a threat level, selects a weapon, and gives an operator only seconds to approve, human judgment becomes a procedural formality. This issue is worsened by 'automation bias,' where humans over-trust computer recommendations under pressure. True human control requires understanding the recommendation's basis, access to independent information, awareness of system limitations, and a real chance to challenge conclusions or abort strikes. Without these safeguards, the human decision-maker risks becoming a mere legal facade for an algorithm's choice.
Traditional arms control relies on observable, measurable assets like tanks or missiles. AI, however, poses a complex challenge as it is software that can be rapidly copied, modified, and transferred, often serving both civilian and military purposes. For instance, an algorithm for crop image analysis could be repurposed for battlefield imagery, or an object-recognition tool could become part of a target-selection system. AI complicates post-conflict political settlements by making investigations into violations and accountability for orders significantly harder. An algorithm cannot be prosecuted or compensate victims; while states, commanders, and operators remain legally responsible, the chain of responsibility is long and distributed across designers, data providers, integrators, forces, commanders, and operators. Without digital records of data input, model versions, confidence scores, warnings, and human orders, investigators face immense difficulty reconstructing decisions made in fractions of a second within opaque systems.
International humanitarian law applies universally to all weapons, including AI-powered systems. Armed forces are still obligated to differentiate between civilians and combatants, prevent excessive harm, and take all feasible measures to safeguard civilian lives. The challenge lies in applying these principles as decisions speed up and systems grow more intricate. While the law mandates a commander to verify a target, it doesn't specify the time allotted, the information required, the depth of understanding needed for the system's operation, or the level of uncertainty that renders its use unacceptable. The International Committee of the Red Cross (ICRC) distinguishes between the applicability and the sufficiency of current law, advocating for a ban on systems with unpredictable effects and on weapons designed or used for autonomous targeting of human beings.
The promise of military superiority through technology drives nations to acquire new capabilities, and AI intensifies this dynamic by offering faster, cheaper target acquisition and attack. Even aware of autonomy's dangers, states may prioritize developing AI for fear of adversaries gaining an advantage. This military competition can lead to premature deployment, eroded safety standards, and concealed defects or failures. International governance is crucial to prevent unlawful use and alleviate competitive pressures by establishing shared 'red lines' that make restraint strategically viable. The rhetoric surrounding global AI dominance, exemplified by US Undersecretary of War Emil Michael's declaration of AI as 'America's next Manifest Destiny' and the 'no prize for second place' mentality, highlights this rapid race. However, negotiations are slow-moving compared to the technology's pace. While most states agree that international law applies and human responsibility must be maintained, disagreement persists on the definition of meaningful human control and whether rules should be legally binding or voluntary.
AI presents a unique regulatory challenge as it's not a substance to prohibit or a stockpile to count; it's dual-use software often developed in civilian sectors. Therefore, effective treaties are more likely to regulate specific functions rather than the technology itself. Such regulations could prohibit systems from autonomously selecting and attacking humans without meaningful human judgment, ban systems with unreliable or unpredictable behavior, and exclude AI from nuclear weapon launch decisions. Technology companies, which provide essential models, chips, cloud infrastructure, and computer-vision systems for military applications, play a direct role in international security. They must be integrated into accountability frameworks through contracts, export controls, safety standards, and independent audits.
Governance should not be a sweeping rejection of technology, as AI offers benefits for peace. It can enhance civilian protection, analyze destruction, detect mines, intercept missiles, select less harmful weapons, or abort strikes when civilians are present. There's a critical distinction between automating a specific task in a controlled setting and allowing a system to determine if a human should be targeted. An algorithm identifying a combatant based on phone data or movement patterns differs fundamentally from a system intercepting an object in a defined zone. Paradoxically, AI is rapidly integrated into warfare, while its use in conflict prevention and peace-building remains limited. Algorithms can analyze indicators like price hikes, displacement, inflammatory rhetoric, and local violence to provide early warnings of conflict escalation. They can also aid mediation, document violations, assess war consequences, and direct humanitarian assistance. Governance aims to establish rules to prevent AI from becoming a permanent source of instability, not to create a global authority to review code or restrict legitimate technology use. These measures may not end war but can prevent algorithms from driving escalation without oversight or accountability.