ACT3014012015ENGLISH

Download as pdf or txt
Download as pdf or txt
You are on page 1of 29

AUTONOMOUS

WEAPONS
SYSTEMS:
FIVE KEY HUMAN RIGHTS
ISSUES FOR
CONSIDERATION
Amnesty International Publications

First published in 2015 by


Amnesty International Publications
International Secretariat
Peter Benenson House
1 Easton Street
London WC1X 0DW
United Kingdom
www.amnesty.org

© Amnesty International Publications 2015

Index: ACT 30/1401/2015


Original Language: English
Printed by Amnesty International, International Secretariat, United Kingdom

All rights reserved. This publication is copyright, but may be reproduced by any
method without fee for advocacy, campaigning and teaching purposes, but not
for resale. The copyright holders request that all such use be registered with
them for impact assessment purposes. For copying in any other circumstances,
or for reuse in other publications, or for translation or adaptation, prior written
permission must be obtained from the publishers, and a fee may be payable.
To request permission, or for any other inquiries, please contact
copyright@amnesty.org

Amnesty International is a global movement of more than


3 million supporters, members and activists in more than 150
countries and territories who campaign to end grave abuses of
human rights.

Our vision is for every person to enjoy all the rights enshrined
in the Universal Declaration of Human Rights and other
international human rights standards.

We are independent of any government, political ideology,


economic interest or religion and are funded mainly by our
membership and public donations.
CONTENTS
introduction .................................................................................................................................... 5

1. the scope of the convention on certain conventional weapons does not cover law enforcement ...7

2. aws will not be able to comply with relevant international human rights law and policing
standards ........................................................................................................................................ 9

international standards governing the use of force ..................................................................... 11

Less-lethal AWS .........................................................................................................................12

lethal AWS.................................................................................................................................. 13

AWS and other human rights obligations ................................................................................14

3. Developments in existing semi-autonomous weapons technology pose fundamental challenges


for the IHRL framework .................................................................................................................. 15

states’ duty to review weapons for law enforcement ..................................................................21

ethical considerations ................................................................................................................ 22

5. aws erode accountability mechanisms ....................................................................................... 24

Conclusion..................................................................................................................................... 26

Recommendations .....................................................................................................................27
Autonomous Weapons Systems: 5
Five Key Human Rights Issues for Consideration

INTRODUCTION
Over the past decade, there have been extensive advances in artificial intelligence and other
technologies. These will make possible the development and deployment of fully autonomous
weapons systems which, once activated, can select, attack, kill and wound human targets, and will
be able to operate without effective human control. These weapons systems are often referred to
as Lethal Autonomous Robotics (LARs), Lethal Autonomous Weapons Systems (LAWS) and, more
comprehensively, Autonomous Weapons Systems (AWS). The rapid development of these
weapons systems could not only change the entire nature of warfare, it could also dramatically alter
the conduct of law enforcement operations and raises extremely serious human rights concerns,
undermining the right to life, the prohibition of torture and other ill-treatment, and the right to
security of person, and other human rights.

Amnesty International has taken the view that AWS is a useful term for these weapons systems,
since these systems can (i) be designed to have lethal or less lethal effects and (ii) be used in armed
conflict and/or law enforcement situations. With proliferation they are likely to come to be used by
non-state armed groups, criminal gangs and private companies and individuals. Amnesty
International takes the term ‘autonomous’ to mean weapons capable of selecting targets and
triggering an attack without effective or meaningful human control 1 that can ensure the lawful use
of force. Such systems would use violence (including less-lethal force) against individuals, and could
have adverse consequences for a person’s human rights.

While the development of AWS clearly raises serious and legitimate ethical and societal concerns,
this briefing paper will examine the implications of AWS in the context of international law,
particularly international human rights law and standards. The important concerns around their use
in situations of armed conflict, and thus their ability to comply fully with international humanitarian
law (IHL), has been the focus of previous work on AWS, including by Human Rights Watch, other
members of the Campaign to Stop Killer Robots and the International Committee of the Red Cross
(ICRC). This briefing paper, however, will address some of the implications for human rights related
to AWS, particularly those rights and standards that govern the conduct of law enforcement
operations. Amnesty International believes that the questions surrounding the development and
potential use of AWS outside armed conflict (and the ability of such systems to comply with human
rights law) are at least as daunting as those related to their use on the battlefield and urgently

1There is no agreed or legal definition for the term ‘meaningful human control’, which was a term coined by
NGO Article 36. See ‘Article 36 briefing to UN Secretary General’s Advisory Board on Disarmament for the
Campaign to Stop Killer Robots’ at www.article36.org/statements/article-36-briefing-to-un-secretary-generals-
advisory-board-on-disarmament-for-the-campaign-to-stop-killer-robots/, last accessed on 10 March 2015.
Several definitions of this term have recently been explored in by UNIDIR in ‘The Weaponization of Increasingly
Autonomous Technologies: Considering how Meaningful Human Control might move the discussion forward’ at
www.unidir.org/files/publications/pdfs/considering-how-meaningful-human-control-might-move-the-
discussion-forward-en-615.pdf, last accessed on 10 March 2015.

Index: ACT 30/1401/2015 Amnesty International April 2015


6 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

require attention and consideration2, ultimately leading to concrete steps that will address this
important area of international law.

Amnesty International has identified five key human rights issues for consideration in the current
debate on AWS: 1) The scope of the Convention on Certain Conventional Weapons (CCW) does not
cover non-conflict situations; 2) AWS will not be able to comply with relevant international human
rights law (IHRL) and policing standards; 3) Developments in existing semi-autonomous weapons
technology pose fundamental challenges for the IHRL framework; 4) In the absence of a
prohibition, AWS must be subject to independent weapons reviews; and 5) AWS will erode
accountability mechanisms. The issues identified are by no means exhaustive, but rather seek to
elucidate the principal concerns around the potential use of AWS in law enforcement operations.

This briefing argues that the use of AWS, including less-lethal robotic weapons, in law enforcement
operations would be fundamentally incompatible with international human rights law, and would
lead to unlawful killings, injuries and other violations of human rights. Furthermore, the use of AWS
would pose serious challenges in holding accountable those responsible for serious violations and
could entrench impunity for crimes under international law. Consequently, Amnesty International
supports the call for a pre-emptive ban on the development, transfer, deployment and use of AWS,
including fully autonomous systems that deploy less-lethal weapons and can result in death or
serious injury. In the absence of a prohibition, Amnesty International supports the call of UN Special
Rapporteur on extrajudicial, summary or arbitrary executions, Christof Heyns, to impose a
moratorium on the development, transfer, deployment and use of AWS and ensure that
moratorium covers both lethal and less-lethal weapons.

Amnesty International believes it is crucial that the applicability of IHRL be effectively addressed in
current and future discussions on AWS, and proposes mechanisms to facilitate this below.

2AI recognizes the extremely valuable contributions of Special Rapporteur Christof Heyns (Report of the Special
Rapporteur on extrajudicial, summary or arbitrary executions, May 2013:
www.ohchr.org/Documents/HRBodies/HRCouncil/RegularSession/Session23/A-HRC-23-47_en.pdf, last
accessed on 8 April 2015), Human Rights Watch (‘Shaking the Foundations: The Human Rights Implications of
Killer Robots’, May 2014: www.hrw.org/reports/2014/05/12/shaking-foundations, last accessed on 13 March
2015) and the Geneva Academy (Academy Briefing No.8: Autonomous Weapon Systems under International
Law, November 2014: www.geneva-
academy.ch/docs/publications/Briefings%20and%20In%20breifs/Autonomous%20Weapon%20Systems%20un
der%20International%20Law_Academy%20Briefing%20No%208.pdf, last accessed on 8 April 2015), amongst
others, towards this issue.

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 7
Five Key Human Rights Issues for Consideration

1. THE SCOPE OF THE CONVENTION


ON CERTAIN CONVENTIONAL
WEAPONS DOES NOT COVER LAW
ENFORCEMENT
On 15 November 2013, states participating in the Convention on Certain Conventional Weapons
(CCW) annual meeting of high contracting parties at the United Nations in Geneva adopted a report
that included agreement on a proposal by Ambassador Jean-Hugues Simon-Michel of France to
begin discussions on “lethal autonomous weapons systems” (LAWS). The mandate, contained in
Paragraph 32 of the CCW report, stated that:

The Meeting declared that the Chairperson will convene in 2014 a four-
day informal Meeting of Experts, from 13 to 16 May 2014, to discuss the
questions related to emerging technologies in the area of lethal
autonomous weapons systems, in the context of the objectives and
purposes of the Convention. He will, under his own responsibility, submit
a report to the 2014 Meeting of High Contracting Parties to the
Convention, objectively reflecting the discussions held.

The CCW mandate represented a significant development for the Campaign to Stop Killer Robots,
of which Amnesty International is a member, as it marked the launching of an international process
to discuss various aspects of these weapons systems, only seven months following the launch of the
campaign. In November 2014 the CCW Meeting of High Contracting Parties agreed on a new
mandate on AWS, convening a five-day informal experts’ meeting from 13-17 April 2015 to further
discuss “the questions related to emerging technologies in the area of lethal autonomous weapons
systems, in the context of the objectives and purposes of the Convention.” 3

Having AWS on the agenda at the CCW is an extremely positive and important step, and will allow
states, experts and members of civil society to examine different aspects of AWS, including
technical, legal, military, operational and ethical considerations.

However, the scope of the CCW covers only weapons of warfare and situations of armed conflict.

3 Letter by the Chair, Ambassador Michael Biontino of Germany, addressed to the States Parties and
Signatories:
www.unog.ch/80256EDD006B8954/(httpAssets)/C2623D2C5672D441C1257DC6005FA2C6/$file/LAWS+letter.p
df, last accessed on 10 March 2015.

Index: ACT 30/1401/2015 Amnesty International April 2015


8 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

Article 1.2 of the CCW states that:

...This Convention and its annexed Protocols shall not (emphasis


added) apply to situations of internal disturbances and tensions, such
as riots, isolated and sporadic acts of violence, and other acts of a
similar nature, as not being armed conflicts.

This restriction of the scope of the CCW was decided in December 2001 when states parties
conducted a review of the application of the Treaty in Article 1, and while they agreed to apply the
CCW and its protocols to both international and non-international armed conflict, they did not
agree to apply the CCW to situations of internal disturbances and tensions. Therefore, the CCW’s
scope of application as articulated is clearly restricted and excludes many real life and death
situations where weapons are used.

Thus Amnesty International believes that the establishment of the CCW mandate should not
prevent work elsewhere. Indeed, in May 2013, UN Special Rapporteur on extrajudicial, summary or
arbitrary executions, Christof Heyns, submitted a comprehensive report to the UN Human Rights
Council, raising concerns around various aspects of this technology. In his report, Heyns called on
states to “declare and implement national moratoria on at least the testing, production, assembly,
transfer, acquisition, deployment and use of LARs until such time as an internationally agreed upon
framework on the future of LARs has been established”4, as well as an independent panel
comprising of experts from different fields to examine the issue more closely. Further reports by
Heyns in April 20145 and August 20146 recognized the significant implications AWS would have on
international human rights law, particularly the rights to life and dignity. Heyns called on the
Human Rights Council to “remain seized with the issue of autonomous weapons systems, in
particular, as far as the rights to life and dignity are concerned”7. Heyns also urged the international
community, particularly relevant UN bodies, to adopt a comprehensive and coherent approach to
AWS in armed conflict and in law enforcement, which covers both the international humanitarian
law and human rights dimensions, and the deployment of lethal and less-lethal autonomous
weapons.

During the CCW Experts Meeting on Lethal Autonomous Weapons Systems in May 2014 and the
CCW Meeting of High Contracting Parties in November 2013, several states parties made
statements acknowledging the importance of international human rights law in the discussion on
AWS, including Croatia, Egypt, the Holy See, Mexico, Sierra Leone, and South Africa. Others stated

4Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, May 2013:
www.ohchr.org/Documents/HRBodies/HRCouncil/RegularSession/Session23/A-HRC-23-47_en.pdf, last
accessed on 4 March 2015.
5 Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, April 2014 www.daccess-

dds-ny.un.org/doc/UNDOC/GEN/G14/128/20/PDF/G1412820.pdf?OpenElement, last accessed on 4 March 2015.


6Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, August 2014:
www.daccess-dds-ny.un.org/doc/UNDOC/GEN/N14/497/36/PDF/N1449736.pdf?OpenElement, last accessed on
4 March 2015.
7 Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, April 2014, p.23.

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 9
Five Key Human Rights Issues for Consideration

that the CCW process should not prevent other UN bodies such as the Human Rights Council from
taking action on the issue in accordance with their mandates.

Despite this, the general view of many states was that the AWS under discussion were only military
weapons, and failed to officially recognize that in many instances weapons of warfare are used
outside of armed conflict for supposed law enforcement operations, or even by groups involved in
common crime, that often soldiers are tasked with carrying out law enforcement operations, and
that even in situations of armed conflict international human rights law continues to apply
alongside states’ IHL obligations, hence the practical difficulty of confining this issue only to an IHL
framework.

It is thus imperative that due consideration be given to the human rights implications of AWS.
These must be effectively and actively addressed as soon as possible by relevant UN and other
relevant fora and mechanisms, including continuing consideration within the CCW and the Human
Rights Council.

States should also consider establishing an informal working group that sits within the auspices of
the CCW which specifically examines the human rights implications of AWS. This would ensure that
current discussions focused on defining key issues such as ‘autonomous’ and ‘meaningful human
control’ could continue in the CCW, as well as simultaneous consideration of the human rights
implications of AWS and closer coordination between human rights experts and arms control
experts.

2. AWS WILL NOT BE ABLE TO


COMPLY WITH RELEVANT

Index: ACT 30/1401/2015 Amnesty International April 2015


10 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

INTERNATIONAL HUMAN RIGHTS


LAW AND POLICING STANDARDS
The development, deployment and use of AWS raise serious human rights concerns, threatening
the right to life, the prohibition of torture and other cruel, inhuman or degrading treatment or
punishment and the right to security of person, and potentially undermining other human rights.

It is a fundamental rule of international human rights law that no-one may be arbitrarily deprived of
his or her life. The Universal Declaration of Human Rights (UDHR, Article 3) upholds the right of
everyone “to life, liberty and security of person.”8 Article 6(1) of the International Covenant on Civil
and Political Rights (ICCPR), for instance, provides as follows: “Every human being has the inherent
right to life. This right shall be protected by law. No one shall be arbitrarily deprived of his life.” This
is a provision of international human rights law that can never be suspended or otherwise
derogated from even "in time of public emergency which threatens the life of the nation". 9 Indeed
even in situations of full-blown armed conflict, the right not to be arbitrarily deprived of one’s life
continues to apply, though in zones of armed conflict what is “arbitrary” generally falls to be
determined by the rules of international humanitarian law. 10

The right to liberty and security of the person is protected in Article 9 of the ICCPR. This means that
a person cannot be unlawfully or arbitrarily deprived of his or her liberty, and arbitrary arrest or
detention is prohibited. As regards the right to security of person, the Human Rights Committee
recently explained, it “protects individuals against intentional infliction of bodily or mental injury,
regardless of whether the victim is detained or non-detained. For example, officials of States
parties violate the right to personal security when they unjustifiably inflict bodily injury.” They add
that states “should also prevent and redress unjustifiable use of force in law enforcement, and
protect their populations against abuses by private security forces, and against the risks posed by

8 Universal Declaration of Human Rights (UDHR), adopted on 10 December 1948, 217A (III), U.N. Doc. A/810 at
71(1948): www.un.org/en/documents/udhr/, last accessed on 4 March 2015.
9 See article 4(2) of the ICCPR; Human Rights Committee, General Comment no 29 on States of Emergency (31

August 2001), paragraph 7; Human Rights Committee, General Comment no 6 on the Right to Life (30 April
1982), paragraphs 1 to 3.
10 See International Court of Justice, Legality of the Threat or Use of Nuclear Weapons, Advisory Opinion of 8 July

1996, ICJ Reports 1996, paragraph 25; Legal Consequences of the Construction of a Wall in the Occupied
Palestinian Territory, Advisory Opinion of 9 July 2004, ICJ Reports 2004, paragraph 106; Armed Activities on the
Territory of the Congo (Democratic Republic of the Congo v. Uganda), ICJ Reports 2005, paragraphs 216-20, 345(3);
Human Rights Committee, General Comment no 31 on the nature of the general legal obligation imposed on
States parties to the Covenant UN Doc CCPR/C/21/Rev.1/Add.13 (2004), paragraph 11. See also Reports of the
UN Special Rapporteur on extrajudicial, summary or arbitrary executions: UN Doc E/CN.4/2005/7 (22 December
2004), paragraphs 41-54, 77-79, 84 and 86; UN Doc A/HRC/4/20 (29 January 2007), paragraph 19; UN Doc
A/HRC/4/20/Add.1 (12 March 2007), pp. 342-363; UN Doc A/62/265 (16 August 2007), paragraphs 27- 32; UN Doc
A/HRC/14/24/Add.6 (28 May 2010) [‘Study on targeted killings’], paragraphs 28-36.

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 11
Five Key Human Rights Issues for Consideration

excessive availability of firearms.”11

There is also a danger that AWS would be used to violate the prohibition of torture and other cruel,
inhuman, or degrading treatment or punishment. Like the prohibition of arbitrary deprivation of
life, torture is prohibited in all circumstances, including in armed conflict, and can never be
derogated from. This prohibition is a peremptory norm of international law, which is legally binding
on all states regardless of which treaties they have ratified.

INTERNATIONAL STANDARDS GOVERNING THE USE OF FORCE


The international community has elaborated standards to help guide states in ensuring human
rights compliant use of force in law enforcement, in particular with due attention to the protection
of the rights to life and to security of person, and the prevention of torture and other ill-treatment,
such as UN Code of Conduct for Law Enforcement Officials (UNCCLEO, 1979) and the UN Basic
Principles on the Use of Force and Firearms by Law Enforcement Officials (UNBPUFF, 1990), as well
as guidelines for international and domestic law for citizens held in prisons and other forms of
custody, such as the Standard Minimum Rules for the Treatment of Prisoners (SMRTP). It is
virtually inconceivable that AWS could be used in a manner that complies with these standards.
UNBPUFF’s core provisions on the use of force are an elaboration of legal rules binding on states by
way of treaty obligations or obligations under customary international law. The process of its
development and adoption involved a very large number of states. At least, the substance of Article
3 of the UNCCLEO and Principle 9 of the UNBPUFF reflect binding international law.12

The UNCCLEO establishes the overall principle that “Law enforcement officials may use force only
when strictly necessary and to the extent required for the performance of their duty” (Article 3).
That means force of any sort may only be lawfully used when no other means are likely to achieve
the legitimate objective. It also should be clear that no greater force should be used than what is
necessary to achieve the objective. To be lawfully used in policing AWS would have to be able to
make this determination and act accordingly.

Any use of force must have a sufficient legal basis that is in line with international standards. In
particular, it must serve a legitimate objective established by law. Secondly, the use of any force by
police should be strictly limited to those situations where it is absolutely necessary for the
achievement of a legitimate law enforcement aim. If the use of force is unavoidable, police and law
enforcement officers must always exercise restraint in its use.

Thirdly, the level of any force used must also be strictly proportional to the law enforcement
objective, which sets a ceiling on the level of force that may be used for a particular law
enforcement objective. In any use of force the police must at all times respect human rights,
including the right to life and the prohibition of torture and other ill-treatment. Therefore they

11 Human Rights Committee, General Comment no 35 on liberty and security of person, UN Doc CCPR/C/GC/35
(2014), paragraph 9.
12See UN Special Rapporteur on extrajudicial, summary or arbitrary executions, UN Doc A/61/311, paragraph 35;
and Nigel Rodley, The Treatment of Prisoners under International Law, 3 rd Edition, pp.257-258.

Index: ACT 30/1401/2015 Amnesty International April 2015


12 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

must always take steps to minimize the risk of injury and death. Finally, the importance of the
duties carried out by law enforcement officers and the large powers granted to them make it
paramount that law enforcement agencies are held accountable for the fulfilment of their duties
and for their compliance with the law. This comprises not only the individual law enforcement
official to be held accountable for their actions and omissions, but also all superiors who order,
supervise or otherwise have law enforcement officials under their command and control, as well as
the agency as a whole. (This is expanded upon in key consideration number 5 ‘AWS erode
accountability mechanisms’).

LESS-LETHAL AWS
In the exercise of their duty, police and law enforcement officers must apply non-violent means
before resorting to the use of force, which may be used only if non-violent means have proven to
be, or are likely not to be, effective. As Principle 4 of the UNBPUFF states:

Law enforcement officials, in carrying out their duty, shall, as far as


possible, apply non-violent means before resorting to the use of
force and firearms. They may use force and firearms only if other
means remain ineffective or without any promise of achieving the
intended result.

On an operational level, this principle requires law enforcement officials to proactively seek to
resolve any situation through other means than the use of force, such as the means of persuasion,
negotiation and de-escalation. These techniques require human empathy, negotiating skills, a high
level of training and an ability to assess and respond to often dynamic and unpredictable situations,
and it would not be possible for a robot to be programmed to perform these duties in a manner that
respects international standards. As Special Rapporteur Christof Heyns has argued, while robots
may be effective at dealing with quantitative issues, they have a limited capacity to make the
qualitative assessments that are required when dealing with human life. As Heyns states in his
report, these assessments:

…often require human judgement, common sense, appreciation of


the larger picture, understanding of the intentions behind people’s
actions, and understanding of values and anticipation of the
direction in which events are unfolding. Decisions over life and
death in armed conflict may require compassion and intuition.
Humans – while they are fallible – at least might possess these
qualities, whereas robots definitely do not.13

13
Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, May 2013, pp.10-11:
www.ohchr.org/Documents/HRBodies/HRCouncil/RegularSession/Session23/A-HRC-23-47_en.pdf, last
accessed on 4 March 2015.

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 13
Five Key Human Rights Issues for Consideration

While Heyns refers to armed conflict in the extract above, this observation is, if anything, even
more relevant to law enforcement duties outside of armed conflict.

In addition, under Principle 4 of the UNBPUFF any use of force must be guided by a graduated
response, with a view to minimizing harm. Law enforcement officials should not resort immediately
to the easiest means at their disposal, but must choose – among the available means that are likely
to be efficient – the one that presents the lowest risk of causing harm and injury. On an operational
level, this means that different types of protective equipment and means of communication, of less
lethal equipment and weapons allowing for a graduated response, as well as of sufficient resources
and back up means must be made available to police and law enforcement officers. It also means
being in a position to decide on the appropriate time and place for any law enforcement action with
a view to minimizing risks and damage. Law enforcement officials must also provide assistance and
medical aid as swiftly as possible to those injured or otherwise affected by police use of force, and
relatives or close friends of the injured or affected must be notified. These are incredibly complex
judgements that require thorough and continuous training, as well as an assessment of unique and
ever-evolving situations. Thus it would be very unlikely that an AWS, operating without meaningful
human oversight, would be able to perform such duties.

Also relevant is Principle 20 of the UNBPUFF, which states: “[I]n the training of law enforcement
officials, Governments and law enforcement agencies shall give special attention to issues of police
ethics and human rights, especially in the investigative process, to alternatives to the use of force
and firearms, including the peaceful settlement of conflicts, the understanding of crowd behaviour,
and the methods of persuasion, negotiation and mediation, as well as to technical means, with a
view to limiting the use of force and firearms.” The elements particularly related to ethics, peaceful
settlement of conflicts, understanding of crowd behaviour and method of persuasion, negotiation
and mediation are inherently human skills which cannot be automated or roboticized, especially
given the ever-evolving, dynamic and unpredictable nature of law enforcement operations.

LETHAL AWS
There are situations where it is lawful for police to use firearms and lethal force. This is foreseen in
Principle 9 of the UNBPUFF14, which places an extremely high threshold on their use:

Law enforcement officials shall not use firearms against persons except in
self-defence or defence of others against the imminent threat of death or
serious injury, to prevent the perpetration of a particularly serious crime
involving grave threat to life, to arrest a person presenting such a danger
and resisting their authority, or to prevent his or her escape, and only
when less extreme means are insufficient to achieve these objectives. In

14 The Special Rapporteur on extrajudicial, summary or arbitrary executions has affirmed that: “All uses of
firearms against people should be treated as lethal or potentially lethal.” He has also clarified that “A common
sense understanding of the scope of application of Principle 9 suggests that all weapons that are designed and
are likely to be lethal should be covered, including heavy weapons such as bombs and (drone) missiles, the use
of which constitutes an intentional lethal use of force.” See HRC A/26/36, paragraphs 70-71.

Index: ACT 30/1401/2015 Amnesty International April 2015


14 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

any event, intentional lethal use of firearms may only be made when
strictly unavoidable in order to protect life.

This principle deals with two different thresholds: a) when it is appropriate to use firearms
(potentially lethal force) and b) the even higher threshold of when the intentional lethal use of
firearms is permissible. Each of these situations involves a complex assessment of potential or
imminent threats to life or serious injury and how to respond to them appropriately, and it involves
deciding how best to protect the right to life, which is an absolutely fundamental duty of the state
under human rights law. Such life and death decisions must never be delegated to AWS.

In order to be able to carry out policing and law enforcement operations in a lawful manner, AWS
would need to be able to effectively assess the degree to which there was an imminent threat of
death or serious injury, identify correctly who is posing the threat, consider whether force is
necessary to neutralize the threat, be able to identify and use means other than force, have the
capacity to deploy different modes of communication and policing weapons and equipment to
allow for a graduated response, and have available back up means and resources. To add to this
complexity, each situation would require a different and unique response, which would be
extremely challenging to reduce to a series of complex algorithms.

It is not possible that AWS, without meaningful and effective human control and judgement, would
be able to comply with these provisions, especially in unpredictable and ever-evolving
environments. In an open letter in October 2013, computer scientists, engineers, artificial
intelligence experts, roboticists and professionals from related disciplines from 37 countries
asserted that “in the absence of clear scientific evidence that robot weapons have, or are likely to
have in the foreseeable future, the functionality required for accurate target identification,
situational awareness or decisions regarding the proportional use of force, we question whether
they could meet the strict legal requirements for the use of force” and that “[G]iven the limitations
and unknown future risks of autonomous robot weapons technology…,[D]ecisions about the
application of violent force must not be delegated to machines.”15

AWS AND OTHER HUMAN RIGHTS OBLIGATIONS

AWS could even be used to facilitate violations of the right to freedom of expression and right to
freedom of peaceful assembly. Indeed, as Christof Heyns has stated:

On the domestic front, LARs could be used by States to suppress


domestic enemies and to terrorize the population at large,
suppress demonstrations and fight “wars” against drugs. It has
been said that robots do not question their commanders or
stage coups d’état.16

15
‘Computing experts from 37 countries call for ban on killer robots’, 16 October 2013:
www.icrac.net/2013/10/computing-experts-from-37-countries-call-for-ban-on-killer-robots/ and
www.icrac.net/call/, last accessed on 14 March 2015.
16 Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, p.16:
www.ohchr.org/Documents/HRBodies/HRCouncil/RegularSession/Session23/A-HRC-23-47_en.pdf, last

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 15
Five Key Human Rights Issues for Consideration

Given the potentially grave consequences of such technology and states’ existing obligations under
international human rights law and IHL, Amnesty International believes the onus should be on
states that wish to develop and deploy AWS to first demonstrate that specific uses of each type of
weapon can be fully lawful and, in particular, consistent with international human rights and
humanitarian law in operational circumstances.

3. DEVELOPMENTS IN EXISTING
SEMI-AUTONOMOUS WEAPONS
TECHNOLOGY POSE FUNDAMENTAL
CHALLENGES FOR THE IHRL
FRAMEWORK
Alongside the development of robotic “military weapons”, companies in the US, UK, Jordan, Israel,
the UAE, Spain and likely elsewhere have been developing robotic weapons, even if not yet fully
autonomous, for law enforcement purposes. These weapons are designed to fire toxic chemical
irritants (tear gas), rubber or plastic projectiles (bullets) and electric shock stun darts. Such weapons
can have lethal effects, as well as result in serious injuries. They could also be used to facilitate
violations of the rights to freedom of expression and right to freedom of peaceful assembly, the
right to health and other human rights.

accessed on 14 March 2015.

Index: ACT 30/1401/2015 Amnesty International April 2015


16 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

Furthermore, AWS being developed for use in conflict situations will very likely spill over into the
law enforcement sphere. As Christof Heyns has noted:

…[I]t should be recalled that aeroplanes and drones were first used in
armed conflict for surveillance purposes only, and offensive use was ruled
out because of the anticipated adverse consequences. Subsequent
experience shows that when technology that provides a perceived
advantage over an adversary is available, initial intentions are often cast
aside. Likewise, military technology is easily transferred into the civilian
sphere. If the international legal framework has to be reinforced against
the pressures of the future, this must be done while it is still possible.17

Although these weapons systems are not fully autonomous, relying on direct human control over
targeting and firing decisions, current technological advancements and trends demonstrate that it
is only a matter of time before such weapons systems acquire full autonomy. In addition, there are
already serious concerns as to whether the level of control over semi-autonomous weapons
systems is sufficient and whether there is access to effective remedy for misuse of these weapons.

Below is a sample of some of the existing semi-autonomous weapons systems and their
capabilities.

Country of Weapons System Capabilities


Manufacture

USA Vanguard Defense Industries: This is an unarmed aerial vehicle, (UAV)


ShadowHawk UAV which can be armed with 37mm and 40mm
grenade launchers, or a 12-gauge shotgun
with laser designator. The avionics have
both a semi-autonomous and a fully
autonomous function.18

The manufacturer’s website states that


these systems are not available for law
enforcement, however, past reports indicate
that the ShadowHawk can also be armed
with the Taser XREP and has been sold to
law enforcement personnel in Texas. A news
article from 2011, reports that a
ShadowHawk had been “unveiled by the
ShadowHawk specification sheet, 2011. Obtained from
Montgomery County Sheriff's office and will
Milipol 2009.

17Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, p.6:


www.ohchr.org/Documents/HRBodies/HRCouncil/RegularSession/Session23/A-HRC-23-47_en.pdf, last
accessed on 14 March 2015.
18 Vanguard Defense Industries: www.vanguarddefense.com/specifications/, accessed on 15 March 2015.

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 17
Five Key Human Rights Issues for Consideration

be operational within a month.” It further


stated that “[A]lthough its initial role will be
limited to surveillance, the ShadowHawk
Unmanned Aerial Vehicle, previously used
against suspected terrorists in Afghanistan
and East Africa, and has the ability to tase
suspects from above as well as carrying 12-
gauge shotguns and grenade launchers.”19

Spain TechnoRobot: RiotBot The TechnoRobot website describes RiotBot


as “the first robot in the world designed
specifically for riot control.”20 The company
also states that the robot is designed for “a
wide range of police, military and general
security operations, mainly those in which
the personal safety of the members of the
intervention units is not fully guaranteed or
could be in danger”.

Some of the scenarios that have been


studied for its development include riot
control, civil order, jails and prisons, area
denial, SWAT team operations, police
TechnoRobot Company Brochure, p.1. Obtained from round-ups, boundary defence and
Milipol 2009. intervention, neutralization of suspects and
dissuasive activities.

In addition, images on the website show the


RiotBot armed with Pepperball launchers.21

USA Combined Systems: V-series A high-capacity non-lethal tube launch

19 ‘DHS-Funded Taser Drone Launched in Texas’, 1 November 2011:


www.telepresenceoptions.com/2011/11/dhsfunded_taser_drone_launched/, accessed on 15 March 2015.
20 TechnoRobot: www.technorobot.eu/en/index.htm, accessed on 15 March 2015.

21 TechnoRobot: www.technorobot.eu/en/riotbot_gallery.htm, accessed on 15 March 2015.

Index: ACT 30/1401/2015 Amnesty International April 2015


18 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

munition system with 40MM rifled barrels


scalable in 10, 20 or 30 tube bank
configurations.

The Combined Systems website shows it


mounted on an unmanned ground vehicle.22

©Robin Ballantyne, the V-series device is on the right-


hand side.

Israel G-NIUS: Guardium UGV™ The Guardium is a semi-autonomous


unmanned ground system, which can be
deployed to perform routine missions, such
as programmed patrols along border routes,
but also to autonomously react to
unscheduled events, in line with a set of
guidelines.

The Guardium can carry remote-operated


lethal and less-lethal weapons.
©Robin Ballantyne

USA MetalStorm: FireStorm™ FURY The MetalStorm website states that


FireStorm™ FURY is “a configurable multi-
barrel, multi-caliber crew-served or remotely
operated electronic weapon system that
provides escalation of force and scalable
effect across less lethal and lethal
continuum”. MetalStorm manufactures a
variety of launchers for ground or air. It also
provides munitions: lethal, non-lethal,
marker rounds, single or multi-barreled

22 Combined Systems: www.combinedsystems.com/products/?cid=148, accessed on 15 March 2015.

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 19
Five Key Human Rights Issues for Consideration

MetalStorm Company Brochure, p.2. Obtained from weapons.23


Milipol 2009.
Some of the scenarios that have been
studied for its development include military
operations on urban terrain (MOUT),
reconnaissance patrol, border patrol, critical
infrastructure protection and crowd control.

Jordan Jordan Electronic Logistics Support: The The Lynx Robot is a medium tracked military
Lynx Robot robot designed for military tasks ranging
from reconnaissance to combat. It is
equipped with the following set of sensors:
front drive camera, back drive camera, PTZ
camera, GPS and digital compass.

Smart guidance functions are provided to


support the following modes: squad
architecture, predefined mission and long
range remote guidance. Different weapons
can be placed on the Lynx, such as the M16
rifle, 7.62 machine guns and rocket-
propelled grenades.

Jordan Electronic Logistics Support Brochure, p.1.


Obtained from DSEi 2013.

USA MSI Delivery Systems: AB2K Chemical irritant dispenser which can be
incorporated onto unmanned ground
vehicles and aircraft. This can also be
mounted on walls, and fixed on buildings,
vehicles including small craft, military and
law enforcement boats, unmanned vehicles
including riot control and other armor
piercing vehicles24.

According to its website, the AB2K:

23 MetalStorm: http://www.metalstorm.com/IRM/content/robotic-platforms.html, accessed on 15 March 2015.


24 MSI Delivery Systems: www.msi-deliverysystems.com/ab2k-mmads-variants, accessed on 15 March 2015.

Index: ACT 30/1401/2015 Amnesty International April 2015


20 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

 “can also be incorporated into


drones for deployment and has
been tested for compatibility with
military small craft boats”;
 “can be operated remotely from up
to 4 miles distance”;
 “is capable of dispensing many
less-than-lethal formulations in a
high density aerosol form”,
including training smoke and
standard non-toxic training smoke
mixed with irritants such as OC,
CS, or Pepper.
 can “deny access to rooms,
corridors and perimeters by rapidly
dispensing agent.”

4. IN THE ABSENCE OF A
PROHIBITION, AWS MUST BE
SUBJECT TO INDEPENDENT
WEAPONS REVIEWS
Under Article 36 of 1977 Additional Protocol I to the four Geneva Conventions of 1949 (henceforth,
Article 36), States Parties are required to conduct a review of the legality of a new weapon, means
or method of warfare under international humanitarian law and other applicable international law.
According to Article 36:

In the study, development, acquisition or adoption of a new


weapon, means or method of warfare, a High Contracting Party is
under an obligation to determine whether its employment would,
in some or all circumstances, be prohibited by this Protocol or by
any other rule of international law applicable to the High
Contracting Party.

Weapons reviews are crucial for ensuring that weapons and their use will comply with international

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 21
Five Key Human Rights Issues for Consideration

law, national laws and relevant international and national standards. A growing number of states
have argued during discussions on AWS that Article 36 could provide a mechanism for ensuring that
AWS will comply with IHL, without the need for imposing a prohibition on AWS. While the mention
and engagement with Article 36 is welcome, it is not enough, for several reasons.

Firstly, Article 36 is not clear on how the review of weapons should take place. Indeed, there is a
lack of established state practice in implementing Article 36 and according to the International
Committee of the Red Cross (ICRC), ‘only a few countries are known to have set up formal review
mechanisms for new weapons’.25 Those states that have established formal review mechanisms
have done so on varying levels of detail and following differing standards. There is also often a lack
of transparency in how states conduct weapons reviews, when they conduct them and the results of
those reviews.

Secondly, a weapons review under Article 36 does not apply to all weapons and equipment and
does not examine how they might be used in policing and law enforcement operations. Some
weapons may be lawfully used in armed conflict but not in policing and the converse is also true
(e.g. tear gas and dum-dum bullets cannot be used in armed conflict but may lawfully be used in
some circumstances in police operations). Therefore, some lethal and less-lethal AWS would not be
covered in such weapons reviews. Although Article 36 also requires States Parties to examine the
legality of new weapons, means and methods of warfare under ‘any other rule of international law
applicable to the High Contracting Party’, which would necessarily include a review of compliance
with international human rights law, this would apply only to military operations in situations of
armed conflict. Thus in the absence of a prohibition on AWS, states, civil society organizations,
technical, legal and other experts that are currently examining the issue of AWS must address this
gap as discussions surrounding the applicability of Article 36 continue.

STATES’ DUTY TO REVIEW WEAPONS FOR LAW ENFORCEMENT


The UNBPUFF places a due diligence requirement upon states to review weapons used in law
enforcement. As Principle 3 of the UNBPUFF states, “the development and deployment of non-
lethal incapacitating weapons should be carefully evaluated in order to minimize the risk of
endangering uninvolved persons”. This review is limited to less-lethal weapons but is still important
to ensure that those weapons will comply with relevant international standards and national laws
and, moreover, given that evidence shows that “non-lethal” weapons can often have lethal effects
which is why the term “less-lethal” is more appropriate. The requirement of a review of weapons
used for law enforcement is even more important given the increasing ‘militarization’ of law
enforcement operations, whereby military personnel assume roles often held by law enforcement
agencies, such as policing of public assemblies.

In the absence of a prohibition on AWS, states intending to develop, acquire, or use AWS must
therefore be required to thoroughly review whether they can be used in a manner that fully
respects relevant law and standards be it for law enforcement or military operations. This testing

25
ICRC, ‘Review of New Weapons’, Overview, 29 October 2010: www.icrc.org/eng/war-and-law/weapons/new-
weapons/overview-review-of-new-weapons.htm, accessed on 13 March 2015.

Index: ACT 30/1401/2015 Amnesty International April 2015


22 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

should be carried out by an independent body. The rapid technological advances that are moving
towards full autonomy in weapons systems present serious concerns. The technology to allow fully
autonomous operations may be reached soon; but it is extremely unlikely that programming that
could ensure AWS perform law enforcement functions lawfully would be developed in the
foreseeable future.

Any new law enforcement equipment should be introduced based on clearly defined operational
needs and technical requirements with a view to reduce the amount of force used and the risk and
level of harm and injury caused. They must be subject to rigorous testing, by an independent expert
body, and the testing, review and selection process should be legally constituted. In addition to
assessing compliance with the UNBPUFF themselves, the process must test AWS compatibility
with other key human rights treaties and standards, including ICCPR, International Covenant on
Economic, Social and Cultural Rights (CESCR), the Convention Against Torture, the SMRTP and the
UNCCLEO.

It is unclear what operational needs and technical requirements could only be met by lethal or less-
lethal AWS as opposed to other weapons or equipment that would have an element of effective
and meaningful human control. New law enforcement equipment should be introduced with a view
to reduce the amount of force used and the risk and level of harm and injury caused as well as
possible unwarranted effects. So introducing AWS to carry out duties that require complex
judgements in the context of the inherently dynamic and unpredictable nature of law enforcement
operations, would carry unacceptably high risks and would be unwarranted. As Human Rights
Watch argues, “interpreting more subtle cues whose meaning can vary by context, such as tone of
voice, facial expressions, and body language, requires an understanding of human nature.”26

In addition, the use of any device must be subject to thorough supervision and control mechanisms
with a view to continuously evaluate the device with regards to its efficiency and effects, including
unwarranted effects. Hence effective and meaningful human control would be a necessity for this
to be done effectively.

Even in the extremely unlikely event that all the challenges that have been raised above could be
satisfactorily addressed and resolved, fundamental questions of ethics and of legal accountability
(see key consideration number 5) would remain.

ETHICAL CONSIDERATIONS
Principle 1 of the UNBPUFF also states that “Governments and law enforcement agencies shall
keep the ethical issues associated with the use of force and firearms constantly under review”.
Here, the ethical considerations of weapons used in law enforcement are given a specific
prominence. Quite apart from serious concerns as to whether autonomous technologies will be
technically capable of conforming to existing IHRL and the UNBPUFF, AWS raise numerous
important ethical and social concerns, including lowering the threshold for the use of lethal and
less-lethal force, reducing the cost and risks of launching law enforcement operations with more

26 26
Human Rights Watch, ‘Shaking the Foundations: The Human Rights Implications of Killer Robots’, May
2014, p.20: www.hrw.org/reports/2014/05/12/shaking-foundations, last accessed on 13 March 2015.

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 23
Five Key Human Rights Issues for Consideration

frequency, especially since AWS would not be able to refuse orders, and the delegation of human
decision-making responsibilities to an autonomous system designed to injure and kill. As UN
Special Rapporteur Christof Heyns asserts, “[T]here is widespread concern that allowing [fully
autonomous weapons] to kill people may denigrate the value of life itself.”27 This also links to the
right to dignity, which features in the preamble of the ICCPR and the UDHR, and is recognized in
Article 10 of the ICCPR: “All persons deprived of their liberty shall be treated with humanity and
with respect for the inherent dignity of the human person”.

In addition, rather than providing states with a more effective, risk-averse solution to the conduct
of law enforcement operations, AWS would most likely have the opposite effect. Removing human
beings from some policing functions is likely only to further exacerbate the often tense relations
between police and communities. Given the pervasive nature of discrimination, it is foreseeable
that AWS would be deployed in deprived and crime-ridden neighbourhoods where people have
historically had bad experiences of the police. Such mistrust would not be remedied by introducing
machines to do the job of a human being. Faced with a robot, people will react differently than they
would with a human to which they can relate. People will not be able to negotiate with a fully
autonomous robot, or signal their intention to cease their unlawful acts or surrender. This could
therefore cause escalation and could result in the arbitrary deprivation of life.

This poses further concerns for the development and potential use of lethal and less-lethal AWS in
law enforcement operations and further highlights that the use of AWS would most likely not be
able to comply with international standards governing the use of force.

27UN Human Rights Council, Report of the Special Rapporteur on extrajudicial, summary or arbitrary
executions, Christof Heyns, Lethal Autonomous Robotics, p. 20.

Index: ACT 30/1401/2015 Amnesty International April 2015


24 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

5. AWS ERODE ACCOUNTABILITY


MECHANISMS
The development, deployment and use of AWS raises important issues related to accountability for
human rights violations and individual criminal responsibility. Under the UNBPUFF, the level of
responsibility, duty and powers granted to law enforcement officers requires that they are held
accountable for the fulfilment of their duties and for their compliance with the law. This comprises
not only the individual law enforcement official to be held accountable for their misconduct and
negligence, but also all superiors who order, supervise or otherwise have law enforcement officials
under their command and control, as well as the agency as a whole. Under Principle 22 of the
UNBPUFF:

Governments and law enforcement agencies shall ensure that an effective


review process is available and that independent administrative or
prosecutorial authorities are in a position to exercise jurisdiction in
appropriate circumstances. In cases of death and serious injury or other
grave consequences, a detailed report shall be sent promptly to the
competent authorities responsible for administrative review and judicial
control.

Thus all deaths and serious injuries that occur during the course of a law enforcement operation
must be subject to an obligatory investigation under judicial control, and bring those responsible to
justice. For this to occur, a complete and transparent system needs to be in place to hold law
enforcement officials accountable for their decision to resort to the use of force. This requires the
existence of an external accountability mechanism that is mandated to carry out independent,
impartial and effective investigations. States have an obligation to respect the prohibition of
arbitrary deprivation of life, and an obligation to take all appropriate measures to prevent,
investigate, punish and redress the harm caused by human rights abuses by private persons or
entities. A failure to investigate an alleged violation of the right to life could in and of itself
constitute a breach of the right. As Christof Heyns has stated, “A failure to investigate and, where
applicable, punish those responsible for violations of the right to life in itself constitutes a violation
of that right.”28

28 Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, Christof Heyns, 13
September 2013, p.20: www.un.org/en/ga/search/view_doc.asp?symbol=A/68/382, last accessed on 8 April

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 25
Five Key Human Rights Issues for Consideration

Under international human rights law states have an obligation to investigate allegations of human
rights violations and bring the perpetrators to justice as part of the right to an effective remedy, a
right which is applicable at all times.

In the case of lethal and less-lethal AWS, it is not possible to bring a machine to justice and no
criminal sanctions could be leveled against it. Actors involved in the programming, manufacture
and deployment of AWS, as well as superior officers and political leaders could instead be held
accountable. However, it would be impossible for any of these actors to reasonably foresee how an
AWS will react in any given circumstance, given the countless situations it may face. Furthermore,
without effective human oversight, superior officers would not be in a position to prevent an AWS
from committing unlawful acts, nor would they be able to reprimand it for misconduct.

Another aspect of accountability is the ability of victims to access the right to effective remedy.
States responsible for violating their obligations under international human rights and/or
international humanitarian law are required to provide victims with adequate, effective and prompt
reparation for the harm suffered, which can take the form of restitution, compensation,
rehabilitation, satisfaction and guarantees of non-repetition. As Human Rights Watch has argued,
‘the actions of fully autonomous weapons would likely fall within an accountability gap that would
contravene the right to a remedy’29, given the potential for impunity outlined above. Thus the
obligation to ensure that victims and families of victims of human rights violations by law
enforcement officers receive full reparation, which should include restitution, compensation,
rehabilitation, satisfaction, and guarantees of non-repetition, could not be met.

Furthermore, as contemporary use of drones has shown, investigations into unlawful killings
through drone strikes are seldom conducted. In its report on US drone strikes in Pakistan, 30
Amnesty International examined the prevailing secrecy surrounding US drone strikes in Pakistan, as
well as restrictions on access to drone-affected areas. It also exposed the refusal of the US
administration to explain the international legal basis for individual attacks, raising concerns that
other strikes in Pakistani Tribal Areas may have also violated human rights.

Amnesty International has repeatedly called on the USA to comply with its obligations under
international law to ensure thorough, impartial, and independent investigations are conducted into
the killings documented in its report. No such investigations have been launched. Although drones
do not fall under the definition of a fully autonomous lethal or less-lethal weapons system by virtue
of being operated under human supervision and direct control, the case above serves as an
important lesson to be learned. Even where there has been human supervision in the use of drones,
and thus a direct line of responsibility, investigations have not taken place and perpetrators of
human rights violations have not been brought to justice. The survivors of drone attacks and the
families of the victims have had little or no chance of securing justice.

2015.

29Human Rights Watch, ‘Shaking the Foundations: The Human Rights Implications of Killer Robots’, May 2014,
p.19: www.hrw.org/reports/2014/05/12/shaking-foundations, last accessed on 13 March 2015.
30
Amnesty International, “Will I be next?” US drone strikes in Pakistan’, October 2013:
www.amnestyusa.org/sites/default/files/asa330132013en.pdf, last accessed on 15 March 2015.

Index: ACT 30/1401/2015 Amnesty International April 2015


26 Autonomous Weapons Systems:
Five Key Human Rights Issues for Consideration

As illustrated above there is thus a huge problem of accountability regarding drone strikes, and this
is partly due to the human distance from taking the decision to launch a strike. AWS without
meaningful and effective human control would add an extra layer of distance in both the targeting
and killing decisions, and would present further obstacles to the existing challenges of ensuring
accountability for these killings. One can reasonably conclude the use of AWS without effective
human control would make accountability impossible and render the right to remedy and full
reparation virtually meaningless.

CONCLUSION
If left unchecked, rapid advances in technology, as well existing sophisticated semi-autonomous
lethal and less-lethal weapons systems, make the development and deployment of fully
autonomous weapons systems virtually inevitable, both in situations of armed conflict and law
enforcement operations. The establishment of an international process to examine various aspects
of these weapons systems under the mandate of the CCW is an extremely welcome, important and
valuable initiative. However, the CCW discussions understandably have tended to focus on the
implications of the use of AWS in situations of armed conflict, examining this issue primarily
through an IHL lens. It is absolutely crucial that as states, experts from various fields, and civil
society representatives continue their discussions on this issue, they examine and address the
implications of AWS on human rights law and policing, within the CCW and in other relevant
international fora. This is especially important given existing lethal and less-lethal semi-
autonomous weapons systems that are designed specifically for law enforcement operations, some
of which possess fully autonomous functions (such as the ShadowHawk UAV).

Similarly, the proposal by some states that the Article 36 of the 1977 Additional Protocol I to the
four Geneva Conventions of 1949 could provide a sufficient mechanism for ensuring that AWS will
comply with IHL, would not apply to weapons and equipment used in policing and law enforcement
operations. The UNBPUFF requires states to carefully evaluate law enforcement equipment “in
order to minimize the risk of endangering uninvolved persons”. Thus the use of any device must be
subject to thorough supervision and control mechanisms with a view to continuously evaluate the
device with regards to its efficiency and effects, including unwarranted effects. Meaningful and
effective human control would be a necessity for this to be done effectively.

Furthermore, given the current status of the technology, as well as the unlikelihood that AWS could
ever reach human levels of judgment required in the lawful conduct of law enforcement, it is
improbable that AWS could comply with international standards governing the use of force. It is
particularly doubtful that the guiding human rights principles of legality, necessity and
proportionality could be adhered to by AWS. Lethal and less-lethal AWS without meaningful and
effective human control would not have the capacity to correctly assess complex policing situations
and comply with international standards that prohibit the use of lethal force except in defence
against an imminent threat of death or serious injury (UNBPUFF Principle 9).

Unlike highly trained and strictly accountable law enforcement personnel, robots could not by
themselves distinguish between legal and illegal functions, or make decisions regarding the use of
force, seriously undermining accountability and remedy for arbitrary, abusive and excessive uses of
force.

Amnesty International April 2015 Index: ACT 30/1401/2015


Autonomous Weapons Systems: 27
Five Key Human Rights Issues for Consideration

Given the potentially grave consequences of such technology and states’ existing obligations under
international human rights law and IHL, Amnesty International is calling for a pre-emptive ban on
the development, production, and use of fully autonomous weapons systems. In the absence of a
prohibition, Amnesty International supports the call of UN Special Rapporteur on extrajudicial,
summary or arbitrary executions, Christof Heyns, for a moratorium on the development, transfer,
deployment and use of lethal and less-lethal AWS. In any event, the onus is on states that wish to
develop and deploy AWS, as will any new weapons system, to demonstrate first that specific uses
of each type of weapon can be fully lawful and, in particular, consistent with international human
rights and humanitarian law in operational circumstances.

RECOMMENDATIONS
 Amnesty Internationals calls for a pre-emptive ban on the development, transfer, deployment
and use of AWS, which covers autonomous weapons systems that are lethal and less-lethal. In the
absence of such a prohibition, Amnesty International calls on states to publicly support and
implement the call by the UN Special Rapporteur on extrajudicial, summary or arbitrary executions
to impose a moratorium on the development, transfer, deployment and use of lethal and less-lethal
AWS;

 With regard to the international human rights law implications of AWS, Amnesty International
supports UN Special Rapporteur on extrajudicial, summary or arbitrary executions, Christof Heyns’
call for the Human Rights Council to “remain seized with the issue of autonomous weapons systems,
in particular, as far as the rights to life and dignity are concerned”;

 Amnesty International proposes the establishment of an informal working group that sits
within the auspices of the CCW, which specifically examines the human rights implications of AWS.
This would ensure that current discussions focused on defining key issues such as ‘autonomous’ and
‘meaningful human control’ could continue in the CCW, as well as simultaneous consideration of the
human rights implications of AWS and closer coordination between human rights experts and arms
control experts;

 Amnesty International calls on states to send human rights experts to meetings and
discussions on AWS, including CCW Experts Meetings on Lethal Autonomous Weapons Systems, in
order to ensure that the human rights implications of AWS and the need to address those
implications are discussed fully at those meetings;

 Discussions on AWS must ensure that states’ obligations under international human rights law
– as well international humanitarian law –are effectively and actively addressed by UN mechanisms
and other international forums;

 Amnesty International urges all governments to develop and articulate a national policy on the
multiple challenges posed by AWS, including less-lethal AWS, that takes full account of their
obligations to respect and ensure IHRL and, where applicable, IHL.

Index: ACT 30/1401/2015 Amnesty International April 2015


www.amnesty.org

You might also like