When robots rule the waves?
The new enthusiasm for robots in military and policy circles has led to philosophers and ethicists paying increased attention to issues surrounding the military uses of robots. In particular, there is now a flourishing literature on the ethics of drone warfare and an emerging literature on the ethics of the development and deployment of autonomous weapon systems. (4) However, the high profile of aerial drones in the public eye--along with the fact that these are the systems that have seen most active service--has led to the latter literature focusing almost entirely on the ethical issues raised by autonomous uninhabited aerial vehicles and uninhabited combat aerial vehicles. To date there has been comparatively little discussion of the ethical issues raised by the prospect of autonomous submersibles or autonomous surface vessels. (5)
We believe it is high time that philosophers and military ethicists begin to address this lacuna, especially given the rapid development and military potential of autonomous unmanned underwater vehicles (UUVs) and unmanned surface vehicles (USVs). (6) Moreover, we believe that there are a number of ethical dilemmas specific to these technologies by virtue of the distinctive character of war at sea. This paper represents its two authors' initial attempt to collaborate --from somewhat contrasting ideological perspectives--in surveying and discussing these issues. We suggest that a number of unique and complex ethical questions are likely to arise regarding the applications of autonomous UUVs and USVs, including the following:
1. Should armed autonomous UUVs and USVs be understood (as the comparatively modest body of legal literature to date has posed the problem) as "vessels" or as "weapons"?
2. With what sorts of operations might autonomous UUVs and USVs legitimately be tasked in international, as opposed to territorial, waters?
3. Is the operation of armed autonomous systems compatible with freedom of navigation in international waters?
4. What is the capacity of future maritime and underwater autonomous systems, when weaponized, to abide by the requirements of distinction and proportionality in naval warfare?
5. What are the implications, with regard to the design and the ethics of the use of autonomous UUVs and USVs, of customary maritime duties, e.g., toward persons lost at sea?
Several of these issues may stand as significant barriers to the ethical deployment of autonomous UUVs and USVs, in some roles at least, for the foreseeable future. (7)
Our investigation of these questions proceeds through eight sections. In section 1, we provide a brief account of our reasons for believing that unmanned systems will come to play an increasingly vital role in future naval combat, and, by way of illustration, we introduce briefly a number of UUVs and USVs already deployed by the U.S. Navy or currently under development. In section 2, we argue that war at sea has a distinctive ethical character. Consequently, the use of unmanned--and especially autonomous--systems in this context may generate ethical issues that the larger discussion of the ethics of unmanned systems may have missed. Section 3 highlights the importance of a question about the appropriate way to conceptualize armed USVs and UUVs. We suggest that whether we think of particular systems as "vessels" or "weapons" will have implications for our understanding of the ethics of their applications, beyond merely the distinct legal regimes that apply to each, which we explore further in subsequent sections. Section 4 examines a range of issues that will arise about the operations of UUVs and USVs in different sorts of waters (e.g., territorial, international). In particular, we examine at length the implications of the operations of armed AWSs for freedom of navigation on the high seas. We then turn, in section 5, to discussing the ethical issues raised by the requirements of the principle of distinction for the operation of AWSs. While there are a number of reasons to believe that distinction poses fewer problems for AWSs on and under the seas than in other domains of warfare, we highlight the existence of four different cases in which it nevertheless remains a profound challenge. Section 6 considers the question of proportionality. As was the case with distinction, there are some reasons to expect proportionality calculations regarding civilian casualties to be easier in the context of war at sea than in other forms of warfare. However, once we acknowledge that both damage to the environment and enemy combatant casualties are relevant to the ethical (if not the legal) requirement of proportionality, even at sea proportionality also looks very difficult tor machines. Section 7 complexifies the discussion of the preceding two sections by considering the standard of compliance with the principles of distinction and proportionality that we should require of AWSs; the possibility that maintaining a human being "in the loop" (or perhaps "on the loop") could prevent attacks on illegitimate targets; and the implications of the UMS for the requirement of "precautions in attack." In section 8 we discuss the implications of the duty of rescue that exists in the context of war at sea for the design and applications of UUVs and USVs. We suggest that the fact that coming to the rescue of combatants lost at sea would not risk the lives of the crew in the case of UMSs means that the duty of rescue may be especially stringent on such systems; on the other hand, unless they are designed to possess the capacity to rescue, they may have no such obligation. It will be especially important therefore to think through the question of the obligations on UMSs in war at sea when it comes to rescue before many (more) such systems are designed and deployed. Finally, by way of a conclusion, we offer some brief remarks about the overall nature of our discussion and some suggestions for productive lines of inquiry for further research.
SECTION 1: ROBOTIC WEAPONS FOR WAR ON AND UNDER THE SEA
While aerial drones may have been hogging the limelight thus far when it comes to the military uses of robotics, there is currently an enormous amount of interest in the development and application of remotely piloted, semiautonomous and autonomous weapons to fight wars on and under the sea. (8)
The existence of waves, currents, tides, and submerged obstacles and the difficulties of maintaining reliable communications through water in some ways make the oceans a more difficult environment for robots than the air. However, remaining afloat or submerged at a given depth is less technically demanding than remaining airborne, and surface vessels need to move in only two dimensions rather than the three required of aerial vehicles. The relatively small number of terrain types in war at sea and the virtual nonexistence of legitimate commercial traffic beneath the sea, as well as the fact that blue-water operations often may proceed without regard to concerns about running aground, also mean that for robots the oceans are a more tractable environment in which to conduct warfare than is the land.
Moreover, the results that might be achieved through the further development and deployment of UUVs and USVs are substantial. Operations at sea--especially underwater--are always dangerous, often dull, and often dirty, at least in the sense of being uncomfortable for and wearing on those involved. As such, many missions at sea are well suited to being assigned to robots. As we discuss further below, the military advantages to be secured by the development of autonomous systems for war on and under the seas, in particular, are enormous. (9)
For all these reasons, we expect that naval operations will be the next frontier for the development and deployment of robotic weapons in the coming decade(s). As we are most familiar with the U.S. UUV and USV programs, we will support and illustrate this claim with a brief discussion of the U.S. Navy's progress in this area. However, a number of countries currently are developing such systems. (10)
A graphical overview of the U.S. Navy's inventory of systems at the time of writing may be found in Unmanned Systems Integrated Roadmap FY2013-38.
Unmanned surface vehicles have enormous potential in naval operations, although this potential is just beginning to be explored. The fact that these UMSs operate on the surface means that maintaining a human being in (or on) the loop is more feasible than it is for submersibles. Nevertheless, as in the case of UMSs more generally, there are still powerful military and economic dynamics pushing toward the development of systems that are capable of fully autonomous operations.
The U.S. USV inventory already includes a number of systems of different sizes and intended for different roles, with more under development. Navy scientists are using self-propelled, self-guided, and self-sufficient "wave gliders" (essentially modified solar- and wave-powered surfboards) manufactured by Liquid Robotics to gather meteorological and oceanographic data; in the future these systems might be used for intelligence, surveillance, and reconnaissance (ISR) missions. (11) The U.S. Navy has trialed USVs for maritime security and fleet protection. The Spartan Scout is a rigid-hull (aluminum) inflatable boat that is capable of remote-controlled and semiautonomous operations. Software called CARACaS (for "Control Architecture for Robotic Agent Command and Sensing"), which allows one human supervisor to oversee the operations of a number of USVs, has been used to provide USVs with the capacity for swarming to intercept enemy vessels. (12) Of course, the same systems might serve as weapons platforms that could be deployed in aggressive forward postures without placing crews at risk. The U.S. Navy tested a version of Spartan Scout armed with .50 caliber machine guns as early as 2002 and successfully demonstrated the firing of missiles from it in 2012. (13) The technology that makes possible defensive swarming also enables unmanned craft to swarm offensively, with the aim of overwhelming enemy ship-based defenses.
The U.S. Navy is also actively interested in developing an antisubmarine warfare (ASW) capability using USVs. The Defense Advanced Research Projects Agency (DARPA) has responded to the threat posed to U.S. vessels by the new generation of quiet diesel submarines by initiating a program to build and test an autonomous trimaran capable of tracking submerged enemy submarines for extended periods. (14) The Anti-submarine Warfare Continuous Trail Unmanned Vessel, or "Sea Hunter," is currently scheduled for trials beginning in 2016; the key navigational and collision-avoidance systems for this vessel underwent successful trials using a test boat in January 2015. (15) Should this project come to fruition, we would expect to see extended-range autonomous navigation and collision-avoidance capabilities rolled out to any number of other surface vessels.
Submarine operations are notoriously dangerous, so removing human crews from submersibles wherever possible is arguably a moral imperative; it also has a number of other benefits. Because unmanned systems carry no crew, they can be significantly smaller than the manned systems required to carry out similar operations. This permits UUVs to operate more quietly, for longer periods, and with a longer range. Autonomous UUVs, in particular, show enormous potential for operating for very long periods without needing to surface to replenish oxygen or fuel supplies or to return to base to rotate crews. This renders them ideal for roles in which the capacity to loiter undetected is an advantage. Indeed, because any emissions risk giving away two of the most vital secrets of a submersible--its presence and its location--the capacity to operate autonomously is a requirement for an effective unmanned submersible.
It is therefore no surprise that the U.S. Navy has an ambitious program of research and development of UUVs, especially autonomous UUVs, as well as a number of existing systems already deployed. For reasons of space, we will discuss only a few of these. (16)
UUVs' capacities for stealth and for use in circumstances in which it might be too expensive or dangerous to deploy a manned vessel make them ideal for ISR. Almost every UUV we have seen discussed in the literature is advertised as having a valuable role to play in ISR. For instance, the Sea Maverick and Sea Stalker UUVs (see the figure) are small(ish) semiautonomous submarines intended to carry out reconnaissance missions in depths of up to one thousand feet. (17) The Littoral Battlespace Sensing-Glider uses an innovative propulsion system involving changes of buoyancy to travel the oceans for up to a month at a time and return oceanographic data useful for submarine warfare. (18) The U.S. Navy also is experimenting with more-speculative systems such as the Cyro jellyfish, with the thought that a network of small, submersible, low-cost but hard-to-detect systems could provide valuable intelligence on enemy activities in contested waters. (19)
Similarly, UUVs have an obvious utility in countermine warfare, which role can be especially dangerous for manned vehicles. The U.S. Navy possesses a number of systems intended to perform this function, including the Mark 18 (Mod 1) Swordfish, the Mark 18 (Mod 2) Kingfish, and the Littoral Battlespace Sensing autonomous underwater vehicle (AUV), all derived from variants of the Remote Environmental Monitoring Unit System (known as REMUS) AUV manufactured by Elydroid, as well as the AN/BLQ-11 autonomous unmanned underwater vehicle (formerly called the Long-Term Mine Reconnaissance System), which may be launched from the torpedo tubes of Los Angeles- and Virginia-class submarines. (20) The mine countermeasures package for the littoral combat ship is based around an autonomous remote multimission vehicle (RMMV) that detects mines with a variable-depth, towed-array sonar. (21)
Importantly, as we discuss further below, armed UUVs themselves share much in common with naval mines (what is an autonomous torpedo but a "swimming" mine?) and may be used in a similar role. Indeed, mine warfare is on the verge of a profound revolution, made possible by the capacity to separate the sensor packages that detect enemy vessels from the submerged ordnance that is tasked with destroying them. While the U.S. Mk 60 encapsulated torpedo (CAPTOR) deepwater mine already had provided proof-in-principle of this possibility, recent innovations in sensors, marine propulsion, and autonomous navigation have expanded the prospects for development of such systems radically. In the future, nations may defend themselves--or deny the sea to others--using large arrays of networked sensors that communicate targeting information directly to a smaller number of autonomous armed UUVs lurking in the depths nearby. (22)
Finally, perhaps the most ambitious set of roles anticipated for any UUV consists of those the large-displacement unmanned underwater vehicle (LDUUV) is supposed to fulfill. The LDUUV is an experimental autonomous submarine intended to be able to navigate and operate under water for extended periods after being launched from a shore-based facility, an appropriately equipped nuclear submarine, or a surface vessel. The tasks envisioned for it include underwater reconnaissance and mine countermeasures, but extend to carrying and deploying smaller UUVs, or even to launching aerial drones for surface reconnaissance. (23) The U.S. Department of Defense recently announced a tender process to provide LDUUVs with an ASW capability. (24)
It is clear that the ultimate conclusion of the technology trajectory being explored in this system is a fully autonomous submersible capable of the same range of operations as a manned submarine. (25) In the discussion that follows, it is often the LDUUV, including future developments thereof, that we have in mind when we discuss the issues raised by the prospect of armed autonomous UUVs.
SECTION 2: THE DISTINCTIVE ETHICAL CHARACTER OF WAR AT SEA
There has been a small but productive discussion in the literature concerning the legal status of UUVs and USVs. (26) However, to date there has been little discussion of the ethical issues these systems raise. Our concern here is primarily with the latter topic. Insofar as legal instruments reflect, at least in part, the existing consensus on the duties and obligations of those whose activities they govern, we sometimes will refer to legal texts and precedents in the course of our argument. Nevertheless, we write in the conviction that the law does not exhaust ethics. Not only do provisions of the law fail to address ethical concerns, but those very legal constraints may pose moral dilemmas that will need to be addressed in operational policy and naval warfare strategy. In addition, there may be obvious ethical demands on warfighters that are yet to be codified in law. Indeed, there may be activities that are legally permitted but morally impermissible. (27) Ethical principles may provide useful guidance to warfighters where current law is silent or lacking. They also may motivate and inform attempts to revise, extend, or supplement existing law.
One reason to believe that the development of robotic weapons for naval warfare might raise new ethical issues is that war at sea differs in important respects from war in (most) other environments. (28) As a result, the moral norms and customs that have evolved to regulate naval warfare are arguably more demanding than those regulating warfare elsewhere, are more deeply entrenched in the consciousness of warfighters, and have distinctive elements.
A full investigation of what is ethically distinctive about naval warfare is beyond the scope of this article. However, a brief excursion into this topic will prove useful to frame our subsequent discussion. We believe that four features of war at sea play a key role in shaping the ethical (and legal) codes that regulate the activity of naval combatants. (29)
(1) In wartime as in peacetime, the sea itself is a deadly adversary of those who travel on or under it. Even in peacetime, hazards--in the form of strong winds, rough seas, and hidden reefs--abound, while shipwreck and drowning are ever-present dangers. In wartime, seafarers who are forced to abandon ship after an enemy attack may find themselves facing nearly certain doom: alone in freezing waters or floating in a small life raft, and thousands of miles from land.
(2) Because of the hostile nature of the marine environment, life at sea is primarily a collective life, one in which men (and increasingly women) are thrown together in a mutual endeavor framed by the possibility of misadventure. (30) Few people go to sea by themselves. Rather, people go to sea together in vessels, and therein form miniature--or, on modern capital ships, quite large--societies in the midst of a hostile environment.
These first two facts already have two important consequences for ethical understandings regarding war at sea.
First, the collective nature of life at sea and the shared vulnerability of all seafarers to misadventure and drowning mean that a strong expectation of mutual aid has grown up among those who go to sea. In particular, all those who go to sea are understood to have a duty to come to the aid of those who are lost at sea, whenever it is possible to do so without serious danger to themselves. This duty transcends ordinary national loyalties and has no direct analogue in land warfare. (31) The development of this expectation may be accounted for as a function of the need for a form of social insurance for this risky endeavor; each and every person at sea is safer if there is an expectation that everyone will come to the rescue of anyone as required, and consequently it is in each and every individual's interest if this expectation is promulgated widely and failures to live up to it are subject to sanctions, both formal and informal. Obviously, war--and the dehumanization of the enemy that often accompanies it--places this expectation under stress. Nevertheless, because enemy sailors in the water are no longer combatants, by virtue of being hors de combat, and because the risk of being in need of rescue is higher for all seamen during wartime, the expectation remains that vessels will render aid to, and will attempt to rescue, individuals lost at sea regardless of their nationality when they have the capacity to do so and as long as doing so would not jeopardize the safety of the vessel and those on board. (32) Moreover, the extent to which all those who go to sea share a distinct way of life compared with those who remain on land--and the solidarity that this encourages--along with the constant danger posed by the sea to all combatants ensures that this duty of rescue remains central to maritime culture, even in wartime. (33)
Second, the ethical and legal codes that govern war at sea are primarily concerned with the activities and fates of "vessels." As the operations of a ship are the result of a cooperative activity, it is often not possible to distinguish between the intentions of the commanding officer and that of his or her crew. Nor is it usually possible to attack some persons on board a vessel without targeting the vessel as a whole and thus risking the lives of everyone aboard. For these reasons, seamen literally sink or swim together. Thus, it is both natural and appropriate that the vessel be the primary locus of attention in ethical (as well as legal) deliberation about naval warfare.
Two other features of war at sea are important to bear in mind when thinking about ethics in that context. These concern the unique relationship between combatants and noncombatants in naval combat.
(3) The sea is more sparsely populated than the land, and in wartime the vessels that sail on or under it divide more or less naturally into those that are participating actively in the conflict and those that are not. (34) That is to say, especially with the benefits of modern sensor packages, military vessels are distinguished more easily from civilian vessels than groups of armed men are distinguished from civilians in land warfare, and it is more difficult for combatants to hide among the noncombatant population. Thus, with the exception of merchant vessels (of which, more below), which might have been pressed into service to carry cargo or personnel for military purposes, it is generally much easier to distinguish legitimate from illegitimate targets at sea than it is in other forms of warfare. (35)
On the other hand, (4) the comparatively featureless nature of the oceans and the lack of local geographical references for national and other relevant political boundaries mean that it is harder to separate combatants and noncombatants geographically. This problem is exacerbated by the fact that oceangoing commerce is essential to the flourishing--and even to the survival--of modern nations, with the consequence that, even during wartime, merchants will continue to ply the seas with their goods and passenger ships and ferries will continue to transport civilians. (36) At least partly in recognition of this fact, the high seas remain a "commons," owned by no one and available for use by everyone.
These latter two features of war at sea have led to the development of a sophisticated set of practices and agreements around the activities of belligerent and neutral parties intended to allow neutral parties to continue to navigate the seas peacefully even when wars are being fought. Customary international law relating to naval warfare attempts to balance the competing demands of national sovereignty and freedom of navigation, and distinguishes among belligerent and neutral nations' internal waters, territorial waters, and exclusive economic zones (EEZs) as well as the high seas, and places limits on the sorts of activities that legitimately may be pursued in each. (37) As we shall see below, understanding the competing considerations informing these treaties also will prove useful to resolving ethical issues relating to the areas and roles in which UUVs and USVs legitimately may be deployed.
We do not want to exaggerate the extent to which the ethics of war at sea differs from the ethics of fighting wars in other environments. The fundamental moral framework for naval warfare, as for land or air warfare, is outlined in just war theory. The special features we have highlighted here may be accounted for as consequences of the application of just war theory to the peculiar character of war at sea. Moreover, each of the various features of war at sea highlighted above may have some counterparts in other domains of warfare. (38) Nevertheless, drawing attention to the way in which the ethics of war at sea is structured by its special contextual circumstances may productively inform deliberation about the ethics of the development and deployment of robotic weapons in this context.
SECTION 3: THE STATUS OF ARMED USVS AND UUVS--VESSELS OR WEAPONS?
As noted above, the legal and ethical codes that govern war at sea are mostly concerned with the activities of ships and submarines and place demands on individuals primarily--although not exclusively--through their roles on these vessels.
A number of legal authorities already have begun to consider whether or when UUVs and US Vs should be considered "vessels" under the law of the sea. The emerging consensus seems to be that autonomous UUVs and USVs, at least above a certain size, should be classed as vessels. (39) While remotely piloted vehicles plausibly might be held to be extensions of the vessel(s) from which they are operated, systems capable of extended autonomous operations should be understood as vessels in their own right. (40)
As we shall see below, the question of how we understand USVs and UUVs is also central to the ethics of their design and application. The more we think of these systems as autonomous and controlled by an onboard computer, and the more roles they become capable of fulfilling, the more natural it is to think of them as vessels. However, as the discussion below highlights, understanding them as vessels appears to impose demanding ethical requirements on their capacities and operations, especially relating to distinction, proportionality, and the duty of rescue.
An alternative way of addressing these requirements, in the light of such conundrums, is to think of armed autonomous USVs and UUVs themselves instead as weapons, which may be deployed by warfighters, who then become responsible for ensuring that the use of the weapon meets the requirements of distinction, proportionality, and so on. (41) Yet as we shall see, this way of proceeding generates its own challenges. An important early finding of our research, then, is that much work remains to be done to clarify the best way of understanding the status of armed UUVs and USVs in the context of the larger ethical framework governing war at sea (as opposed merely to their current legal status).
SECTION 4: DEPLOYMENT--WHERE, WHEN, AND WHY?
The UN Convention on the Law of the Sea (UNCLOS) attempts to balance the competing claims of national sovereignty and freedom of navigation in peacetime by distinguishing among different sorts of waters regarding their statuses and the permissibility of different sorts of activities therein. Customary international law relating to naval warfare extends this to regulate the relations between belligerent and neutral parties insofar as possible. The research and analysis required to assess the operations of USVs and UUVs within these frameworks are beginning to be undertaken now, and some initial results are starting to emerge. (42) Thus, for instance, Andrew Henderson suggests that "UUVs may operate freely in both the high seas and the EEZ while exercising the requisite due regard for the interests of other vessels and posing no threat to the territorial integrity of the coastal state" and remain submerged while exercising transit passage in international straits and archipelagic-sea-lanes passage in archipelagic sea-lanes. In territorial seas, he suggests, UUVs must operate on the surface to exercise the right of innocent passage and display appropriate lights and make sound signals to facilitate safety of navigation. (43) Brendan Gogarty and Meredith Hagger also suggest that USVs and UUVs would be restricted in the activities they can undertake while exercising the right of innocent passage. (44) Rob McLaughlin emphasizes that USVs and UUVs are clearly subject to the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs) and must be capable of avoiding collisions to such a degree that they could be said to maintain what he paraphrases as a "proper and sufficient lookout." He also allows that the presence of a foreign submerged UUV within a nation's territorial waters might constitute a sovereign affront justifying the use of armed force. (45)
We leave the task of settling the legal questions raised by the deployment of UUVs and USVs in various sorts of waters to others qualified to complete it. However, some discussion of the deeper ethical questions underpinning and surrounding the relevant legal frameworks is appropriate here, and we hope it will inform the ongoing legal debate usefully.
It does seem reasonable, for instance, that the moral right nations have over their territorial waters, and to a lesser extent their continental shelves and EEZs, should allow them to exclude USVs and UUVs conducting--or perhaps just capable of conducting--certain sorts of operations. If nations have a right to prevent other nations from conducting mining or survey operations in their EEZs or carrying out operations injurious to their security in their territorial waters, this right surely would carry over consistently to exclude unmanned vessels just as much as manned vessels. Indeed, arguably the fact that UUVs and USVs are unmanned makes their use in these sorts of waters more suspicious and threatening to the interests of sovereign governments, on the assumption that other nations will be more likely to deploy such vessels in hazardous environments that might generate a military response, given that doing so will not place a human crew at risk of death or capture. Requiring such systems to confine themselves to innocent passage through territorial waters is at least a partial solution to this problem.
The ethics of the use of autonomous UUVs and USVs on the high seas remains an open--and controversial--matter. At first sight at least, the right to freedom of navigation in international waters appears to extend to inclusion of these systems, presuming that they do not pose too much of a navigational hazard to other vessels. However, interestingly, this presumption rests on an understanding of them as vessels and may be unsettled when we start to consider the prospect of armed autonomous UUVs and US Vs and whether such systems should be thought of, instead, as weapons.
Roughly speaking, the operations of vessels in international waters are permissible as long as they are compatible with the right of free navigation of other vessels through the same waters. Thus, if they are to operate on the high seas, UUVs and USVs must have the capacity reliably to avoid posing a hazard to other vessels. At a bare minimum, this requires taking the appropriate measures to minimize the risk of collision. While the COLREGs spell this out as requiring all vessels "at all times [to] maintain a proper lookout by sight and hearing"--phrasing that encourages the reader to presume a human being will be on board, or at least supervising remotely--we can see no reason why a fully autonomous system that proved equally capable of avoiding collision with other vessels without human supervision should not be judged to meet the appropriate standard. (46)
Of course, armed UUVs and USVs operating on the high seas would appear to pose risks to commercial shipping and to the warships of neutral nations beyond simply the risk of collision; they might (accidentally) fire on them, for example. Their significance for the right of freedom of navigation is therefore likely to depend on their capacity to distinguish between legitimate and illegitimate targets of attack, as discussed in sections 5 and 6 below.
A key question in the larger debate about the ethics of autonomous weapons concerns whether--by analogy to what we suggested was the case with regard to the capacity to avoid collision--it would be sufficient to render the use of such weapons permissible if they were capable of achieving results similar to the standard required of human beings with respect to compliance with the moral principles of distinction and proportionality. Those inclined to understand the principles of jus in bello as grounded primarily in a concern for the rights of noncombatants are likely to believe that this would be sufficient to render the use of AWSs permissible--and indeed may be tempted to the conclusion that their use will be mandatory once such weapons become capable of exceeding human performance in this regard. (47) On the other hand, a number of authors have suggested that if we think of the requirements of jus in bello fundamentally as ethical demands on the human being making the decision to use lethal force, we may conclude that the absence of a human will at the moment the attack is carried out means that autonomous weapons cannot be said to comply with these principles at all. (48) Insofar as our concern is with the compatibility of the operations of AWSs with the right to freedom of navigation rather than with the wider conceptual debate concerning the ethics of autonomous targeting, though, it appears that the relevant standard of discrimination is just that required of human beings in similar circumstances.
However, there is another reason to worry that achieving a high standard when it comes to the capacity to distinguish between legitimate and illegitimate targets may not be sufficient to render the use of AWSs ethical on the high seas. The presence of AWSs operating in particular waters might exercise a "chilling" effect on commercial shipping over a wide area--and thus impinge on the right of freedom of navigation--even if the chance of an accidental attack by AWSs was extremely remote, given the capacities of these systems. This possibility seems especially likely if we think of autonomous UUVs and USVs as weapons rather than vessels. Indeed, one well might argue that armed autonomous UUVs at least should be understood as sophisticated versions of free-floating mines, and consequently should be prohibited. (49) The use of drifting mines that do not disarm themselves within an hour is prohibited under international law because of the threat they pose to freedom of navigation. (50) The fact that the chance of any particular ship being struck by any particular drifting mine is small does not seem to affect the force of this concern.
An important point of reference for our intuitions here is CAPTOR, which is a moored torpedo-launch system capable of detecting the acoustic signature of approaching enemy submarines and firing a torpedo to destroy them. (51) This system is arguably already autonomous insofar as the "decision" to launch a torpedo is made without direct human input at the time. Versions of the system have been in use since 1979 without causing significant international outcry, which suggests that concerns about freedom of navigation in open waters need not rule out the deployment of autonomous weapon systems.
However, there are at least three reasons to be cautious about this conclusion. First, because the CAPTOR itself is fixed--even if its range of operations is extended--the system would appear to pose less of a danger to navigation than hypothetical free-ranging AWSs. (52) Second, insofar as this weapon is advertised as an antisubmarine system, those plying the surface of the waters may feel they have little to fear from it. International opinion might be very different should it become common knowledge that similar systems were being tasked with destroying surface vessels. Finally, the absence of any outcry against CAPTOR and similar systems needs to be understood in the context of a history over which they have not been responsible--to date--for any noncombatant casualties. The first time an AWS deployed at sea attacks a commercial--or, worse, a passenger--vessel, we might expect public and international opinion about their legitimacy to change dramatically.
Even very reliable AWSs therefore may jeopardize freedom of navigation if vessels are unwilling to put to sea in waters in which AWSs are known to be operating. While fear of (accidental) attack by an AWS might appear to be irrational when compared with the risks that manned systems pose, beliefs about risk are notoriously complex and difficult to assess because they often contain hidden value judgments. In this case, a reluctance to risk attack by an AWS may express the value judgment that human beings alone should be responsible for decisions to take human lives. Insofar as the right of freedom of navigation exists to protect and sustain international commerce, what matters is the willingness of ships to ply the oceans. Subjective judgments of risk may be just as significant for the existence of freedom of navigation as--indeed, maybe more so than--the objective risks that ships actually take when they leave port.
Therefore, it may turn out that the international community will be required to adjudicate on the balance of the interests of states in deploying AWSs and the desire of operators of civilian vessels not to be at risk of attack by an autonomous weapon. Any attempts to embed this judgment in legislation also will need to consider what is realistically achievable in this regard, especially given the military advantages associated with unmanned systems and the force of the logic driving their uptake. In many ways, such a debate would hark back to that which took place with the advent of submarine warfare, which effectively was resolved in favor of permitting the operations of military submersibles. We suspect that this is the most likely outcome with regard to armed autonomous UUVs and USVs as well. However, it is important to acknowledge the competing considerations in this debate, summarized above.
A number of further questions may arise concerning the operations of armed autonomous UUVs and USVs in various waters, but space limitations permit mere mention of them here. The difficulty in imagining autonomous weapons having the capacity to capture enemy or neutral vessels suggests that they could play at most a limited role in naval blockades or taking neutral merchant vessels as prizes. (53) The requirement to record the locations of mines so that they can be removed or rendered harmless after the cessation of conflict would appear to be moot, when "mines" are themselves mobile and autonomous. (54) However, the considerations motivating this requirement--reducing the subsequent hazards to shipping postconflict--imply that autonomous weapons must be able reliably to render themselves harmless on instruction or after some defined period. There undoubtedly are other issues that require further investigation.
SECTION 5: DISTINCTION
Perhaps the most fundamental ethical requirement in wartime is to confine one's attacks to enemy combatants, and as much as possible to try to avoid civilian casualties. Thus the jus in bello principle of distinction requires that warfighters refrain from targeting noncombatants and take appropriate care to minimize the noncombatant casualties caused by attacks targeted at combatants.
Much of the current criticism of AWSs proceeds from the claim that robotic weapons are unlikely to be capable of meeting the requirements of distinction for the foreseeable future. In counterinsurgency warfare in particular, identifying whether someone is a combatant requires a complex set of contextual judgments that probably will be beyond the capacity of machines for the foreseeable future. (55) Whether this problem is insurmountable or exists in all roles in which we might imagine AWSs being used is a controversial question that is larger than we can resolve here. In this context we will settle for observing that the problem of distinction is arguably less demanding in naval warfare because there are fewer potential targets and because sonar and radar are more capable of distinguishing between military and civilian vessels than image recognition, radar, and lidar (light detection and ranging--"laser radar") are at distinguishing among targets in land warfare. (56) Indeed, one reason advanced for favoring the use of autonomous systems on or under the sea, especially in blue-water missions, is that, in comparison with on the land or in the air, on the high seas the "civilian footprint" is comparatively small, even allowing for commercial shipping and recreational boating. Moreover, the problem of distinction looks especially tractable in the context of ASW, given the relative paucity of civilian submarines with tonnages or acoustic signatures comparable to those of military submarines, and the fact that those few civilian systems that do exist tend to operate in a limited range of roles and locations (primarily around oil rigs and submarine cables). Therefore we might expect that if robots are to become capable of distinction in any context, they will become capable of it in war on and under the sea.
Nevertheless, there are at least four sorts of cases in which the requirements of distinction pose a formidable challenge to the ethical operation of autonomous weapons in naval warfare.
First, to avoid attacks on military ships of neutral nations, AWSs will need to be able to identify the nature and the nationality of potential targets, not just to determine that they are warships. In some cases, in which ships of the enemy's fleet are easily distinguishable from those of other nations because of distinctive radar or acoustic profiles, this problem may not arise. However, in some circumstances identifying that a ship carries guns or torpedoes, is of a certain tonnage or class, or both will not be sufficient to establish that it is an enemy warship. Instead, making this identification will require the ability to form reasonable conclusions about its identity on the basis of its historical pattern of activity and its threat posture within the battle space. One obvious way to solve this problem would be to program autonomous UUVs and USVs to confine their attacks to targets that are themselves firing weapons. (57) However, this would reduce significantly the military utility of AWSs, especially in strike and area-denial roles. Whether computers ever will be able to make the necessary judgments to avoid the need for this restriction remains an open question.
Second, as enemy vessels that have clearly indicated their surrender are no longer legitimate targets under the Geneva Convention, AWSs must be able to recognize surrender. (58) It is possible that in the future warships may be expected to carry a "surrender beacon" capable of communicating to any AWS operating in the area that in fact they have surrendered. Until that day, however, AWSs will need to have the capacity to recognize and respond to the communication of surrender under existing conventions, i.e., through changes in threat posture and display of signal lights or flags. Again, at this stage it is unclear whether robots ever will be able to do this reliably.
Third, AWSs must be able to identify when an enemy ship is hors de combat by virtue of being so badly damaged as to be incapable of posing any military threat. In rare circumstances it may not be possible for a badly damaged and listing ship to signal surrender. Thus, morally, if not legally, speaking, even an enemy warship that has not indicated surrender is not necessarily a legitimate target if it is no longer capable of engaging in hostilities. (59) Human beings are (sometimes) able to discern when this circumstance applies, using their rich knowledge of the world and of the motivations and likely actions of people in various situations. Before the use of AWSs would be ethical, they would need to be at least as capable as human beings of making such discriminations.
Importantly, these last two issues appear in a different light depending on whether we think of AWSs as vessels or as weapons. If an enemy warship surrenders after a torpedo is launched from a manned submarine, for instance, the ship's destruction would be a tragedy but not a crime. However, if a ship fires on an enemy vessel that clearly has indicated surrender, that is a war crime. If we think of an AWS as a weapon, therefore, then as long as the officer who deploys it does not do so knowing the intended targets have surrendered or otherwise become hors de combat, its use will be legitimate even if there is some chance that the status of its targets may change after it is deployed. On the other hand, if we think of the USV or UUV as a vessel, then it seems it must have the capacity to detect whether a potential target has surrendered or otherwise become hors de combat to avoid attacks in such circumstances. Of course, if the delay between deploying an AWS understood as a weapon and its carrying out an attack is too long--a matter of days rather than hours, for instance--this might shake our conviction that it is sufficiently discriminating to be ethical. (60)
Fourth, when it comes to operations to interdict or attack merchant shipping, the problem of distinction is especially challenging just because it is so sensitive to context. AWSs would seem to be poorly suited, for instance, to making judgments about whether merchant vessels are carrying enemy troops or "otherwise making an effective contribution to military action." (61) The fact that AWSs are unlikely to be capable of searching or capturing merchant ships also limits their utility in making this discrimination.
SECTION 6: PROPORTIONALITY
The ethical requirements of proportionality under jus in hello ask whether the military advantage to be gained by an attack on a military target is sufficient to justify the death and destruction the attack reasonably might be expected to cause. Importantly, while the legal requirement of proportionality usually is understood to require only that the noncombatant casualties ("collateral damage") that it is reasonable to expect an attack on a military target to cause are not excessive in relation to the military advantage the attack seeks to secure, the ethical principle grants weight to the lives of combatants in this calculation as well. (62) Thus, for instance, a deliberate attack on an enemy military installation housing a large number of enemy warfighters who posed no immediate threat, when it was already known that the enemy had signed an agreement to surrender effective the next day, would be unethical by virtue of being disproportionate.
One of us (Sparrow) previously has argued elsewhere that the requirements of proportionality stand as a profound barrier to the ethical use of AWSs. (63) The calculations of military advantage required to assess whether a given number of civilian (or military) casualties is proportionate are extremely complex and context sensitive. They require a detailed understanding of the way the world works that is, Sparrow has argued, likely to remain beyond the capacities of autonomous systems for the foreseeable future. (64) The other of us (Lucas) is less pessimistic, believing that AWSs' potential to exceed the limited abilities of human beings when it comes to making judgments of proportionality is an important part of their promise. (65)
Regardless, there are reasons to believe that these sorts of calculations of proportionality are likely to be easier in the context of war at sea. To begin with, as noted above, the relative lack of civilian "clutter" on the oceans means that the risk of civilian casualties in attacks on legitimate military targets in naval engagements is much lower than in land warfare, reducing the number of circumstances in which a judgment of the proportionality of anticipated civilian casualties is required. There are also typically fewer units involved in naval engagements than in land warfare and the scope of operations available to individual units is less, which makes it more plausible to think that a computer could calculate the military advantage associated with a particular attack and thus whether a given number of military deaths would be justified. (66)
On the other hand, there is another proportionality calculation that is especially difficult in the context of war at sea. Military operations may have significant and long-term implications for civilian life via their impact on the environment. (67) Consequently, combatants now are also held to be under an obligation to consider and, where possible, to minimize the damage to the environment their activities cause. These obligations must be balanced against considerations of military necessity. In practice, then, combatants are required to make a calculation of proportionality when contemplating an attack to determine whether the military advantage the attack will achieve justifies the environmental damage it is likely to cause. However, the role played by wind, waves, and tides in distributing the debris resulting from war at sea and the complex nature of marine ecosystems make calculations of the environmental impacts of naval operations especially difficult. Determination of the intrinsic value of significant features of the environment (such as, for instance, clean rivers, coral reefs, or the spawning grounds of fish) is controversial, as is assessment of the instrumental value they have in terms of their contribution to human well-being. Judgments about such matters inevitably involve balancing a range of complex considerations as well as arguments about matters of (moral) value. For both these reasons, calculations of proportionality in attack in relation to damage to the environment seem likely to remain beyond the capacity of computers for many years yet.
Thus, once we admit that the marine environment and enemy combatant casualties are relevant to the proportionality calculation (in ethics, if not in law) and we take the broader strategic context into account, as well as the possible interactions of naval, ground, and air forces, it once more appears that making judgments of proportionality is fiendishly difficult and requires knowledge of the world and reasoning capacities that computer systems currently lack and seem likely to continue to lack for the foreseeable future. (68) Thus, at the very least, proportionality appears to remain a more difficult issue for AWSs in naval warfare than distinction.
SECTION 7: AWSS, "SUPERVISED AUTONOMY," AND PRECAUTIONS IN ATTACK
Of course, human beings also have significant limitations when it comes to their capacity to achieve distinction and make judgments of proportionality, so it might be argued that machines eventually will be able to perform at least as well as humans at these tasks. (69) This is an empirical matter. However, there is also a deeper philosophical question involved regarding the nature and force of the ethical imperatives underpinning the requirements of jus in hello. While human beings often fail to behave ethically, when it comes to the duty to avoid taking human life unnecessarily, morality demands perfection. Consequently, it might be argued that there is something troubling about justifying the use of an autonomous weapon solely on the basis that it makes as few mistakes as or fewer mistakes than the alternative. (70)
We cannot hope to settle these questions here. Indeed, the authors well may disagree upon them. (71) A partial solution to both the problem of distinction and proportionality might be achieved by requiring AWSs to seek input from a human supervisor whenever the risks of attacking an illegitimate target exceed some predetermined threshold. A number of authorities already advocate "supervised autonomy" as a way of attempting to combine the benefits of autonomous operations and human decision making in complex environments. (72) Yet this proposal has obvious limitations. To begin with, it presumes that the task of accurately assessing the risk of inadvertently attacking an illegitimate target is easier than identifying a potential target as legitimate or not in the first place, which may not be the case. Perhaps more importantly, relying on human supervision to carry out combat operations ethically would sacrifice two of the key benefits of autonomous operations. It would require maintaining a robust communications infrastructure sufficient to allow the AWS to transmit the relevant data to a base station and receive instructions from the human operator, which is especially challenging in the context of operations under water. It also would jeopardize the capacity of autonomous systems to conduct stealthy operations. In particular, submersibles would need to transmit and receive signals in real time--and thus risk giving away their locations--to allow a human supervisor to provide input to their decisions. While supervised autonomy may be a solution in the context of operations against technologically unsophisticated adversaries without the capacity to contest the electronic battle space or launch kinetic attacks against communications infrastructure, it seems unlikely to be an attractive solution in the longer term.
There is, however, a further complexity here. The jus in bello principles of distinction and proportionality not only distinguish between legitimate and illegitimate targets but also demand that warfighters make all feasible efforts to avoid attacking illegitimate targets in circumstances in which, for various reasons, it is difficult for them to distinguish between the two. Thus, as the San Remo Manual on International Law Applicable to Armed Conflicts at Sea notes, warfighters "must take all feasible measures to gather information which will assist in determining whether or not objects which are not military objectives are present in an area of attack" and "take all feasible precautions in the choice of methods and means to avoid or minimize collateral casualties or damage." (73) While the question of what sorts of measures or precautions are "feasible" in a given context is obviously complex and often controversial, the level of risk to warfighters involved in the various options available to them is clearly relevant: there must be some limit to the amount of risk that we can reasonably expect warfighters to take on to achieve any given degree of confidence about the nature of the targets they intend to attack. The fact that no human lives would be placed at risk--directly (see below)--by requiring autonomous UUVs and USVs to take any given sort of actions to minimize the chance of inadvertently attacking civilian targets or causing disproportionate casualties suggests that the requirements to take "all feasible measures" and "all feasible precautions" might be significantly more demanding for these systems.
Thus, for instance, unmanned submersibles might be required to launch sensor buoys, use active sonar, or even surface to facilitate identification of targets. Indeed, AWSs might even be required to await authorization from a human supervisor before carrying out an attack. (74) According to the strongest version of this line of argument, fully autonomous operations of a UUV or USV (or, one suspects, any AWS) would be unethical.
There are two obvious ways in which this conclusion might be resisted. First, given the military utility of unmanned systems--and an argument from military necessity--it might be argued that the risk to the "vessel," regardless of the absence of any crew on board, is properly relevant to judgments about feasibility: it would be unreasonable to include in the range of "feasible" precautions those that likely would result in the destruction of the system if carried out during an engagement. Second (in addition), while exposing an unmanned system to risk may not threaten any lives directly, the destruction of the vessel would jeopardize the safety of friendly forces who might have been relying on it to carry out its mission. Thus, human lives may well be at stake when we risk the safety of a UMS. These two considerations speak in favor of allowing autonomous systems to prioritize their own "safety" over the safety of those whose lives they potentially threaten through their targeting decisions.
The capacity of UMSs to take more precautions prior to launching an attack often is cited as an argument in favor of developing and deploying them. (75) The fact that they are unmanned means that they plausibly might be used in more-risky operations to try to achieve any worthwhile goal. Perversely, when the goal is the preservation of the lives of noncombatants, this might even mean placing (what would otherwise be) autonomous systems at risk by requiring them to seek authorization for each attack from a human operator. Yet this would vitiate many of the military advantages of autonomous operations, including the extent to which the use of UMSs reduces the risk to the lives of friendly forces. (76) The advent of armed autonomous systems therefore will require a potentially difficult conversation within the international community about the balance to be struck between military necessity and humanitarian considerations and about the role of human supervision of autonomous systems in securing this balance. (77)
SECTION 8: RESCUE
While the details of what is needed to satisfy the requirements of discrimination and proportionality may differ somewhat in naval warfare from war on land or in the air, these principles themselves apply to all warfare by virtue of their place at the heart of the doctrine of jus in bello. However, the duty of rescue that exists in the context of war at sea is especially stringent in, if not entirely unique to, naval warfare. (78)
We have suggested that, even in wartime, all vessels are ethically required to render aid to and attempt to rescue individuals lost at sea, regardless of their nationality, when they have the capacity to do so and as long as doing so would not jeopardize the safety of the vessel and those on board. Both clauses in the italicized caveat merit some discussion in the context of the operations of UUVs and USVs.
Whether this duty of assistance will impinge on the operations of USVs and UUVs will depend on whether we think these systems have, or should have, the capacity to conduct rescue operations. For instance, the fact that cruise missiles have no capacity to rescue those rendered helpless in the water after an attack is not thought to rule out their use in attacks on ships. It therefore seems likely that some AWSs--particularly those that we are inclined to classify as weapons, such as "smart" long-loiter-time torpedoes--will be excused from any obligation in this regard. However, when it comes to the operations of (currently hypothetical) larger autonomous USVs and UUVs, themselves armed with weapons--those it would be more natural to regard as vessels--the question will arise whether they should be required to have at least some capacity to conduct rescue operations. Even if such vessels were, as seems likely, incapable of taking prisoners on board, they might be provided with the capacity to launch inflatable life rafts or deploy emergency locator beacons to draw the attention of other vessels to the presence of people requiring rescue. In all likelihood, the costs associated with fitting such systems would be significant in terms of the military utility of the vessel, not least because deploying them might give away the location of a submersible. What seems clear, though, is that vessels without this capacity would be significantly less capable of achieving proportionality in attack. It might even be argued that the deployment of armed autonomous vessels without the ability to contribute to rescue operations would be unethical on this basis. (79)
Like the concepts of "feasible precautions" and "feasible measures" in attack, the duty of rescue is qualified with reference to the risk involved in attempting to provide assistance. Thus, because any attempt to provide assistance while combat is ongoing in the area would expose a vessel to a high risk of destruction by other enemy ships, in wartime this duty is understood to exist only "after an engagement." (80) However if UUVs or--more plausibly--USVs did have the capacity to conduct rescue operations, they might be held to be under a stronger obligation to do so than manned vessels simply because doing so, even in the course of a military engagement, would not endanger any human lives directly. Acknowledging this fact may even strengthen the intuition that AWSs should be provided with the capacity to conduct rescue operations.
Again, focusing on the safety of the vessel rather than that of its (nonexistent) crew, the lives of friendly combatants elsewhere (which might be threatened if the UUV or USV was destroyed), or both might provide grounds to resist this conclusion. We certainly expect that states deploying armed autonomous vessels will be reluctant to risk those vessels' destruction by programming them to provide assistance to enemy combatants lost at sea. Nevertheless, we expect it will be tough to sell the international community on prioritizing the "safety" of a machine over the lives of human beings lost at sea. Whether autonomous UUVs and USVs should be required to have some capacity to provide assistance to those lost at sea and the extent of their obligation to provide this assistance when they do have the capacity to do so are key questions to be answered by further research on this topic.
We are conscious that our deliberations have raised more questions than they have answered. We cannot claim that this survey of the main issues is exhaustive; there are undoubtedly further issues to be considered than those we have had the opportunity to discuss here.
Nevertheless, our investigations suggest that the distinctive ethical character ot war at sea generates a number of novel ethical dilemmas regarding the design and use of UUVs and USVs, dilemmas that do not arise for unmanned systems operating in the air or on land. In particular, the importance of freedom of navigation on the high seas and the obligation to come to the aid of those shipwrecked or lost at sea pose difficult challenges for the ethical operation of UUVs and USVs, especially armed and autonomous systems. Moreover, some of the ethical issues that do arise regarding the (hypothetical) operations of armed autonomous systems more generally are differently inflected in the context of war on and under the seas, including the implications of the requirements of proportionality and distinction for the operations of these systems.
Finally, what seems clear to both authors, despite specific differences, is that much more work remains to be done to resolve the question whether--or perhaps which--UUVs and USVs should be conceptualized as vessels or weapons, and to settle the role that should be accorded to legal conventions and historical debates about mine warfare in shaping future practice regarding UUVs. The fact that such systems blur the lines between weapons platforms and weapons means that ethical as well as legal frameworks may need to be rethought and refined in the pursuit of an appropriate balance between the demands of military necessity and humanitarian concerns in the naval warfare of the future.
We hope that our discussion of these issues here will prove a useful starting point for future research into these questions.
Thanks are due to Ray Buettner, Ryan Calo, Stephen Coleman, Shane Dunn, Mark Gubrud, John Jackson, Chris Jenks, Jeff Kline, Scott Littlefield, Tim McCormack, Rob McLaughlin, Andrew Norris, Heather Roff, and B. J. Strawser for conversations and correspondence in the process of developing this manuscript. Mark Howard ably assisted us with sources and with preparing the paper for publication.
(1.) Thomas K. Adams, "Future Warfare and the Decline of Human Decisionmaking," Parameters 31, no. 4 (Winter 2001/2002), pp. 57-71.
(2.) Gary E. Marchant et al., "International Governance of Autonomous Military Robots," Columbia Science and Technology Law Review 12 (2011), pp. 272-315; P. W. Singer, Wired for War: The Robotics Revolution and Conflict in the 21st Century (New York: Penguin Books, 2009).
(3.) U.S. Navy Dept., The Navy Unmanned Surface Vehicle (USV) Master Plan (Washington, DC: 2007); U.S. Army, U.S. Army Unmanned Aircraft Systems Roadmap 2010-2035: Eyes of the Army (Fort Rucker, AL: U.S. Army UAS Center of Excellence, 2010); U.S. Air Force, RPA Vector: Vision and Enabling Concepts 2013-2038 (Washington, DC: 2014); U.S. Defense Dept., Unmanned Ground Systems Roadmap (Washington, DC: Robotic Systems Joint Project Office, 2011); U.S. Defense Dept., Unmanned Systems Integrated Roadmap: FY2013-2038 (Washington, DC: 2014).
(4.) For drones, a useful starting point in what is now a very large body of work is Bradley J. Strawser, ed., Killing by Remote Control: The Ethics of an Unmanned Military (New York: Oxford Univ. Press, 2013). For AWSs, see, for instance, Jurgen Altmann, "Arms Control for Armed Uninhabited Vehicles: An Ethical Issue," Ethics and Information Technology 15, no. 2 (2013), pp. 137-52; Kenneth Anderson and Matthew C. Waxman, "Law and Ethics for Robot Soldiers," Policy Review, no. 176 (2012), pp. 35-49; Kenneth Anderson and Matthew C. Waxman, "Law and Ethics for Autonomous Weapon Systems: Why a Ban Won't Work and How the Laws of War Can" (national security and law essay, Jean Perkins Task Force on National Security and Law Essay Series, Hoover Institution, Stanford University / WCL Research Paper 2013-11, American University / Columbia Public Law Research Paper 13-351, 10 April 2013), available at papers.ssrn.com/; Ronald C. Arkin, "The Case for Ethical Autonomy in Unmanned Systems," Journal of Military Ethics 9, no. 4 (2010), pp. 332-41; Peter Asaro, "On Banning Autonomous Weapon Systems: Human Rights, Automation, and the Dehumanization of Lethal Decision-Making," International Review of the Red Cross 94, no. 886 (2012), pp. 687-709; Jason Borenstein, "The Ethics of Autonomous Military Robots," Studies in Ethics, Law, and Technology 2, no. 1 (2008); Armin Krishnan, Killer Robots: Legality and Ethicality of Autonomous Weapons (Farnham, U.K.: Routledge, 2009); Alex Leveringhaus and Tjerk de Greef, "Keeping the Human 'in-the-Loop': A Qualified Defence of Autonomous Weapons," in Precision Strike Warfare and International Intervention: Strategic, Ethico-legal and Decisional Implications, ed. Tom Dyson et al. (Abingdon, NY: Routledge, 2014); Marchant et al, "International Governance of Autonomous Military Robots"; Mary E. O'Connell, "Banning Autonomous Killing: The Legal and Ethical Requirement That Humans Make Near-Time Lethal Decisions," in The American Way of Bombing: Changing Ethical and Legal Norms, from Flying Fortresses to Drones, ed. Matthew Evangelista and Henry Shue (Ithaca, NY: Cornell Univ. Press, 2014); Michael N. Schmitt, "Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics," Harvard National Security Journal, 5 February 2013, harvardnsj.org/; Michael N. Schmitt and Jeffrey S. Thurnher, '"Out of the Loop': Autonomous Weapon Systems and the Law of Armed Conflict," Harvard National Security Journal 4, no. 2 (2013), pp. 231-81; Noel E. Sharkey, "The Evitability of Autonomous Robot Warfare," International Review of the Red Cross 94, no. 886 (2012), pp. 787-99; Robert Sparrow, "Killer Robots," Journal of Applied Philosophy 24, no. 1 (2007), pp. 62-77; Robert Sparrow, "Robots and Respect: Assessing the Case against Autonomous Weapon Systems," Ethics and International Affairs 30, no. 1 (2016), pp. 93-116.
(5.) Those few discussions of which we are aware include William Matthews, "Murky Waters: Seagoing Drones Swim into New Legal and Ethical Territory," Defense News, 9 April 2013, available at auvac.org/, and Donald P. Brutzman et al., "Run-Time Ethics Checking for Autonomous Unmanned Vehicles: Developing a Practical Approach" (proceedings of the Eighteenth International Symposium on Unmanned Untethered Submersible Technology [UUST], Portsmouth, NH, 2013), available at calhoun.nps.edu/.
(6.) As is often the case, science fiction is ahead of philosophy here, with a recent novel--P. W. Singer and August Cole, Ghost Fleet: A Novel of the Next World War (New York: Houghton Mifflin, 2015)--describing the near-future history of a war fought by autonomous robot ships and submarines. Singer is, of course, also the author of the most widely read book to date on the remarkable advances in (and policy challenges posed by) military robotics: Singer, Wired for War.
(7.) There is, it is important to acknowledge, a larger debate going on at the moment about the ethics of autonomous weapons per se, in which both authors are actively involved, albeit on different sides. We have deliberately undercommitted on this question in the current manuscript to concentrate on the ethical issues that might arise out of the use of UUVs and USVs in particular. If the use of autonomous weapon systems is unethical in and of itself, their use in war at sea will, of course, also be unethical.
(8.) Bruce Berkowitz, "Sea Power in the Robotic Age," Issues in Science and Technology 30, no. 2 (2014), pp. 33-40; Matthews, "Murky Waters"; U.S. Navy Dept., The Navy Unmanned Surface Vehicle (USV) Master Plan; U.S. Defense Dept., Unmanned Systems Integrated Roadmap, pp. 8, 80-91. A particularly interesting--and arguably problematic--category of AWSs would be vessels that were autonomous in some of their operations but were also staffed by human beings. Thus, we might imagine an autonomous submersible that navigated and chose targets autonomously but relied on onboard human engineers to maintain its mechanical and hydraulic systems. Similarly, we might imagine autonomous light attack craft that require human beings to carry out these roles. Finally, one might imagine vessels that were controlled by humans but that carried guns or missile systems that chose targets and fired autonomously (indeed, on some accounts any vessel that carries the Phalanx or Aegis system is already in this category). To our knowledge, there has been little discussion anywhere in the literature to date of the issues raised by these classes of systems.
(9.) Matthews, "Murky Waters."
(10.) Drones launched from land, ships, or submersibles clearly have tremendous potential in the context of war at sea. However, given that the ethics of the military uses of drones has been extensively discussed elsewhere, we will not consider them here except insofar as their activities may also be subsumed under our discussions of the ethics of attacks on vessels on or under the water.
(11.) "Sea Maverick UUV" "Sea Stalker UUV," and "Wave Glider," Naval Drones, www .navaldrones.com/; Clay Dillow, "Drones Come to the High Seas," Fortune, 11 April 2013, fortune.com/; "Reimagine Ocean Monitoring and Operations. Unmanned Robots Powered by Nature," Liquid Robotics, liquid-robotics.com/.
(12.) Jeremy Hsu, "U.S. Navy Tests Robot Boat Swarm to Overwhelm Enemies," Automaton (blog), IEEE Spectrum, 5 October 2014, spectrum.ieee.org/.
(13.) Gary Martinic, "Unmanned Maritime Surveillance and Weapons Systems," Headmark 151 (March 2014), pp. 86-91, available at www.informit.com.au/; Matthews, "Murky Waters"; "Spartan Scout USV)' Naval Drones, www.navaldrones.com/. Since 2009 Israel has deployed an armed USV, "The Protector," a nine-meter, four-thousand-kilogram-displacement, remotely operated vessel manufactured by Raphael. See "Protector Unmanned Surface Vehicle (USV), Israel," Naval-Technology.com, and Carl O. Schuster, "Drones Take South China Sea Plunge," Asia Times Online, 29 August 2012, www.atimes .com/.
(14.) See Scott Littlefield, "Anti-submarine Warfare (ASW) Continuous Trail Unmanned Vessel (ACTUV)," DARPA: Defense Advanced Research Projects Agency, www.darpa.mil/.
(15.) "Anti-submarine Warfare (ASW) Continuous Trail Unmanned Vessel (ACTUV) 'Sea Hunter,"' Naval Drones, www.navaldrones .com/. Scott Littlefield, at DARPA, was kind enough to clarify the revised date for the beginning of the trials of Sea Hunter, which were originally scheduled to begin in 2015.
(16.) UUVs may be divided up by tonnage/ displacement or by intended role; we have chosen the latter schema to better bring out the ethical issues that might be raised by operations in each role.
(17.) "Sea Maverick UUV"; "Sea Stalker UUV."
(18.) Alan M. Petrillo, "Navy Plans Fleet of Unmanned Underwater Gliders," AUVAC: Autonomous Undersea Vehicle Application Center, auvac.org/.
(19.) Fiona Keating, "Jellyfish 'RoboCop' Will Help Save the World's Oceans by Patrolling US Waters like an Aquatic Spy," DailyMail.com, 29 March 2013, available at www.dailymail.co.uk/.
(20.) "Remote Environmental Monitoring Unit System (REMUS)," Naval Drones, www .navaldrones.com/; "AUV System Spec Sheet; LMRS Configuration," AUVAC: Autonomous Undersea Vehicle Application Center, auvac org/.
(21.) Strictly speaking, the RMMV is semisubmersible rather than fully submersible. See "United States Navy Fact File: Remote Minehunting System (RMS)," America's Navy, www.navy .mil/.
(22.) An early proposal along these lines, Sea Predator, was later cancelled. For discussion of ongoing research into distributed networks and their potential for area denial, see Joshua J. Edwards and Capt. Dennis M. Gallagher, USN, "Mine and Undersea Warfare for the Future," U.S. Naval Institute Proceedings 140/8/1,338 (August 2014), available at www .usni.org/; Bryan Clark, The Emerging Era in Undersea Warfare (Washington, DC: CSBA, 2015); and Scott C. Truver, "Taking Mines Seriously: Mine Warfare in China's Near Seas," Naval War College Review 65, no. 2 (Spring 2012), pp. 30-66.
(23.) David Larter, "ONR: Large Underwater Drone Set for 2016 West Coast Cruise," Navy Times, 16 April 2015, www.navytimes.com/; Clark, The Emerging Era in Undersea Warfare, p. 13.
(24.) R. Scott, "ONR to Swim Ahead on ASW Package for Large UUV)' IHS Jane's Navy International, 20 November 2014.
(25.) See, for instance, U.S. Defense Dept., The Role of Autonomy in DoD Systems (Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, 2012), p. 85.
(26.) See, for instance, Brendan Gogarty and Meredith Hagger, "The Laws of Man over Vehicles Unmanned: The Legal Response to Robotic Revolution on Sea, Land and Air," Journal of Law, Information and Science 19, no. 1 (2008), pp. 73-145; Cdr. Andrew H. Henderson, USN, "Murky Waters: The Legal Status of Unmanned Undersea Vehicles," Naval Law Review 53 (2006), pp. 55-72; Rob McLaughlin, "Unmanned Naval Vehicles at Sea: USVs, UUVs, and the Adequacy of the Law," Journal of Law, Information and Science 21, no. 2 (2011), pp. 100-115; and Capt. Andrew Norris, USCG, Legal Issues Relating to Unmanned Maritime Systems (Newport, RI: Naval War College, 2013).
(27.) While any particular claim will inevitably be controversial, a plausible example in the military realm might be the use of nuclear weapons. That the law does not determine what is ethical is amply demonstrated by the long history of the legal toleration of practices (such as slavery) that we now acknowledge to have been profoundly morally wrong.
(28.) For another account of these differences, along similar lines, see Geoffrey S. Corn et al., The Law of Armed Conflict: An Operational Approach (New York: Wolters Kluwer Law & Business, 2012), pp. 418-19.
(29.) Note that we are here concerned primarily with war among ships and submarines, not fire from oceangoing systems directed at targets on the land or in the air.
(30.) War itself, more generally, has always been a collective endeavor, to be sure. However, the boundaries of the social collectivity in naval warfare are inevitably, if not exclusively, the physical confines of particular vessels.
(31.) This obligation is reflected in the International Convention for the Safety of Life at Sea, 1 November 1974, 32 U.S.T. 47, 1184 U.N.T.S. 278, chap. V, regulation 10(a), and United Nations Convention on the Law of the Sea, 1982, UN Doc. A/CONF.62/122 and Corr., art. 98(1). For a useful discussion, see Martin Davies, "Obligations and Implications for Ships Encountering Persons in Need of Assistance at Sea," Pacific Rim Law and Policy Journal 12, no. 1 (2003), pp. 109- 41. Walzer seems to suggest, in his discussion of the Laconia affair, that the duty of rescue applies only to noncombatants, and thus in the context of attacks on merchant shipping. Michael Walzer, Just and Unjust Wars: A Moral Argument with Historical Illustrations, 4th ed. (New York: Basic Books, 2006), p. 147. On the other hand, article 18 of the Second Geneva Convention of 1949 refers specifically to shipwrecked members of the armed forces--a matter that for decades complicated the formation of international law governing submarine warfare. Geneva Convention (II) for the Amelioration of the Condition of Wounded, Sick and Shipwrecked Members of Armed Forces at Sea, 12 August 1949 [hereafter Geneva Convention 1949], available at www.icrc.org/.
(32.) As we will see below, the interpretation of the italicized clause is central to the operationalizing of this duty and also to the question of its implications for unmanned systems.
(33.) The legal formulation of this duty, in article 18 of Geneva Convention 1949, specifies that it applies "after each engagement," but it is hard to see why this duty should lapse before or between engagements. This restriction is most naturally understood as acknowledging that parties to the conflict are unlikely to have the capacity to conduct rescue safely in the midst of combat rather than as denying the existence of a generalized duty to rescue. For some discussion, see Wolff Heintschel von Heinegg, "Submarine Operations and International Law," in Law at War: The Law as It Was and the Law as It Should Be, ed. Ola Engdahl and Pal Wrange (Leiden, Neth.: Nijhoff, 2008), pp. 160-61.
(34.) Walzer, Just and Unjust Wars, p. 147.
(35.) The legal right of warships to fly "false flags" during wartime complicates this claim somewhat when it comes to the challenges human combatants face. However, it is unlikely that autonomous systems will be relying on visual sightings of national flags to identify the nationality of vessels; they are much more likely to rely on acoustic signatures or radar silhouettes, which are harder to disguise.
(36.) W. Heintschel von Heinegg, "The Protection of Navigation in Case of Armed Conflict," International Journal of Marine and Coastal Law 18, no. 3 (2003), p. 402.
(38.) Thus, for instance, identification of legitimate targets in air-to-air combat is also arguably easier than in land warfare, while an obligation to provide assistance to those who are hors de combat may also exist in other extreme environments, such as deserts and snowfields.
(39.) There is also a debate about when and whether such systems can be considered "warships," especially in relation to the status of merchant shipping. See, for example, McLaughlin, "Unmanned Naval Vehicles at Sea."
(40.) Ibid., pp. 108-109, 112; Gogarty and Hagger, "The Laws of Man over Vehicles Unmanned," pp. 114-16; Heintschel von Heinegg, "Submarine Operations and International Law," p. 146; Henderson, "Murky Waters," p. 66; Norris, Legal Issues Relating to Unmanned Maritime Systems. McLaughlin thinks they should be granted sovereign immunity on the basis that they are "government ships operating for noncommercial purposes," even though he thinks it is a stretch to argue that they are themselves "warships." He agrees, however, that they are "vessels" under COLREGs. The question he raises--whether unmanned systems are "warships"--is an issue with implications mostly for the ethics of attacks on these systems rather than attacks by them, and as such is of less interest to us here.
(41.) Cdr. Chris Rawley, USN, "Return to Trust at Sea through Unmanned Autonomy," U.S. Naval Institute, www.usni.org/.
(42.) Gogarty and Hagger, "The Laws of Man over Vehicles Unmanned."
(43.) Henderson, "Murky Waters," pp. 68-69.
(44.) Gogarty and Hagger, "The Laws of Man over Vehicles Unmanned," pp. 117-18.
(45.) McLaughlin, "Unmanned Naval Vehicles at Sea," pp. 113-14.
(46.) Convention on the International Regulations for Preventing Collisions at Sea, 20 October 1972, 1050 U.N.T.S. 16, rule 5.
(47.) George R. Lucas Jr., "Automated Warfare," Stanford Law and Policy Review 25, no. 2 (2014), pp. 317-39.
(48.) Asaro, "On Banning Autonomous Weapon Systems"; Sparrow, "Robots and Respect."
(49.) Berkowitz, "Sea Power in the Robotic Age." Under the San Remo Manual, part 4, section 1, p. 79, "it is prohibited to use torpedoes which do not sink or otherwise become harmless when they have completed their run." Louise Doswald-Beck, ed" San Remo Manual on International Law Applicable to Armed Conflicts at Sea (Cambridge, U.K.: Cambridge Univ. Press, 1995). See also part 4, section 1, p. 82, on free-floating mines, which are prohibited unless they are directed against military objectives and become harmless an hour after being deployed.
(50.) The 1907 Hague Convention VIII prohibited the use of "automatic contact mines." However, as Heintschel von Heinegg notes, these principles "are generally recognized as customary international law and thus also govern the use of modern naval mines." Heintschel von Heinegg, "The Protection of Navigation in Case of Armed Conflict," p. 415.
(51.) "MK 60 Encapsulated Torpedo (CAPTOR)--Dumb Bombs," Federation of American Scientists, www.fas.org/.
(52.) The San Remo Manual (p. 169) notes that CAPTOR should arguably be considered a system capable of delivering a weapon rather than a weapon itself. Heintschel von Heinegg also argues that this system should be governed by the rules applicable to torpedoes. Heintschel von Heinegg, "Submarine Operations and International Law," p. 154.
(53.) Heintschel von Heinegg, "Submarine Operations and International Law," p. 149. However, they might play a useful role in supporting manned operations--as long as effective communications with the autonomous system could be maintained.
(54.) For a discussion of these obligations, see Doswald-Beck, San Remo Manual, pp. 172, 174-76.
(55.) Marcello Guarini and Paul Bello, "Robotic Warfare: Some Challenges in Moving from Noncivilian to Civilian Theaters," in Robot Ethics: The Ethical and Social Implications of Robotics, ed. Patrick Lin, Keith Abney, and George A. Bekey (Cambridge, MA: MIT Press, 2012), pp. 129-44; Noel Sharkey, "Autonomous Robots and the Automation of Warfare," International Humanitarian Law Magazine 2 (2012), pp. 18-19, available at www.redcross.org.au/.
(56.) Brutzman et al., "Run-Time Ethics Checking for Autonomous Unmanned Vehicles," p. 3.
(57.) John S. Canning, "A Concept of Operations for Armed Autonomous Systems" (paper presented at the Third Annual Disruptive Technology Conference, National Defense Industrial Association, Washington, DC, 2006), available at www.dtic.mil/.
(58.) Robert Sparrow, "Twenty Seconds to Comply: Autonomous Weapon Systems and the Recognition of Surrender," International Law Studies 91, no. 699 (2015), pp. 699-728; Protocol Additional to the Geneva Conventions of 12 August 1949, and Relating to the Protection of Victims of International Armed Conflicts (Protocol I), 8 June 1977, art. 41, available at www.icrc.org/.
(59.) The destruction of a crewed ship in these circumstances would generate disproportionate casualties.
(60.) McLaughlin, "Unmanned Naval Vehicles at Sea," pp. 105-106, offers a useful discussion of relevant considerations in this and similar contexts; see also Sparrow, "Twenty Seconds to Comply."
(61.) Doswald-Beck, San Remo Manual, p. 20.
(62.) Walzer, Just and Unjust Wars, p. 156.
(63.) Sparrow, "Robots and Respect." See also Heather M. Roff, "Killing in War: Responsibility, Liability, and Lethal Autonomous Robots," in Routledge Handbook of Ethics and War: Just War Theory in the 21st Century, ed. Fritz Allhoff, Nicholas G. Evans, and Adam Henschke (New York: Routledge, 2014), pp. 352-64; Human Rights Watch, Losing Humanity: The Case against Killer Robots (19 November 2012), available at www.hrw.org/; and Markus Wagner, "Taking Humans Out of the Loop: Implications for International Humanitarian Law," Journal of Law, Information and Science 21, no. 2 (2011), pp. 155-65.
(64.) Sparrow, "Robots and Respect."
(65.) Lucas, "Automated Warfare."
(66.) On the other hand, to the extent that it is difficult to predict whether a given munition will sink or merely damage a vessel, the number of combatant deaths likely to result from any given attack is harder to calculate in naval warfare than in land or air warfare.
(67.) For an extended discussion of the legal obligations on combatants in this regard, see Yoram Dinstein, The Conduct of Hostilities under the Law of International Armed Conflict, 2nd ed. (Cambridge, U.K.: Cambridge Univ. Press, 2010), pp. 197-217. See also Doswald-Beck, San Remo Manual, p. 15; Philippe Antoine, "International Humanitarian Law and the Protection of the Environment in Time of Armed Conflict," International Review of the Red Cross 32, no. 291 (1992), pp. 517-37; Richard Desgagne, "The Prevention of Environmental Damage in Time of Armed Conflict: Proportionality and Precautionary Measures," Yearbook of International Humanitarian Law 3 (December 2000), pp. 109-29; and Richard G. Tarasofsky, "Legal Protection of the Environment during International Armed Conflict," Netherlands Yearbook of International Law 24 (December 1993), pp. 17-79.
(68.) Walzer, Just and Unjust Wars, p. 156.
(69.) Arkin, "The Case for Ethical Autonomy in Unmanned Systems"; Ronald C. Arkin, "Lethal Autonomous Systems and the Plight of the Non-combatant," AISB Quarterly 137 (2013); Ronald C. Arkin, Patrick Ulam, and Alan R. Wagner, "Moral Decision Making in Autonomous Systems: Enforcement, Moral Emotions, Dignity, Trust, and Deception," Proceedings of the IEEE 100, no. 3 (2012), pp. 571-89. See Lucas, "Automated Warfare," for an account of the origins and components of this test of satisfactory robot behavior.
(70.) Sparrow, "Twenty Seconds to Comply"; Sparrow, "Robots and Respect"; George R. Lucas Jr., "Industrial Challenges of Military Robotics," Journal of Military Ethics 10, no. 4 (2011), pp. 274-95; George R. Lucas Jr" "Engineering, Ethics, and Industry: The Moral Challenges of Lethal Autonomy," in Killing by Remote Control, ed. Strawser, pp. 211-28.
(71.) Compare, for instance, Sparrow, "Robots and Respect," and Lucas, "Automated Warfare."
(72.) Ronald Arkin, Governing Lethal Behaviour in Autonomous Robots (Boca Raton, FL: CRC Press, 2009); Brutzman et al., "Run-Time Ethics Checking for Autonomous Unmanned Vehicles"; Leveringhaus and De Greef, "Keeping the Human 'in-the-Loop.'"
(73.) Doswald-Beck, San Remo Manual, p. 16.
(74.) Even if AWSs were as reliable as human beings at discriminating between legitimate and illegitimate targets, checking with a human being might nonetheless be a further reasonable precaution.
(75.) Arkin, Governing Lethal Behaviour in Autonomous Robots, pp. 29-30, 108-109; Arkin, "The Case for Ethical Autonomy in Unmanned Systems."
(76.) Adams, "Future Warfare and the Decline of Human Decisionmaking."
(77.) See, for discussion, Anderson and Waxman, "Law and Ethics for Robot Soldiers"; Aaron M. Johnson and Sidney Axinn, "The Morality of Autonomous Robots," Journal of Military Ethics 12, no. 2 (2013), pp. 129-41; V. Kanwar, "Post-human Humanitarian Law: The Law of War in the Age of Robotic Weapons," Harvard National Security Journal 2, no. 2 (2011), pp. 616-28; and Wagner, "Taking Humans Out of the Loop."
(78.) It might be possible, we suspect, to account for the duty of rescue as a function of the ethical requirement of proportionality within jus in hello. Rescuing combatants who have been rendered hors de combat by virtue of being lost at sea after an attack serves to reduce the number of deaths that are effectively surplus to the military advantage secured by the attack on their vessel. If this is true, a duty of rescue may also exist in war in other inhospitable environments, such as deserts or areas of extreme cold.
(79.) The San Remo Manual (and Additional Protocol I, art. 40) notes that "it is prohibited to order that there shall be no survivors, to threaten an adversary therewith or to conduct hostilities on this basis." Doswald-Beck, San Remo Manual, p. 15.
(80.) Heintschel von Heinegg, "Submarine Operations and International Law," p. 160.
Robert Sparrow received his BA from the University of Melbourne and his PhD from the Australian National University and is presently at Monash University. At Monash he is a professor in the philosophy program, a chief investigator in the Australian Research Council Centre of Excellence for Electromaterials Science, and an adjunct professor in the Centre for Human Bioethics. He has authored numerous papers and book chapters on the ethics of military robotics, just war theory, human enhancement, and nanotechnology. He is a co-chair of the IEEE Technical Committee on Robot Ethics and was one of the founding members of the International Committee for Robot Arms Control.
George Lucas received his BS from the College of William and Mary and his PhD from Northwestern University. He is professor emeritus at the U.S. Naval Academy and the Naval Postgraduate School. He is the author of the book Military Ethics: What Everyone Needs to Know.
|Printer friendly Cite/link Email Feedback|
|Author:||Sparrow, Robert; Lucas, George|
|Publication:||Naval War College Review|
|Date:||Sep 22, 2016|
|Previous Article:||Distributed STOVL operations and air-mobility support: addressing the mismatch between requirements and capabilities.|
|Next Article:||A thousand splendid guns: Chinese ASCMs in competitive control.|