Improving our view of the world: police and augmented reality technology.
One of the 21st century's emerging and potentially powerful technologies is Augmented Reality (AR). Sports fans view one popular use of AR technology each weekend: the major networks augment their coverage of professional football by superimposing yellow "first-down" lines over the gridiron. While not interactive, this is AR in its simplest form. Advanced AR overlays virtual (computer-generated) images onto a person's real-world field of vision or into a real-world experience in a way that improves and enhances the ability to accomplish a wide variety of tasks and assignments. Situational awareness is greatly improved, theoretically allowing one person equipped with AR technology to do the same amount of work as three unequipped individuals.
While still years from perfection, wearable AR units are currently under development: the Naval Research Laboratory is working to develop the Battlefield Augmented Reality System (BARS). The BARS will be a wearable device incorporated into a soldier's personal equipment load for the purpose of enhancing his or her combat capability by gathering and displaying realtime intelligence, overlaying surrounding geographic information such as maps and floorplans, presenting high priority sensor data, providing improved optical capabilities and maintaining constant communications with team members and higher headquarters.
AR technology will also have a dramatic impact on policing, creating innovative methods for combating crime and terrorism. It will also afford criminals and terrorists new opportunities for exploiting, disrupting and harming society. In order to be used effectively, police personnel need to fully understand current AR capabilities and what will emerge in the coming decades. Some possible AR applications for policing include:
* Real-time language translation along with data on cultural customs and traditions,
* Real-time intelligence about crimes and criminals in the patrol area,
* Facial, voice-print and other biometric recognition data of known criminals to allow instantaneous identification,
* Integration of chemical, biological and explosive sensors to immediately notify officers of any local contamination and recommend appropriate protective measures for themselves and the public,
* Scalable, three dimensional maps, complete with building floor plans, sewer system schematics, public utility information and public transportation routes, accessed at will to improve situational awareness and response to problems, and
* Patrol car operator data and regional traffic management information on a heads-up display to make driving safer and more efficient, especially in pursuit and rapid response situations.
* Improved situational awareness during dynamic and dangerous incidents, enhanced cohesiveness between team members and better coordination with command personnel to make SWAT operations safer and more effective,
* In tactical situations, modulates the audio effects of gunshots (both hostile and friendly) to enhance concentration while providing the user with superior hearing capabilities over long distances,
* Advanced optics to provide zoom, thermal and infrared imaging for the location and apprehension of fleeing criminals, buried or concealed disaster survivors, or missing persons,
* Identification Friend or Foe (IFF) technology, worn by every police officer, to reduce or eliminate friendly fire casualties by visually, audibly and/or haptically highlighting fellow police officers both on and off-duty, and
* Interface of human-machine components that extend human capabilities and "presence" to remote locations.
* Enhanced ability to gather information, follow leads and visualize large amounts of data in real-time to solve crimes and more quickly identify and capture dangerous criminals and terrorists,
* Speaker recognition capability to provide investigators the ability to accurately match voices against known criminals,
* Advanced optics to allow investigators to lip-read from great distances in situations where listening devices are impractical,
* The use of AR video, audio and sensing devices used to visualize blood patterns, blood stains and other sensor-detectable forensic data available at crime scenes,
* Automatic sensor readings that calculate distance and height and directly create digital and AR maps for court presentation, and
* The coordinated use of robots, unmanned aerial vehicles (UAVs) and police officers managed through an AR network to enhance surveillance activities.
* Realistic training scenarios to simulate dangerous police environments while blending real-world equipment and fellow trainees into the scenario.
* Real-time monitoring of patrol activities through the use of a video/audio feed from the street,
* Display of location, activity and status information projected on a 3 dimensional map of the community,
* Supervision of critical incident response to include the monitoring of the physiological status of all personnel, permitting the assignment of dangerous tasks to those who are mentally and physically best able to perform, and
* Coordination of widely dispersed units through the use of visual, audible and haptic cues from the supervisor.
In addition to the technological hurdles still to be overcome before AR can be deployed as a viable policing asset, other factors must be considered as well: Will the public accept its use? What affect will the appearance of "wired up" officers have on the community? Will the benefits that AR offers be perceived by the street officer to offset its costs (e.g., less autonomy)? How will AR systems be financed, especially in smaller agencies? How will the information loaded into AR systems be scrutinized in order to ensure accuracy? Will there be any long-term medical effects associated with the use of AR systems? What legal issues will be raised? How can the use of AR by criminal or terrorist organizations be anticipated/prevented?
The profession's customary luxury of playing 'catch-up' to technological advances is no longer a viable option. The accelerated speed of change in the information age requires police agencies to adopt a new approach and redefine missions in new, strategic terms. Otherwise law enforcement will forfeit the advantage of technology to criminal networks and dedicated terrorists, at too high a cost to public safety and confidence.
The purpose of this research paper is to provide the policing profession an introductory source document and overview of Augmented Reality (AR). Fundamental principles and components of the technology are examined, along with research developments occurring today that have the potential to directly enhance individual human performance through the augmentation of reality. The implications of AR and some potential applications for its future use as a law enforcement tool are offered. As a source document for police officers it does not present new scientific research or highly technical data. It is intended to provide a starting point for the development of practical and beneficial policing applications through the use of AR.
Much has been written since the September 11th, 2001, terrorist attacks about how the world has changed. Politicians scramble to do something--anything--to assuage a growing level of fear and uncertainty about the future. Pundits speculate and pontificate about what is needed to make our world and country safer. Much time has been spent discussing the role technology should and can play in the war on terrorism. In the face of this new threat the nation's police--federal, state and local--are viewed as untrained, under-equipped and ill-prepared. Unprecedented efforts to repair the technical deficiencies and strengthen law enforcement's capacity to fight terrorism are straining our concept of liberty and causing us to closely examine our definitions of privacy and constitutional rights.
There is no escaping the fact that we live in a rapidly changing society driven by higher and higher levels of technological advancement. These new technologies are creating newer, greater and more powerful capabilities for humanity, offering opportunities for change that are fundamentally deeper than additional hours of relaxation and greater economic prosperity through innovative and fancy gadgets. New capabilities are changing the way people view the world and their place in it.
In his book Profiles of the Future: An Inquiry into the Limits of the Possible, Arthur C. Clarke (1984) said, "Any sufficiently advanced technology is indistinguishable from magic." Many of today's new and emerging technologies appear magical to those ill equipped for rapid change. These new technologies are changing not simply the world, but humanity's view of itself, giving people the ability to redefine the limitations and capabilities of the species, changing the nature of what it means to be human.
Yesterday's science fiction is rapidly becoming a very real part of everyday life. In many cases only the commitment of resources and engineering effort are needed to make seemingly fantastic scientific theory a reality. It is also clear that the rate of change is rapidly increasing. As information and communications technologies permeate the global community the level of collaboration among scientists, engineers and entrepreneurs speeds the development of new manufacturing techniques and consumer products. Traditional corporate governance models with centrally controlled hierarchies are being replaced by more flexible networked organizations capable of instantaneous reaction to changing markets (Denning, 2002, p. 1-14). Crosspollination among dissimilar disciplines, from genetics and biotechnology to computers and nanotechnology, also aids development across the technical and scientific spectrum and further increases the rate of advancement. Finally, a growing consumer population with an everincreasing lifespan, hungry for things to ease its members' lifestyles, improve their standard of living, increase their leisure activities and overcome or mitigate common disabilities brought on by injury or disease, is driving innovation while continuously expanding the boundaries of traditional cultural norms (Broderick, 2001).
Raymond Kurzweil, technology entrepreneur, winner of the 1999 National Medal of Technology and author of the book The Age of Spiritual Machines: When Computers Exceed Human Intelligence, has defined what he calls the "Law of Accelerating Returns." Kurzweil's research demonstrates that technological advancement has been occurring at an exponential rate throughout the course of history, though until the later half of the 20th century the increase has seemed slow due to the initially small incremental advances on the flat side of the exponential curve. According to Kurzweil, in the late 20th century we entered the steep portion of the curve, the point at which the exponential trends begin to climb very rapidly (Kurzweil, 1999).
If Kurzweil and others are correct, traditional predictions about the impact of coming advances that rely on comparisons with previous rates of change in prior eras are inadequate. The rate of change is changing, compressing significantly the time it takes for advancement to take hold and alter the way in which people live and interact with one another. Based upon his Law of Accelerating Returns and barring a global catastrophe or extreme level of international conflict, we could see 100 years of equivalent 20th Century technological advancement in the first decade of the 21st century and 25,000 years of advancement before the dawn of the 22nd (Kurzweil 2001). The impacts on society will be enormous, creating increased tension between nations, communities and interest groups and opening new opportunities for criminals and terrorists to exploit those tensions to their own advantage.
The nation's police departments, the majority of which have fewer than 50 sworn officers, continue traditional operations despite a rapidly changing social and scientific landscape. Working in small communities with limited and overlapping jurisdictions and dependent upon austere local budgets, most police departments struggle to provide even the most basic traditional investigative and uniformed-response services. Computer technology for a large percentage of the nation's smaller police departments is extremely limited.
According to the Bureau of Justice Statistics, Law Enforcement Management and Administrative Statistics for local police departments, in 1999 55% of departments did not employ personal computers in criminal investigations and only 31% gave officers access to mobile computers in the field (U.S. Department of Justice, Bureau of Justice Statistics, Local Police Departments 1999, 2001). Actual hardware employed is only part of the picture. This same report failed to capture any statistics on training requirements or the number of hours allotted to computer training within those departments that actually use computers, even though training is a huge factor in the effective employment of any technology.
Within this organizational environment advanced technologies that offer new crimefighting capabilities are an unaffordable luxury to most police executives and are viewed by officers as a distraction from their fundamental and more traditional law enforcement roles of patrol and investigations. Many police officers and police leaders are simply not concerned or equipped to deal with rapidly changing technology. Instead, they value most what they know, favoring the way they have always operated and the tools they have used with which to operate (NYSTEC, 1997, p. 51).
Falling farther and farther behind the technological curve, many police departments live in a constant state of trying to respond to current problems while making small and incremental improvements to their traditional business practices. In a rapidly changing world, however, each new breakthrough in technology adds to overall system complexity making it more difficult both to adapt to current realities and to effectively comprehend subsequent technological developments (Brown, 2000). This situation forces police personnel into a reactive mind-set, hoping to stay current while striving to maintain some relevance to the citizens they serve.
For example, decades after many of the nation's more progressive police departments began using Mobile Data Terminals (MDTs) in patrol cars and years into the widespread use of personal communications devices by the public, many departments have yet to realize the potential capabilities afforded by mobile computing (US DOJ, BJS, State and Local, 2001). These capabilities include immediate access to motor vehicle and criminal history data, streamlined records submission linked to computer-aided dispatch systems and automatic vehicle location to improve officer safety and coordination of resources. These are only some of the more basic tools available to modern departments that make them effective and capable of combating criminals and terrorists in the information age, though they are still under-used or non-existent in many police agencies.
Indeed, while much of the rest of the world entered the digital age in the 1990s and radically altered its business processes and methodologies to take advantage of new technology, many police departments, especially smaller more rural departments, have changed very little. Many are conducting internal police business and external police operations in essentially the same way it was accomplished decades earlier.
The events of September 11th and law enforcement's reaction to them have only heightened the potential for increased disparity between high-tech criminals and low-tech police, between departments that have embraced technology and those that have ignored it. Not only have the complexities of crime fighting increased with the imminent danger of nuclear, biological and chemical terrorist attacks on major metropolitan areas, but the tools and training necessary to deter or respond to those attacks is also much more extensive, complex and time consuming. While human intelligence and traditional police work are still important aspects of law enforcement in the terrorist arena, high-tech systems are becoming a principal and very necessary means of maintaining domestic security.
As these critical technological systems become increasingly capable, the level of knowledge, education and training required to operate them effectively increases correspondingly, along with the educated insight and wisdom needed to employ them against crime and terrorism in a democratic society. Determining how those same systems and technologies might be used to commit acts of terrorism, and thereby thwart them, requires not only knowledge of the technologies but also imagination and vision about how they might be used against our communities.
The critical component to effective policing in a rapidly changing world is this ability to think creatively about emerging technologies and how they can be used successfully within the constitutional limitations of a free society. Many new devices give us the ability to do what has always been done better or faster or more efficiently than in the past. In addition, those applications are relatively easy to discern, though even then can take many years to implement. However, many times old processes and ways of thinking need to be abandoned completely because new technologies create opportunities to accomplish things never before attempted or contemplated (Tristram, p. 54).
This is particularly true as we enter an era where previously supposed science fiction is becoming reality--where many emerging technological capabilities appear to the non-technical mind as magic and where the criminal and terrorist elements in the world have the creativity, capacity and willingness to use technology to their destructive advantage. The detailed knowledge and understanding of the September 11th terrorists regarding the various components, strengths and weaknesses of the American airline industry, including the knowledge to actually fly and navigate a jumbo jet, allowed them to exploit a highly technological system to perpetrate the largest terrorist event in American history. Emerging technologies will offer similar and more lucrative opportunities in the future. Unless law enforcement can think creatively and understand the capabilities and weaknesses of those same technologies police departments will be blind to the dangers or incapable of safeguarding their citizens.
This kind of thinking and understanding is impossible to attain without a concerted and continuing effort to be fully informed about emerging technologies. Police agencies must stay abreast of near-term and future developments and create new strategies and scenarios about how and when various technologies will and could be used, by criminals and law enforcement alike. No longer can they be concerned only with side arms, patrol cars and street survival tactics. No longer can police executives afford to train their people only in traditional response-oriented ways and organize around centrally controlled bureaucracies. The Brave New World (Huxley, 1932) of the 21st century is changing too rapidly, spurred on by radically new and different technological devices, systems, applications and processes. The growing interrelationships and synergistic effects between once dissimilar technologies is further complicating the process of grasping future trends and imminent technical possibilities, making real-time research and analysis of our changing world a requirement for effective policing.
This paper is an examination of just one rapidly emerging technology arena. It is an assessment of the field of AR and its potential applications to policing. AR is an emerging technology focused on integrating virtual (computer generated) objects (sights, sounds and feeling) into the real-world experience, aligning both the real world and virtual objects with each other in a complementary manner (Azuma, Baillot, Behringer, Feiner, Julier, MacIntyre, 2001, p. 34).
Unlike Virtual Reality (VR) where the user is totally immersed in a computer-generated virtual world and Augmented Virtuality (AV), where real world objects are included as part of a virtual simulation, AR combines real and virtual objects and displays them in real time to an individual operating in the real environment in a way that is beneficial to accomplishing specific tasks or missions. Figure 1 illustrates the relative position of AR in the continuum of reality environments and the overall relationship between reality, virtuality and augmentation (Milgram, Kishino, 1994).
[FIGURE 1 OMITTED]
AR brings together a variety of technologies to supply immediately relevant information to individuals in a way that is instantly applicable to a given task or situation, utilizing virtual graphics, textual annotations, auditory information and haptic (touch) sensations in a coordinated real-time presentation. Any situation that would be enhanced or helped by the visual, audible, or haptic "display" of information not available or detectable by normal human senses may be positively influenced by the use of AR technology (Robinett, 1992).
Information or objects within an AR environment can be strategic, tactical and technical in nature and accessed from several different sources. AR information can be transmitted wirelessly from a centralized computer network, accessed directly from a wearable computer carried by the individually equipped AR user, acquired from purposely embedded devices within a surrounding intelligent environment and acquired from an array of AR sensors scanning the immediate or visible location of the user. The information is then projected onto a see-through heads-up display, transmitted audibly to a headset, or felt through a haptic interface like a glove.
AR technology can also be used to block out certain aspects of the real world that might either detract from the task or mission at hand, a process known as "diminished reality", or allow the user to filter extraneous or confusing information and see things not normally visible to the unaided eye, a freeze-frame process called "mediated reality." Of particular interest to law enforcement is the apparent benefit of enhanced memory of certain details (e.g., facial features) while using a mediated reality visual memory prosthetic (Mann, 1996).
One popular use of AR technology is viewed by millions of sports fans watching broadcast television each weekend. The major sports networks have been augmenting their coverage of events like auto racing and professional football, using annotations with driver and speed information tagged to race cars hurtling around the track and yellow "first-down" lines superimposed on the gridiron to help fans better understand the real-time action of the event. While not interactive, this is AR in its simplest form.
Another easily recognizable form of AR technology is the heads-up display (HUD) used by fighter pilots. A HUD superimposes real-time aerodynamic and aircraft system data onto the forward view of the cockpit canopy, allowing the pilot to see critical information about the current flight situation while maintaining focus on the enemy target. While not a wearable form of AR, HUD/AR technology could be beneficial for police vehicles, particularly those involved in high-speed pursuits or surveillance operations.
A more robust example of AR technology is real-time facial recognition. A user wearing an AR system containing a dataset of business and personal contacts (or a police officer with access to a known-criminal database) that included facial recognition features would always know the names and associated information of people in his or her presence that are matched within that database. Upon approaching any person the AR system could automatically capture and compare their facial or biometric features and if found, superimpose a heads-up textual annotation of the person's name and available statistics in the user's field of view, or provide an auditory announcement into an earphone. This very basic example of a single AR application is not possible with today's state of facial recognition and biometric technology or the current processing power and storage capacity of wearable computers, but should be achievable within the next few years given the current rate of technological progress.
An even more complex AR application now under development explores the transition between Mixed Reality states, i.e.--real environment to AR to virtual reality. Called the "Magic Book" research project within the Human Interface Technology Lab at the University of Washington, researchers are developing a system that allows readers to view three-dimensional animated virtual scenes as part of their book reading experience. Multiple readers experience the same book simultaneously and can choose to enter the animated scenes and "walk-around" in a completely immersive virtual reality. This on-demand seamless transition between reality and virtual reality while interacting with programmed and optimized devices or intelligent environments has a wide range of applications in education and training, entertainment and engineering and architectural design (Magic Book: Project Description).
Police applications of Magic Book-type systems might include crime scene investigations and raid rehearsals for tactical operations. These more complex applications, along with the incorporation of multiple sensory inputs and alternate display applications within the same AR system, will require additional advances in computing, storage and display technology, all of which should mature rapidly over the coming decade and make possible much more intricate and dynamic AR systems.
AR systems consist of a variety of technical components that have the ability to enhance human physical and mental performance. Fundamentally, an AR system consists of a wearable computer, a Head Mounted Display (HMD), tracking and sensing devices, along with advanced software and virtual 3D rendering applications. Depending on the specific functional application the basic system could incorporate wireless transceivers, GPS location devices, laser range finders, hearing and acoustical devices, environmental sensor arrays, advanced optics, haptic interfaces and a host of other integrated hardware and software components adapted for specific functions (McAllister, Nyland, Popescu, Lastra, McCue, 1999).
The technology can utilize database and image information stored on the wearable computer, analysis software to interpret real-time optical and sensor input, as well as information gathered from remote sources linked by wireless connections. In the future, information will be available directly from "intelligent environments," specifically designed computing systems, microprocessors and programmable chips intentionally embedded within physical spaces that will use wireless links to communicate with AR equipped individuals and other sensing devices to provide information about the local area, buildings, or landscape (Shafer, Brumitt, Cadiz, 2001).
This paper does not consider all of the possible technical developments that may impact or improve the applicability of AR to policing. The organizational improvements in the use of information technology, local and wide area networks and infrastructure-based wireless systems including short-range wireless technologies, while critical to the effective future employment of AR, were also excluded from this study. While these related technologies and associated research projects are fundamental enabling mechanisms and will be a large factor in the ultimate success and practical viability of AR, they do not constitute the main body of AR technology.
AR is an emerging field of study, loosely coordinated, incorporating many different scientific and engineering research efforts to accomplish a wide variety of tasks or achieve widely divergent technical or practical results. The authors of this paper gathered information from many different engineering and government resources, scientific and academic research papers and projects and popular technology-oriented literature to develop a comprehensive picture of AR technology, various AR initiatives and how they might impact the law enforcement profession over the next ten years and beyond.
Much of the information and research documentation was derived from government, commercial and academic websites and resources available on the Internet. In several cases, email and telephone discussions with researchers involved in various technology projects were conducted. Field research was conducted on the Battlefield Augmented Reality System (BARS) being developed at the Naval Research Lab in Washington, D.C. under the direction of Dr. Simon Julier. (1)
Drawing on these resources and the information provided by Dr. Julier it was possible to develop a comprehensive picture of AR technology today, where it is heading in the near future and anticipate the most likely near-term developments. This picture makes it possible to derive some direct applications to policing, particularly when coupled with other related technologies capable of enhancing its effectiveness and applicability to police officers.
Research results are organized as follows: Section 1 describes the technology--wearable computers, display devices, sensors for tracking head position and location, control/input technology and various AR-related technologies such as haptics, sonification, enhanced optical devices and sensors-- that will potentially increase the effectiveness of AR; section 2 is a synopsis of the Naval Research Lab's BARS; section 3 discusses some potential police applications for AR technology; and section 4 outlines some implications for policing.
1. AR TECHNOLOGY
Basic AR systems are composed of three major elements: A wearable computer; Human-Computer Interface devices including a Head-Mounted Display; and a suite of sensors. These categories contain a wide variety of hardware and software components, including location and tracking devices for precise graphics registration and a lightweight power unit. All of this must be integrated into a wearable package that affords ease of mobility for the user (see figure 2).
[FIGURE 2 OMITTED]
a. Wearable Computer
One of the fundamental components of AR is the wearable computer system used to render virtual images, process and store a wide array of incoming and outgoing data and integrate AR system components. The computer must be capable of high-resolution and highspeed processing that renders high-performance 3D graphics in real-time (Feiner, 1997). Because the wearable computer is physically connected to the user by direct contact, it is capable of engaging the physical senses to enhance his or her performance. As such, it can also enhance the performance of others networked together with the AR user (Smailagic, Siewiorek, 1999).
In order for AR to be practical for police use the system has to be mobile, lightweight and compact, conforming to the user's body in a way that makes it unobtrusive and non-hindering to the employment of other police equipment. It must also be rugged, capable of withstanding extremes of cold and heat as well as the rigors of street police work. Research labs and a handful of commercial manufacturers are developing wearable computers that could form the basis of a prototype police AR system (see figure 3).
[FIGURE 3 OMITTED]
Some of today's commercial wearables use handheld screens and are not sold with seethrough HMDs suitable for AR applications, but could ultimately serve as AR platforms with some design modifications. Many of these systems have the capability for incorporating a variety of user interfaces, wireless links and alternative displays customizable for different uses and applications. (2) The current commercial market for wearable computers is relatively small; hence, prices are high and capabilities remain limited. However, the military and various research institutions such as the Massachusetts Institute of Technology (MIT) Media Lab and the University of Toronto have been exploring the use of wearable computers for many years (Mann, Niedzviecki, 2001). As user-centered designs make wearable computers more accessible to the general public, the speed, capability and usability of wearables will increase, making them ever more suitable for policing applications. International Data Corporation, an IT research firm predicts that the market for wearable computers will be $600 million by 2003 (Kiser, 2000).
One of the more critical aspects of wearable computers is battery power and the size, weight and power duration of current battery technology. Here, too, great strides are being made, both in the commercial and military arenas over a wide range of conventional and novel approaches to power generation, from throw-away and rechargeable batteries, micro-fuel cells and thermo (body heat) generators, to Synthetic Multi-Functional Materials (SMFM) that weave electrical storage capability directly into wearable fabrics (Defense Advanced Research Projects Agency-Synthetic Multi-Functional Materials; Micro Chemical and Thermal Systems-Compact Fuel Processors).
b. Human-Machine Interface
The ability to smoothly and easily interface with an AR system is central to its effectiveness. It requires the uncomplicated input to the system of our real-world desires and intentions (commands), then the receiving of easily understood data output back from the system. Traditional methods of computer input control (e.g., keyboard, mouse, trackball, etc.) and output receptors (e.g., monitors and flat-panel displays) are difficult, if not impossible to use in a mobile environment. To be effective for police use, AR systems need to use a combination of modern input/output devices, features and methodologies that allow the user to concentrate on real-world tasks while seamlessly enhancing the real-world experience with additional and useful data.
Flexible keyboards and tactile controls woven into clothing or worn on the body are an improvement over standard and bulky computer keyboards (though the distracting requirement to focus attention on the device and away from one's surroundings remains.) Some AR researchers have experimented with handheld devices such as personal digital assistants (PDAs) and Tablet portable computers (PCs) using pen-based "Graffiti" hand-recognition software for the input and output of information, but these too require additional focus on the device and away from the user's real-world environment (Feiner 1999).
Consciously having to control various input devices in order to manage an AR system ultimately detracts from its primary purpose, that of enhancing the user's ability to accomplish a mission other than simply operating the wearable computer. There is a large amount of research currently being conducted in the field of user-system interaction to improve our ability to control mobile computers and wearable technology in order to overcome current limitations (Rauterberg, 1999).
i. Head Mounted Display (HMD)
There are two basic types of HMDs that are suitable for mobile applications: The optical see-through display (figure 4) and the video see-through display (figure 5).
[FIGURES 4-5 OMITTED]
Optical see-through displays place partially transparent screens in front of the user's eyes, allowing the user to see the real world through the screen and view virtual images or data projected onto it. Video see-through displays use opaque screens in front of each eye combined with head-mounted video cameras. The cameras capture the real-world view and project it onto the opaque screens after combining it with virtual images or data (Azuma, 1997).
HMDs are also manufactured in monocular or one-display models (figure 6) leaving one eye completely unobstructed while the other eye observes the video or optical display.
[FIGURE 6 OMITTED]
Two-display HMDs in either binocular or stereoscopic (binocular) models allow the user to see virtual and real-world images using both eyes. Two display models allow for a wider consolidated field of view, with stereoscopic models creating a more immersive and threedimensional view, while one-display models are smaller and less obtrusive (MacIntyre, Feiner, 1996). There have also been concerns regarding adverse physiological effects from using monocular HMDs, particularly in younger users (Peli 2000).
A relatively recent and interesting development in HMDs is the Virtual Retinal Display (VRD), a head-mounted device that projects virtual images directly on the user's retina using low power laser light that is completely safe to the user. A VRD eliminates the problems associated with bright lights washing out the virtual image in a see-through optical HMD, producing high resolution, full color images under any light condition (Virre, Pryor, Nagata, Furness, 1998). Other commercial systems under development that could be used for AR systems include lightweight LCD HMDs used for video games and movie viewing and eyeglass displays that project images directly onto a standard pair of eyeglasses (figure 7)(Azuma et al., 2001).
[FIGURE 7 OMITTED]
Generally speaking there are compromises and trade-offs associated with all the various types of HMDs. Optical see-through HMDs have certain advantages over video see-through devices. Optical see-though allows the user to see the real world directly, without visual degradation, whereas video see-through captures the real world through video cameras and must blend the real and virtual images onto the HMD's opaque screen, thereby reducing the resolution of the real world view. This video blending process also adds tens of milliseconds of delay between real-world action and its display on the HMD, which could be problematic in police situations. Inadvertent power loss can also be hazardous with video see-through HMDs, making the user effectively blind. With optical see-through, a loss of power has no effect on the user's real-world view. Finally, because the cameras of a video see-though HMD are offset from the user's line of sight, they cause a discrepancy between what the user sees and what is actually being experienced (Azuma 1997).
Conversely, video see-through displays have several advantages over optical see-through devices. Video see-through is more flexible in its ability to merge real and virtual images, making the combined display image more realistic. Video see-though can selectively block some real world images such as bright headlights or sunlight. Because of the inherent difference in an optical see-through device between the real world view and the virtually displayed image that is processed through the wearable computer, optical see-through cannot exactly align real and virtual images. Video see-through compensates for the delay by matching and displaying both the real and virtual at the same time (Azuma, 1997).
ii. Haptic Feedback
Haptics will increase the effectiveness of AR systems by connecting the augmented experience to another key human sense--touch (MacIntyre, Feiner, 1996). Haptic or touch feedback, primarily generated through the use of specially designed gloves, provides another avenue for sensory data reception that will increase the situational awareness and add context to the AR experience. Haptics can be used as a human-machine input device, tracking hand and finger motion and providing positive feedback to control virtual or highly flexible keyboards that do not provide positive feedback. More important to AR and policing, haptics can be used to feel or manipulate objects, such as doors or walls, or control devices, such as a robot, that are remote from the AR user (Boeing Wearable Computer Workshop, 1996).
iii. Virtual Interfaces
Virtual interfaces are possible within an AR environment because the user is already viewing virtual images projected onto the HMD. Instead of having to change the visual orientation to manipulate a real keyboard or mouse, an AR system projects a virtual input device onto the user's display, allowing electronic manipulation using haptic feedback. This Visual Panel can be programmed to resemble a conventional keyboard or mouse, or any custom interface of the user's choosing (Wu, Shan, Zhengyou, Shafer, 2000).
iv. Voice Activation/Speech Recognition
Voice activation and speech-recognition technology is rapidly maturing and will streamline much of the manual input processes used today, while allowing user concentration to remain on the task at hand. Voice commands are ideally suited to abstract situations and speech is the most natural form of human communication. Therefore, voice activation technology will allow AR users to input commands naturally and intuitively without the distraction of looking at a separate input device (Everett, Wauchope, Perez-Quinones, 1998). Current state-of-the-art speech recognition applications achieve very reasonable accuracy in controlled situations, though research to refine accuracy and improve other issues applicable to dynamic police operations remains underway (Everett et al., 1998; Young 2001).
Sonification is the use of non-speech audio transmission to convey information. More specifically, sonification is the translation of non-audible data into an acoustic signal in order to facilitate the rapid interpretation of that data. Instruments such as the Geiger counter, sonar and tone signaling on early police radar are examples of sonification. The major benefit of this technology is that it can be used to help the operator interpret data from real-world observations, achieve better overall situational awareness and engage in rapid decision making (Kramer et al., 1997).
vi. Advanced Human-Machine Interaction Methods
More dramatic changes in human-machine interaction are being developed that are directly applicable to AR systems. Affective Computing is a process in which the physiological patterns of the wearer--heart rate, blood pressure, temperature, emotional state, etc.--are sensed by the wearable computer, which then responds in a way appropriate for that particular user in that particular situation. For example, an AR wearable computer could be programmed to begin recording visual and audible events when a police officer's physiological state reaches a certain level of excitement indicating stress or fear, obviating the need for the wearer to manually begin recording the event. If the user becomes injured or incapacitated, the AR system could automatically transmit detailed physiological and situational information to a command center for action (Picard, Healey, 1997).
Support Vector Machines (SVM) are a type of computer learning system that, according to Professor Rajeev Sharma of Penn State University, can be trained to screen for certain data in order to produce an object classification. SVM is being developed now as a method to identify gender using voice and facial recognition technology, which then tailors gender-based digital content to users of such things as information kiosks. The system analyzes data and distinguishes the sex of an observed subject by comparing aural and visual information to a pair of previously defined choices (Bowen, 2002). In the future, an SVM equipped AR system could be used to distinguish between a number of different sensor inputs and speed human-computer interaction processes.
AR systems could also benefit from a concept called Augmented Cognition. Working with DARPA, the MOVES Institute of the Naval Postgraduate School is developing the Context Machine (CM), a device that utilizes AR to provide an overall and comprehensive situational context to a user. Combining things such as location, the presence of other people and objects, the actions presently occurring, the user's goals and other situational components, the system discerns the user's "context" (3) and determines how best the subject can maximize efficiency in the performance of particular tasks; in effect, the CM allows a single individual to successfully accomplish the functions of three or more individuals (DARPA, The Context Machine, 2001).
There are a variety of other potential projects that could similarly assist AR users with input/output tasks and increase situational awareness, such as Task-Driven Computing, a DARPA funded program to let users interact with their computing environments in terms of high-level tasks, freeing them from low-level or manual configuration activities (Wang, Garlan, 2000); Project Aura, a Carnegie Mellon project to provide users with an invisible halo of computing information services that persists regardless of location (Project Aura); and Guardian Angels, holographic digital assistants that can provide information in the form of a conversation with another person. There is also promising research in the area of Brain Activity Scanning and direct neural implants, in which the thoughts of a human user control the activities of the computer (Broll, Schafer, Hollerer, Bowman, 2001). All of these concepts are under development and some or all of them could eventually be used to enhance AR systems for police use.
c. AR Tracking
One of the biggest hurdles that AR systems must overcome before widespread police use is the difficulty in sensing and tracking the location of the user over long ranges relative to the real world (Azuma, 1997). In order for virtual images to be correctly and accurately overlaid or registered onto the user's real world display, the AR system must have the capability to continuously track the user's head movements, exact position, viewing direction and real world orientation (Bajura, 1997). AR systems employ a variety of mechanisms to accomplish this tracking, such as mechanical, magnetic, acoustic, inertial and optical sensors, or a hybrid combination of several of these technologies. The tracker must be capable of providing rapid, accurate and reliable data in an unobtrusive package or the virtual images will be incorrectly displayed, creating disturbing or harmful effects (Allen, Bishop, Welch, 2001).
Of particular concern to policing, the tracking system must be accurate outdoors in open terrain as well as indoors. Outdoor tracking is a much more difficult problem for AR systems and depends very heavily on GPS, dead-reckoning techniques, compasses and gyroscopes to achieve accurate image orientation in unprepared environments (Azuma et al., 2001). Without highly accurate tracking in all types of unprepared environments, including the transition from indoors to outdoors, AR cannot be a practical tool for policing. Sensing the entire environment in real-time using a hybrid tracking system to accurately determine the location of the user as well as natural and manmade terrain features is a necessity.
d. Enabling Technologies
Once the complex and difficult technical hurdles of basic AR functionality are overcome (e.g., practical wearable computers, accurate tracking and sensor devices, lightweight highresolution HMDs and effective human-machine interfaces), additional technologies will enhance overall AR capability by adapting a particular system to a specific policing application. The true strength of AR will be its ability to improve the situational awareness of an individual user or groups of users on a wireless network, by employing a number of related technologies, alone or in combination. These devices include enhanced optical equipment such as night and thermal vision; sensors for such things as biometric identification and chemical/biological and explosives detection; identification and tracking of moving and stationary objects; real-time speech translation and text-to-speech systems; and advanced software applications to provide other data not otherwise visible or readily available to the user.
2. RESEARCH SITE VISIT
On February 22, 2002, a site visit to the Naval Research Lab (NRL) in Washington, DC, was conducted to examine the BARS, under the direction of Dr. Simon Julier. This meeting provided an opportunity for a hands-on demonstration of the state of AR development today and discussion with scientists and engineers engaged in AR system design. A synopsis of this project is presented below and is based upon the BARS website and on-site interviews with Dr. Julier, Dr. Mark Livingston and Mr. Dennis Brown.
BARS is intended to explore the utility of AR in a ground combat, urban environment, primarily designed and developed for the US warfighter. It attempts to address the fundamental difficulties of urban combat by providing the navigation and coordination information necessary for individual ground troops to fight effectively as a unit and the strategic and tactical information necessary to survive and win in battle.
BARS is made up of two major components: The Wearable Augmented Reality System (WARS) and the 3D Interactive Command Environment (3DICE). WARS is comprised of a suite of hardware and software systems, incorporating traditional AR equipment (wearable computer and see through HMD) with more advanced components such as an outdoor tracker, distributed virtual environment software and wireless connectivity to allow for multi-user interaction. It also establishes a relationship between mobile users and command personnel managing the battle from a fixed location. BARS was developed as a network-centric system, capable of sending and receiving information to and from a fixed location command and control site while sharing information among mobile users on the ground.
Mobile WARS users will receive information, reports and orders from commanders operating from the 3DICE and coordinate with each other to conduct combat operations. Among other functions, WARS users can view the location of all unit members, view virtual maps of the local terrain and project registered virtual utility and building information on the real-world view of the HMD as if taking an x-ray of the immediate area. Users also will be able to send short situation reports to each other and to headquarters. Interactive communication between mobile users is achieved through the use of standard pen-based PDAs, voice and a 3D gesture application that is under development.
WARS is a direct descendant of the Touring Machine application of Columbia University's Mobile Augmented Reality System project. (Columbia University, MARS) The current test-bed system is made up of components that are predominantly commercial off the shelf (COTS) hardware capable of displaying the following types of information:
* Global information about the environment
* Absolute location of user and team members
* Planning information from HQ (Objectives, etc)
* Fixed landscape feature information (trees, etc)
* Building information (building name, floor plans)
* Routing information (quickest route to objective)
* Translation of local road signs
* Warnings (enemy sniper in window)
* Infrastructure and utility information
* Virtual objects overlaid on real world view (Office of Naval Research)
In its present state, the BARS hardware is purposely bulky and oversized. This permits a maximization of system performance and facilitates servicing the various system components. Practical, ergonomically configured wearables will be the subject of later prototype development.
Accurate tracking of the user's location and head position, particularly outdoors, along with the precise registration of virtual images and data on the HMD is the biggest hurdle to overcome with BARS and is the subject of continued work at the NRL. The other major obstacle is the human-machine interface (i.e., displaying the data in a way that imparts the necessary information without subjecting the user to information overload). The HMD is also a point of concern, though it appears that the virtual retina display may ultimately be the best choice, particularly for outdoor use, due to its wide field of view and ability to withstand bright sunlight without washing out. All of these issues are the subjects of continuing research at NRL. (Julier, Baillott, Lanzagorta, Brown, Rosenblum, 2000)
The BARS prototype was demonstrated indoors and used an electronic grid system mounted on the ceiling for precise tracking of user head position and location. Even under ideal conditions, image registration tended to be off slightly from the real-world view. Equipped with an older generation optical see-through HMD that was developed primarily for use with video games, the user was restricted to a thirty-degree field of view. The real-world view was also somewhat difficult to discern indoors and tends to wash out in bright sunlight.
Nonetheless, the potential benefits of AR technology were readily apparent. Buildings in the surrounding area and their associated floor plans had been programmed into the system, which allowed the user to have a picture of interior rooms in adjacent buildings while outside on the street. The user could select a wider view, seeing the location of doors and windows on buildings two blocks away, or choose a narrower view to limit the amount of information on the display. A user could see his or her real-time location and that of other AR users projected on a 3-dimensional map that displayed orientation and direction. Though simplistic in its current form, with comprehensive data, accurate sensors and better HMDs, there is great potential for AR use by police officers.
3. POTENTIAL POLICE APPLICATIONS OF AR
Based upon current research and probable avenues of development, it is possible to derive a number of likely police applications or law enforcement scenarios where AR technology could advance police officer performance well beyond current levels. The following is only a partial list of promising uses, categorized by policing specialty. It is intended only to provide a range of potential applications, with many of those listed being applicable to multiple categories within many different operational policing scenarios. The actual number and type of eventual uses for AR technology will be limited only by the knowledge and creativity of police officers.
a. AR on Patrol
Uniformed patrol officers will have many potential uses for AR technology over a wide range of scenarios. Among them are a variety of applications to facilitate interaction with a diverse population and to detect criminals and crimes in progress.
i. Real-time language translation along with data on cultural customs and traditions could strengthen police ties with minority groups and improve information flow to police.
ii. Real-time intelligence about crimes and criminals in the patrol area could be immediately displayed to foster appropriate patrol actions to reduce crime.
iii. Facial, voice-print and other biometric recognition data of known criminals could allow officers to identify wanted subjects merely by observing people on the street.
iv. Integration of chemical, biological and explosive sensors could immediately notify officers of any local contamination and recommend appropriate protective measures for themselves and the public.
v. Scalable, three dimensional maps, complete with building floorplans, sewer system schematics, public utility information and public transportation routes, could be accessed at will to improve situational awareness and response to problems.
vi. Patrol car operator data and regional traffic management information could be displayed in a HUD mode to make driving safer and more efficient, especially in pursuit and rapid response situations.
b. AR and SWAT Operations
Many of the police tactical applications for AR resemble those required of BARS. Improved situational awareness during dynamic and dangerous incidents, enhanced cohesiveness between team members and better coordination with command personnel could make SWAT operations safer and more effective.
i. Advanced audio systems could modulate gunshots and explosions while providing the user with superior hearing capabilities over long distances.
ii. Advanced optics could provide zoom, thermal and infrared imaging for the location and apprehension of fleeing criminals.
iii. Identification Friend or Foe (IFF) technology, worn by every police officer, could reduce or eliminate friendly fire casualties by visually, audibly and/or haptically highlighting fellow police officers.
iv. The human-machine interface could be extended to robots and other mechanical devices, extending human capabilities to remote locations through physical, virtual and haptic interfaces (Colgate, Wannasuphoprasit, Peshkin, 1996).
c. AR and Criminal Investigations
Criminal investigators are making great use of current information technologies. AR could further enhance their ability to gather information, follow leads and visualize large amounts of data to solve crimes and more quickly identify and capture dangerous criminals and terrorists.
i. Speaker Recognition technology is under development that will give investigators the ability to accurately match voices against known criminals (Jin, Waibel, 2000).
ii. With advanced optics, investigators could lip-read from great distances in situations where listening devices are impractical (Meier, Stiefelhagen, Yang, Waibel, 2001).
iii. Interrogations might be improved with the use of thermal imaging to help indicate the truthfulness of subject's statements (Patch, 2002).
iv. Crime scene investigation could be enhanced by the use of AR video, audio and sensing devices used to visualize blood patterns, blood stains and other sensor-detectable forensic data available at the scene.
v. Forensic pathology could benefit from various advanced medical imaging techniques to visualize traumatic penetrating wounds before physical autopsy (Viire et al., 1998)
vi. Surveillance operations could be greatly enhanced through the coordinated use of robots, unmanned aerial vehicles (UAVs) and police officers managed through an AR network.
d. AR at the Academy
Static and controlled environments like a training academy are much more flexible in the types of equipment that can be employed. AR could be used to simulate dangerous police environments while blending real-world equipment and fellow trainees into the scenario.
i. A type of Magic Book for policing programmed for penal and criminal procedure laws, agency rules and regulations, or basic citizen interactions could be used for entry-level training in a controlled setting.
ii. Various types of interactive simulations for police training could be developed, such as realistic firearms "shoot-don't shoot" and use of force scenarios.
e. Augmented Management and Supervision
Leadership from the front is a commonly talked about principle that is rarely utilized in policing due to the difficulty of accessing and visualizing all the available supervisory and management data while mobile.
i. Using video feed from their personnel on the street, supervisors could potentially see what their people are seeing in real-time.
ii. Location, activity and status information could be available on a 3 dimensional map display of the community.
iii. During critical incidents, supervisors could monitor the physiological status of all personnel and make decisions concerning tactical deployment and dynamic action based upon who is mentally and physically best capable to perform.
iv. Coordination of widely dispersed units could be enhanced through the use of visual, audible and haptic cues from the supervisor.
AR Implications for Policing
The case for applying AR to police work is a strong one, but like all emergent technologies, there are potential down sides and foreseeable problems associated with its implementation. It is clear that AR is still emerging. Though several decades of research underlie its development and numerous evolving research projects are currently underway, the technology is still several years from practical viability for law enforcement use and like many emerging technologies today it is still largely unheard of outside the research community. To date, there is no available AR research focused on policing applications in spite of the obvious potential for greatly enhanced individual police officer performance in a wide variety of law enforcement situations.
1. TECHNICAL CONSIDERATIONS
There are numerous technical aspects of AR that require further development and consideration before viable law enforcement applications can be implemented: Real-time highly accurate registration of virtual data within the see-through HMD; faster image refreshment rates, particularly when moving the head rapidly from side to side; less weight and bulk of AR components; improved outdoor tracking and reporting of sensing data from non-intelligent objects in the environment; more user friendly human-computer interfaces and command input devices; and the seamless integration of various sensors and wireless components. Engineering efforts are underway that will dramatically improve all of these areas in a very few years. Without question, many aspects of AR would be beneficial to the average patrol officer or investigator fighting crime within our communities. AR also holds tremendous benefit to policing in our newly engaged fight against terrorism. First, street-level policing is a hands-on profession and increasingly one in which easy access to a wide range of relevant data would enhance effectiveness. The federal COPS MORE program of the 1990's was intended to increase the number of police officers on the street by improving their overall effectiveness through the use of technology (National Institute of Justice, 2000, p.42). AR can potentially produce even more dramatic Full-Time Equivalents (FTE) to the nation's police departments by providing front-line officers direct and unencumbered access to real-time data that increases individual efficiency (and thus time spent engaged in actual policing activities). It has the capacity to give officers greatly enhanced situational awareness and real-time decision-making capabilities.
AR's ability to incorporate a large amount of specific and pertinent database information into an officer's field of view, without having to look away or be distracted from the immediate action at hand, is an individual "force multiplier" with far-reaching significance for situational awareness and officer safety. Information processing and decision-making can occur almost simultaneously in real-time, with no need to manipulate a keyboard or glance away from the action to view a separate monitor.
Second, coupled with broadband wireless networks, intelligent environments and newly developing sensor and optical technologies, the sheer amount of directly relevant information available to the individual police officer will be unprecedented. The ability to directly access tremendous amounts of useful information and intelligence concerning the surrounding environmental landscape and the people occupying that landscape, whether they be fellow police officers or other public safety workers, innocent civilians, or criminal terrorists, will contribute directly to deterring and fighting crime, combating terrorism and enhancing public safety (DARPA, 2001).
There is a tremendous amount of work to be done however, before AR will be acceptable for police use. The design and functional relationships of HMDs, retinal imagers, 3D sound and sonification, haptics and related computer-human interface technologies such as voice activation, necessary to streamline the control of AR systems and improve spatial awareness and efficiently impart the flood of otherwise useful information to the user, will play a large role in the extent to which AR becomes a practical police system. Complex and potentially powerful AR systems for police must become what Michael Dertouzos of MIT's Computer Science Lab calls "humancentered computing" (Dertouzos, 2001). They must achieve the "net benefit" of digital technology, the capacity of digital devices to communicate and cooperate in a way that assumes most of the burden of actually operating complex technology, thereby reducing the cognitive load on the AR user who can devote complete attention to his or her primary mission of policing (Denning, 2002, p.166-167). Effectively dealing with the ability of police officers to easily operate AR systems will be a major component of AR success.
There remains the question of the human psychological reaction to the aesthetics of AR. Technological advances will undoubtedly miniaturize much of the weight and bulk of today's HMDs, computers and sensors. Nevertheless, it is quite probable that for at least the next decade, use of a robust AR system for police applications will impart a somewhat "Star Warslike" appearance to the police officers using it.
Acceptance by the public of civilian police officers whose appearance suggests fundamental and profoundly different physical capability is potentially of great concern. Police officers obviously equipped with physical and sensory devices purposely designed to reach beyond "normal" human capabilities and possibly access personal and sensitive information regarding anyone, criminal and innocent civilian alike, could cause severe public resistance to the technology. Indeed, this apparent physical melding of police officers with powerful technology may induce a visceral negative reaction by some citizens (Mann, Niedzviecki, 2001, p.77-88). This appearance-induced perception and the resulting fear and suspicion associated with it, may be difficult to overcome, making widespread police AR implementation impossible without careful planning, confining it to a few highly specialized and situationally constrained applications.
Acceptance by police officers and police leaders may be even more problematic. Decades after the development and introduction of police body armor the wearing of it is still resisted by some officers due to complaints of bulkiness and increased body heat when worn in warm climates. Leaving aside whether or not the technology is capable of enduring operation in adverse weather conditions, bulk and endurance issues associated with necessary physical conditioning could also cause many officers to reject AR systems.
Appearance is also a major concern of many officers and police administrators alike. The spit-shined para-military culture within many police departments may not readily adapt to the detraction from the traditional starched and pressed appearance that has historically connoted professionalism among police. Similarly, even more progressive Community Oriented Policing (COP) cultures strive to identify and associate more closely with the general populace and abhor the appearance of militaristic garb or uniform accouterments that might offend public sensibilities.
AR may be difficult to employ in the short term, not because of technical issues, but due more to a resistance to changing the appearance of police officers caused primarily by the HMD. Micro-Electro-Mechanical-Systems (MEMS) and nanotechnology will eventually solve the appearance issue, but short-term issues associated with appearance will probably survive the decade. The type of display chosen could mitigate the effect on appearance and acceptability. Optical see-through HMDs, particularly those that cover only one eye, will likely be more easily accepted than video see-through HMDs that completely cover the user's eyes. Along with the operational pros and cons of different HMD configurations, the aesthetic implications will have to be considered when developing AR systems for police (Feiner, 1999).
On the other hand, many police officers today wear prescription glasses and law enforcement tactical situations commonly call for goggles or eye protection as a practical safety precaution. AR HMDs could take the place of both, providing corrected and enhanced vision while granting inherent protection from eye injury in hazardous environments. To be sure, the look of AR systems in the near future will be a radical departure from a traditional police appearance and, as with most significant change, will undoubtedly raise some controversy in its implementation. Nonetheless, there are specific policing applications and circumstances where the appearance issues associated with near-term AR HMDs should not be a significant detractor.
Unlike the gentle rain from the heavens, technological innovation does not fall equally upon the place beneath. Almost a quarter-century since the personal computer became an established feature of offices, classrooms and homes, pundits still refer to a "digital divide" that tracks with income. A comparable phenomenon exists among police and law enforcement agencies (including the FBI) despite the infusion of technology under the COPS MORE initiatives. Many jurisdictions suffer a hardware deficit; others have computer technology but are support-poor, deficient in training, or stuck with obsolete purchases so idiosyncratic that they make the portable Underwood manual typewriter seem like a solid alternative. To suggest that these agencies will be able to step immediately into the future of AR is simply unrealistic.
That leads to the immediate corollary that AR will improve some policing.... but not necessarily policing in general. Policing "as we know it" will survive in many pockets--some isolated, some not--for considerably longer than we might hope, with debatable impacts on the quality of justice.
The most obvious problem area for many police agencies is that of finance. Private industry has a variety of mechanisms for defraying the costs of new equipment, not least of which are tax deductions and passing other costs on to consumers. The federal government also has a broad range of tax revenues to draw upon. Municipalities and to a lesser degree states, have considerably greater limits on their ability to purchase and support technology for the police. Major capital purchases tend to be leased or bond-funded, with 10-, 15- and 20-year capitalization schemes.
As the recent Urban Institute examination of Title 1 indicated, cities had difficulty finding the matching funds to fully avail themselves of the 1994 Crime Act federal assistance (Roth and Ryan, et al., 2000). The ability to procure and maintain AR systems most likely will elude all but the largest departments. Like SWAT, bomb squads, Haz-Mat and certain investigative specialties, it is very likely that the first generation of police AR will be limited to large municipal agencies, State Police or investigative bureaus and regional pooled-resource task forces. For most of the small jurisdictions, the promise of AR will remain just that: a promise.
Even in the well-funded organizations, budget limits are made manifest in subtle ways. At best, support and maintenance are not as "sexy" as the initial capital purchase; at worst, like a boat, they become holes in the strategy into which one pours money. The rapid expansion of technological capacity adds "upgrade" to that list. There is always the possibility that this year's technology will be hopelessly outmoded--even dangerous, exposing the user to technologically superior counterattack--even before this year's new cars are driven into the ground.
Training is also a vulnerable budget item, easily "trimmed" in times of financial crisis. However, the old training technologies (roll-call memos and videos, an hour or two of lecture in the state-mandated forty hours of in-service training) will not suffice for technologically complex systems. Regular users will most likely adapt well and be able to switch into and out of AR mode with ease and with no loss of proficiency; it is not yet known whether occasional users of AR will experience difficulty in the transition or not.
The final element for some departments will be integrating AR skills into the agency's career path and promotional schemes. Unless biological imperatives dictate otherwise (see below), budget considerations will probably limit first-generation AR to a specialty skill. For uniformed officers, any number of comparable assignments ranging from SWAT to D.A.R.E. to Internal Affairs are already well-integrated into career tracks in large agencies. AR technology may also offer a comparable specialty among civilianized positions, notably crime scene technicians and technical support. The latter will be especially crucial as agencies adopt AR systems and then find themselves having to keep up with and adapt to rapid changes in the technological environment.
3. SYSTEM SUPPORT
The core of the AR system is the ability to integrate and communicate relevant data in close-to-real time. As such, it has vast potential to streamline police inquiries, broaden police effectiveness and lay a consistent, defensible foundation for police actions. The various possibilities outlined above include integration of information in three primary modes: from existing databases; from purposely embedded devices in the target environment; and from an AR sensor array on the individual's harness, scanning the immediate environment. The existing databases are a weakness discussed here; embedded devices and portable AR scanning devices are discussed under "Counterattack," below.
Computer programmers have long used GIGO (Garbage In, Garbage Out) as a code for human input problems inherent in computing and GIGO has cousins. A system of technological interface like the AR system is dependent upon the budget, hiring, training, retention and various operational and personnel policies of institutions and units beyond the system agency's control. The AR environment is only as reliable as the database management of the agencies from which the system will draw data.
"Fastest route" information depends upon a real-time system that records construction sites, traffic obstructions (delivery trucks unloading, funeral processions, neighborhood block parties), routine and moving tie-ups of heavy traffic, etc. Some of that may require input from the Department of Public Works, which in turn requires the creation of both a mechanism for relaying the information to the police and a chain of responsibility for seeing that it is entered (and removed) with alacrity. Other inputs may come in trickle-stream form from either sightreports by patrol officers or highway supervisors, or from a monitoring system of cameras such as those monitoring traffic flow on main urban arteries.
Such systems are usually not comprehensive, however and routing information is likely to remain a patchwork system at best. While in the best-case scenario, such responsibility within the police agency can be fixed easily within Dispatch or elsewhere, the optimal conditions do not always exist. Police and public service agencies are as vulnerable to their own budgets' hiring policies as to those of external agencies in this regard: the skill levels of current employees, staffing levels and physical space considerations all intrude upon the endeavor. Moreover, the responsibility for the external inputs cannot be dictated by the police. The problem is more complicated for jurisdictions that contract with other agencies for dispatching and communications services. The policies and practices of regional or multi-service centers may be determined by User Boards and the technologically-deprived may not be as enthusiastic about imposing changes that serve only the technologically favored. All these difficulties may be even more pronounced for agencies serving rural jurisdictions, where routine coordination may be slower and spontaneous information less available.
Similarly, "building information" will be dependent upon offices that have no real interaction with, or sensitivity to, emergency services apart from the occasional fire inspection. The same will be true for infrastructure and utility information, with the additional caveat that some of this involves private enterprise and proprietary database information that the proprietor will be naturally reluctant to share with law enforcement. Corporate culture may be quite different, both in public- and private-domain entities, with paper-driven permit systems and inputs delayed pending any of a number of political or staffing-level reasons. The police will be hard-pressed to compel radical changes in offices of code inspections or building permits simply to have data available against an uncertain (and probably infrequent) number of emergent situations.
That has the potential to leave integrated AR vulnerable to inaccurate descriptions of building interiors, hazards and even ownership. How that will affect response teams' capacity and the ability of the police to adapt, improvise and overcome discordant inputs between augmented and perceived reality remains a large unknown. Sifting partially accurate inputs--identifying and discarding inaccuracies and making real-time adaptations--will likely place great strain on human operators, possibly requiring increased communication (distractions from the mission, in tactical situations) and create as-yet-undetermined obstacles.
Ordinary crime scene work does not carry the same level of danger as live-action tactical work, but certain crime scenes may be fraught with special hazards. Detection of booby-traps and chemical hazards in clandestine drug labs may be greatly augmented by user-borne area sensors, but those sensors compare elements in a database of known hazards (such as precursor chemicals). The advancing technology of designer drugs continuously incorporates and develops new chemical compounds, some of which may not be in the database. The real hazard lies less in the malfunction of the technology than in human over-reliance upon the AR system, to the exclusion of normal caution and common sense.
All integrated database systems presume a fully-functional capacity on the part of the contributing databases (this is, that they are up-to-date, complete and error-free). In reality, such systems are dependent upon invisible human processes that may seriously undermine the assumption of capacity. Though there may be a tendency to call for centralization of the responsibility within the agency (another specialized position, perhaps, depending upon the department's structure), liaison with other agencies does not guarantee timely compliance. It is also useful to recall that policing has not always kept its own house well, particularly where "paperwork" and its electronic counterparts are concerned: the rate of failed updates of and expungements from the NCIC databases perhaps serves as a convenient reality check. And the larger the database, the more time-consuming the chore of data verification and data cleaning; neither UCR nor NIBRS handles the national volume of data at anywhere close to real time, even though the smaller volume Wanted Persons and emergency alerts are handled with efficiency.
For police departments as well as other municipal agencies, local budgets affect hiring practices, as do the invisible lines of political patronage, nepotism and the available labor pool. Even the best systems remain vulnerable to work overload and "break in the new employee" assignment (with the resulting triage of tasks and backlog of routinized functions in the face of "urgent" work), to vacations and sick days, to mandatory training and special functions (e.g., birthdays, baby showers, retirement parties, etc.) that delay entry of new information and changes into the databases feeding AR systems.
While most of the time these delays will be invisible to the AR user and harmless, policing is an occupation built around one-in-a-million circumstances and similarly vulnerable to them. The best available method for minimizing such potential problems is the identification of probable trouble points and a process of systematic review as a form of "preventive maintenance." Under the general rubric of "partnership," it is not beyond possibility that the police might act to assist other agencies in data entry to clear backlogs, for instance. Such a cooperative effort would not involve police officers doing data entry, nor even police agency personnel. Since data entry is relatively low-skill and routinized work and similar to working in Records or Evidence, police-supervised or sponsored interns from local colleges could provide a certain portion of their work hours to such a support task, as an adjunct to their police-related experiences.
4. BIOLOGICAL/PSYCHOLOGICAL CONSIDERATIONS
It is too soon to know whether AR is a viable tool for either the military or the police, in all situations, despite promising early results in the developmental stages. While it seems likely that the remaining technological barriers outlined above will be surmounted, lingering questions remain about the human component. Only a handful of individuals in a smaller handful of places have much experience with AR at this time and they have already noticed some human interface problems.
In addition to the problem of temporary disorientation when the interface is abruptly terminated, researchers at the NRL reported that some individuals experience headaches and other symptoms of disorientation during regular use. The exact cause of those conditions is not yet known and could be a byproduct of technical alignment "bugs" that will be worked out in the near future. This type of interface, like that of freelance pioneers who have "wired" themselves to their wearable computers (Young, 2002), may occur in predictable ways with length of use, or it may vary widely depending upon neurological factors not yet discerned.
AR may produce long-term health problems that have not yet been manifested by limited exposure under controlled conditions, or it may prove to be fundamentally harmless for all but a few individuals (like the epileptic scientist in Michael Crichton's The Andromeda Strain). The lessons of the concerns over radar and testicular cancer, however, should not be lost on police managers: there is a potential for similar correlation and "moral panic," generating a host of medical disability retirement claims and other legal actions that tax the jurisdictional budget and administrative staff capacities.
In a worst-case scenario, premature adoption of a new technology may turn out to be an unintended Stage 2 field test of medical impacts, especially for direct retinal display. Clinical tests would need to be conducted (and the results verified by systematic replication) as part of the overall adoption process. The inherent frustration is that clinical tests delay the application of a desirable system (reviewing the pharmaceutical industry's frustration with FDA trials for new drugs may provide a useful analogy). Since AR is not a regulated form of commerce as drugs are, some will no doubt be tempted to put the technology into use once stage 1 testing proves that it can work operationally; those jurisdictions and individuals run the risk of being the guinea pigs for the neurological affects. Even clinical tests are likely to be short-term; it will take at least a generation of use to understand the long-term implications of AR interaction with the human neurological system.
5. LEGAL ISSUES
Regardless of the technical proficiencies, the full potential of AR as a police tool may not be realized because of social and legal concerns. Tactical and crime scene uses will almost certainly meet the tests of warrant-justified or exigent circumstance searches, both of which can be enhanced by AR. More problematic is the potential for routinely deploying AR scanning while on patrol and the use of AR as an examining tool during routine questioning and interrogations (e.g., Callahan, 2002).
As the sophistication and capacity of technology expands, it butts hard against older legal concepts of property and privacy rights. At the core of this evolving sensitivity is the June 2001 Supreme Court decision in Kyllo v. United States, (99-8508) 190 F.3d 1041 (2001). The 5-4 majority reversed a 9th Circuit Court, ruling that allowed the use of a thermal imaging device to measure "waste heat" emanating from a private dwelling. The majority held that the imaging constituted a 4th Amendment search, because the technology revealed, or had the potential to reveal, "intimate details" of the activities inside the home.
Although the Kyllo case dealt with heat lamps for the clandestine growing of marijuana, a portion of the Court's decision dwelt with "the lady in the sauna," another activity that could be revealed, or at least intimated, through a measurement of escaping heat. Without a trace of irony, the dissent noted that "this case involves nothing more than off-the-wall surveillance by law enforcement officers to gather information exposed to the general public from the outside of petitioner's home." The dissenting Justices essentially sought to anchor the decision in the narrow confines of the facts of the case, equating the escaping heat with "aromas [escaping from] a kitchen, or.. laboratory, or .. opium den [that] enter the public domain if and when they leave the building." As such:
the detection of infrared radiation emanating from the home did not accomplish an unauthorized physical penetration into the premises. (Kyllo, Dissent I, [paragraph] 2.)
The majority saw in the case a wider issue and though they did not directly address whether infrared radiation could be detected without senses-enhancing technology (as the aromas can), they chose to decide the case based upon the larger principle of intrusive technology. The foundation of the decision for the majority opinion was the notion that
At the very core of the Fourth Amendment stands the right of a man to retreat into his own home and there be free from unreasonable government intrusion. Silverman v. United States, 365 U.S. 505, 511 (1961).
While acknowledging precedent that "visual observation is no surveillance at all," and that "Fourth Amendment protection of the home has never been extended to require law enforcement officers to shield their eyes when passing by a home on public thoroughfares," the justices noted that "[t]he present case involves officers on a public street engaged in more than naked-eye surveillance of a home." The enhancement of ordinary sense perceptions by technology--a basic if incomplete definition of AR--is the instrument by which the Fourth Amendment violation occurs. Even if a "jurisprudence of limited through-the-wall technology" were to develop,
no police officer would be able to know in advance whether his through-the-wall surveillance picks up intimate details--and thus would be unable to know in advance whether it is constitutional. (Kyllo, Majority, [paragraph] 12)
Notable both for its unusual alignment and the unusually arch exchange in the opinions, Kyllo set the stage for subsequent cases focusing on new technologies. The two fundamental premises for the Court's ruling on the application of new technology were the existence of an expectation of privacy and social acceptance that such expectation was reasonable:
Where.. the Government uses a device that is not in general public use, to explore details of a private home that previously would have been unknowable without physical intrusion, the surveillance is a Fourth Amendment search and is presumptively unreasonable without a warrant. (pp. 313) ... [But a] search does not occur even when its object is a house explicitly protected by the Fourth Amendment unless the individual manifested a subjective expectation of privacy in the searched object and society is willing to recognize that expectation as reasonable, see, e.g., California v. Ciraolo, supra, at 211. (pp. 35).
The decision left open the issue of future technologies, however, with the qualification of "common use":
... in the case of the search of a home's interior--the prototypical and hence most commonly litigated area of protected privacy--there is a ready criterion, with roots deep in the common law, of the minimal expectation of privacy that exists and that is acknowledged to be reasonable. To withdraw protection of this minimum expectation would be to permit police technology to erode the privacy guaranteed by the Fourth Amendment. Thus, obtaining by sense-enhancing technology any information regarding the home's interior that could not otherwise have been obtained without physical intrusion into a constitutionally protected area, Silverman v. United States, 365 U.S. 505, 512, constitutes a search... at least where (as here) the technology in question is not in general public use. This assures preservation of that degree of privacy against government that existed when the Fourth Amendment was adopted. Pp. 67. (emphasis added)
The Court rejected the Government's "mechanical interpretation of the Fourth Amendment" equating the thermal image obtained from the house exterior with visual observation of the house (the case of California v. Ciraolo, 476 U.S. 207, 213, established that "visual observation is no search at all"; also see Dow Chemical Co. v. United States, 476 U.S. 227, 234, 235, 239, which is distinguished because the property in question was commercial rather than residential):
that approach would leave the homeowner at the mercy of advancing technology--including imaging technology that could discern all human activity in the home.
Also rejected was the Government assertion that the evidence should be admitted because it revealed no intimate details of the activity within the house, only a "fact" that when added to other facts (including information provided by Kyllo's ex-wife and utility records) created an inference:
Such an approach would be wrong in principle because, in the sanctity of the home, all details are intimate details. See e.g., United States v. Karo, 468 U. S. 705; Dow Chemical, supra, at 238, distinguished. It would also be impractical in application, failing to provide a workable accommodation between law enforcement needs and Fourth Amendment interests. See Oliver v. United States, 466 U. S. 170, 181. (pp.712).
While the current developmental use of AR draws from existing databases, as already noted it is reasonable to expect that future generations of AR will be capable of integrating realtime inputs from active scanners. Broadband scanning by police routinely equipped with AR technology (whether vehicle-mounted or individually-equipped) runs the risk of inadvertent violation of the legal threshold.
The intersection of technology and privacy expectations will continue to evolve, perhaps toward the future demonstrated by the involuntarily personalized advertising in the movie Minority Report, perhaps in another direction entirely. In mid-2002 (at the time of this writing) there are a number of debates centering on the definition, scope and importance of privacy: the abortion controversy; the continuation of "sodomy laws" defining acceptable sexual practices; "Big Brother" concerns over the use of closed-circuit television (CCTV) monitoring of public places; marketing of information contained in medical records; the associated issue of the private use of digital fingerprints and retinal scans for fraud protection; the "opt-in/opt-out" debate over personal information surrendered to corporations as a necessary condition of obtaining goods and services; and the concomitant issues of computer database hacking and identity theft. Unanticipated consequences of purportedly helpful technologies will continue to create new issues, such as the airport security x-ray scanners that "are not photo quality but.. are good enough to tell gender" (Merritt, 2001) and interactive television sets that report viewer data back to sponsors ("Advertisers Tune In," 2000; "Interactive TV," 2000). Whether public concerns will eventually lead to a comprehensive definition (or redefinition) of "reasonable privacy" is a matter of conjecture, but where social acceptance is an issue, the promises of technology are vulnerable to the circumspection of law (see also Colbridge, 2001).
While the Kyllo case provides a temporary benchmark for limits on intrusive technology, a question asked from the bench during the oral arguments may set the stage for continued evolution: if it were possible for anyone to assemble a thermal imaging device for $5 or so, out of parts bought at an electronics outlet store, would the use of the more sophisticated technology by the police be permissible? That question tacitly recognizes that technology is not the exclusive domain of the government and raises two separate issues.
The first issue addresses the consequences when today's new- threshold technology becomes tomorrow's commonplace gadget. As Chief Richard Myers of the Appleton, Wisconsin, Police Department noted recently, constitutional restraints affect only government actions, not private ones or corporate ones. The Kyllo case may be revisited in ten years, when both available technology and the commonplace technical skills of the public make "through-the-walls" snooping a broad reality. More immediately and more importantly, Kyllo also sets the stage for a second issue, one of deep concern for police: favorable technology in the hands of those with unfavorable attitudes.
6. POSSIBLE CRIMINAL USES OF AR TECHNOLOGY
There may be a short period of time when the police have a user-end monopoly on the new technologies, but the police market alone is too small to sustain growth for specialty manufacturers. Additional profits must be found in related occupations, beginning with private security and the wider the market, the looser the controls over a product's availability to the criminal element. It is useful to recall the days when body armor seemed like a wonderful propolice technology; those days ended for good on February 28, 1997, when bank robbers clad in body armor destroyed police cruisers, wounded several Los Angeles police officers and civilians and terrorized a neighborhood during the North Hollywood bank shootout.
The proliferation of user-centered computer markets means that the technology will be more and more widely available. It is necessary to anticipate that the bad guys will have access to the technology fairly early and will work to devise both defensive measures and counterattack strategies. Indeed, it may be wise to anticipate that the criminal element may already be ahead of the police in these areas.
At the moment, one of the prime issues of concern, both for individual privacy and for proprietary data, is the vulnerability of the wireless technologies to unauthorized intrusion. The potential dangers of wireless networks have been recognized by the Pentagon (Wrolstad, 2002) and privacy experts are already raising questions about the legal thresholds related to GPS tracking of wireless devices (Robinson, 2002). Unauthorized wireless surfing over WiFi (wireless fidelity) networks provides anonymous access to strangers' Internet accounts through the WiFi ports: all that is required is that a wireless laptop be within the signal range of a WiFi system transmitter (Harmon, 2002). Loss of subscription revenue (analogous to theft of cable TV service) and snooping in unguarded computer systems are recognized hazards (Paulsen, 2002). Also called "war driving" and more prosaically "802.11b access," this wireless "piggybacking" phenomenon has already become recognizable enough to be featured in a Doonesbury strip (Sunday, 21 July 2002). Industrial espionage and identity theft likewise center on the "open door" of wireless transmissions.
The corresponding police concern besets the wireless transmission of police reports, a digital equivalent of "scanner-land." Intercepting data for counterintelligence purposes, tapping in to GPS systems to shadow-monitor the location of police vehicles, even gleaning details of officers' lives from car-to-car transmissions are all potential hazards, in addition to the standard database-hacking dangers. Regarding AR technology, however, the greater danger of wireless is the potential for the criminal element to introduce counterfeit data or overload "spikes" into the data streams, both during database compilations and real-time live action engagements.
Criminal counteroffensives can take multiple forms. False data might be introduced into an agency's records-keeping system: fake exonerative information, misleading clues and false trails pointing suspicion onto competitors and away from confederates, are potential passive intrusions. In tactical situations, the dangers run from counterfeit "Friend-or-Foe" signatures (both to protect bad guys' escapes and to invoke friendly fire by inverting or negating "Friend" signals) to massive electronic inputs designed to overload or distort the systems. Blinding AR-equipped officers, sending sonic distortion to affect their balance, shutting down the AR system at the critical moment that a counter-attack or escape is initiated and probably hundreds of other creative uses should be expected as counter-thrusts from criminal and terrorist networks.
Second-level data corruption would focus on the secondary databases employed for AR: the most obvious would be the insertion of false building blueprints into the Building Inspector's files. That can be done remotely, or through the agency of a suspicion-less confederate hired to work in the office in some clerical job with access to the computer system (another way that police AR is vulnerable to the hiring practices of other agencies). Police agencies have already experienced problems with low-level employees and even police officers selling information, from the janitor who funneled discarded drafts of the Jeffery Dahmer investigation to the news media to corrupt detectives providing case files to organized crime. Emerging technologies both exacerbate that age-old problem and add the new possibilities of direct interception and intervention without relying upon a human inside confederate.
The NRL uses a series of mounted sensors to orient their in-development AR battlefield prototype. Fondly called "the holodeck" after the standard feature of the second-generation Star Trek series, the area runs on a combination of desktop computers and track lighting fixtures, in keeping with NRL's design orientation of using off-the-shelf technology for cost purposes. It is only a small leap of the imagination to envision the interior of a mob bar or restaurant being similarly outfitted to support a rudimentary AR environment.... enough to discern whether an informant is carrying a wire, or where an undercover officer has concealed a backup weapon ("Body Scanner," 2001). Cameras that spot liars (Callahan, 2002) are as useful for drug lords as for detectives. A black market in stolen "Friend/Foe" signatures may well be the future version of today's market for stolen badges and uniforms. Nighttime thermal imaging is useful for detecting police surveillance posts or the approach of a raid team.
Beyond those scenarios are others. When "through-the-wall" technology becomes available, it can be used to pinpoint the location of individuals in a private home (or a police station) for rescue, kidnap, or assassination. It can identify key junctures to cripple electronics, tell when the on-duty data entry clerk has gone to the relief room on break, show the location of Evidence/Property rooms and their electronic monitors, isolate burglar alarms on private residences and send false signals to cover electronic intrusions.
As soon as the technology becomes available for police use, it creates a need for police defense against the same technology. The defacement of web sites is crude and childish, but it should by now serve as a wake-up call that the electronic village is vulnerable 24 hours a day. The police are not the only, nor even the most likely targets: intrusive technology in the hands of the bad guys makes the citizenry far more vulnerable and in need of protection. Leaving timer lights on in the house, even with recorded music or conversation or barking dogs, is of little consequence to a burglar who can establish electronically that the house is not occupied. The thought of portable through-the-walls technology in the hands of a child kidnapper/murderer is terrifying, as the Polly Klaas, Samantha van Dam and Elizabeth Smart cases vividly illustrate.
People will look to the police for protection. Not only are countermeasures expensive (and an almost-constant drain), they present the possibility of a further loss of jurisdiction. If the argument is correct that private police are encroaching upon the domain of the public police, offering services that the public police are incapable of providing, there will be long-term budgetary, jurisdiction and confidence issues to deal with. Corporate money drives the technology expansion, fueled by an economy that operates on a much faster timetable than municipal tax- and fee-based funding and within a much different system of resource generation. One of the potential futures is that global corporations, technologically more sophisticated but holding national allegiance secondary to corporate profitability (not necessarily a bad thing of itself), will take over more and more of the technology crime-fighting. As the events that followed 9/11 revealed, civilian governments have fallen behind the technological power curve due to archaic operational methodologies and snail's pace bureaucratic processes.
Many people today advocate partnerships with corporations and the military to bridge the technology gap and acquire the resources to conduct law enforcement operations against crime in cyberspace realms. However, the long-term implications of those partnerships have not been adequately addressed. In the short term, corporate and military interests may act as an almsgiver for chronically impoverished municipalities. In the long term, those interests may determine that a private technology police or military police, duly authorized by law, serves their interests better. Laws are changed through the customary technique of lobbying legislatures and would free corporate and military entities from the need to rely upon the same legal authority as the public police. While other elaborations on "have and have-not policing" are currently under discussion, the technology divide is a feasible candidate for the re-privatization or militarization of policing, to the undetermined detriment of public interests.
The proliferation of clandestine do-it-yourself methamphetamine labs throughout the country (Associated Press, 2002) provides another useful cautionary parallel. For the bad guy's purposes, technology systems do not have to be the best or safest available; they simply have to work for the limited purpose intended. Criminal uses of technology are not limited by municipal budget constraints, by court-admissibility considerations, or by continuous use in multiple environments. One-time throwaway use is sufficient. They can even be potentially dangerous to the people using them. The devices simply have to work "good enough" to defeat government work.
Police AR research should probably begin with projects that address specific confined and controllable law enforcement situations such as crime scene investigations and heads up displays in patrol cars where the current bulk, inaccuracy and aesthetic concerns of current generation AR equipment will not be as difficult to overcome. As we refine these basic applications and begin to understand the technology in detail it will be easier to incorporate it into wider and more diverse aspects of policing, identify early-on the biological impacts of AR use and ascertaining potential criminal and terrorist uses. Over time, bulkier components will be replaced with smaller and more powerful versions, biological adaptations can be made and we can evolve our policing methodologies to the incremental increases in technological capability that will inevitably occur.
AR technology could potentially have a very dramatic effect on the policing profession. As the underlying component technologies mature and less critical civilian applications start to emerge within society, the benefits and applicability to policing will become more apparent. Current state of the art graphics rendition and registration, tracking precision and accuracy and overall systems integration are still in an early stage of development, but progress in these areas is evolving rapidly.
At the same time, the combined effects of a multitude of technological trends are greatly increasing the speed at which things occur and change in society. The proliferation and incorporation of new technologies to deal with this speed and complexity, while necessary, is in itself creating an environment that is too fast and too complex for effective human control and direction (Adams, 2001). The future prevention and timely termination of crimes and terrorist attacks, like those of September 11th, may depend on the individual police officer's ability to rapidly process and analyze available data and take immediate action. Developed, designed and implemented correctly within the police community, AR technology could be the tool that helps to maintain a tactical edge over a rapidly evolving terrorist threat.
AR is a completely human-centered technology application. Its purpose is not to replace human beings with robots and subjugate human intelligence or judgment with that of faster and more efficient machines. Unlike the quest for Artificial Intelligence (AI), robotics, or some other "smart" technology research designed to eliminate the need for human intervention or involvement in favor of decision-making by computers, AR will directly enhance human performance, allowing people to work both faster and smarter, in full control of technology instead of being controlled by it. As Dr. Steve Mann suggests, this kind of "humanistic intelligence" allows us to shape the computer's behavior, making the technology responsive to our needs and desires (Mann, Niedzviecki, 2001, p. 30).
It has been speculated that AR might be useful in any situation requiring information that is not directly available or detectable by human senses by making that indiscernible information visible, audible, or touchable--an ideal tool in many policing situations (Azuma, 1997). It could give police officers tremendous physical and sensory enhancement that remains completely under the user's control and responsibility. In this post September 11th age, characterized by the criminal and terrorist exploitation of existing technologies and the serious threat they pose to free societies, citizens and politicians are quick to advocate military intervention in homeland security due to a real or perceived inability of civilian police to cope with the increasingly complex threat. AR could be a potent tool for improved policing, perhaps enabling domestic law enforcement and public security to be accomplished effectively without the use of military force inside our borders.
Suggestions for Further Research
As noted, the use of AR will raise numerous operational, physiological, social, legal and constitutional issues, all of which need to be fully understood prior to police implementation of AR technology. The applications for video see-through versus optical see-through HMDs, each with its own strengths and weaknesses, need to be compared and evaluated. The physical and possible psychological effects of wearing a HMD and working in a world continuously imprinted with virtual data for extended periods are still unknown and need to be examined. Additionally, there is a potential for impaired physical performance when abruptly removing the AR system from the user, a potentially dangerous situation if the system power suddenly fails or is damaged in a critical situation.
Officer acceptance of AR technology and the public reaction to the aesthetics of AR equipped police is an important issue--for the short term, bulky HMDs and other system components will be obtrusive. It may be possible to mitigate, lessen or eliminate the negative reactions with specific configurations of components and designs tailored to specific applications. For example, SWAT officers who normally wear helmets, goggles and bulky equipment will not suffer the same aesthetic constraints as patrol officers who deal directly and routinely with the law-abiding public. Those limitations and constraints need to be explored and understood.
Enhancing situational awareness with vast amounts of virtual data while preventing information overload is another critical issue for the effective employment of AR systems. The development of an Augmented Cognition system or Context Machine for police AR, similar to the on-going DARPA project, needs to be undertaken.
Training for AR systems is another area that needs detailed examination. It is unknown today how much time it will take to adequately train an individual to operate successfully using AR. Along with the training necessary to operate the technical components and understand the operational capabilities and constraints, there may be physical adaptation issues that will add time to the training curriculum.
In order to fully understand these issues it will be necessary to develop a prototype AR system to explore the different configurations and components for specific police applications. Partnerships to develop a police AR test-bed with academic and government research labs already experienced with AR system development would facilitate that process and speed the effective deployment of AR systems for policing.
(2) See http://www.via-pc.com/product/default.htm for a description of a commercial wearable computer for firefighters.
(3) Known as a gestalt, it is a structure or configuration of physical, biological, or psychological phenomena so integrated as to constitute a functional unit with properties not derivable from its parts in summation, which completely defines the circumstances.
Adams, T. (2001). Future Warfare and the Decline of Human Decisionmaking. Parameters: US Army War College Quarterly. Winter 2001-02, Vol. XXXI, No. 4. Retrieved February 10, 2002, from http://carlisle-www.army.mil/ usawc/Parameters/01winter/adams.htm
Advertisers Tune In To Interactive TV. (2000). Reuters. New York Times on-line. December 1.
Allen, B., Bishop, G., Welch, G. (2001). Tracking: Beyond 15 Minutes of Thought. SIGGRAPH 2001, Course 11. Retrieved February 2, 2002, from http://www.cs.unc.edu/~tracker/media/pdf/SIGGRAPH2001_CoursePack_11.pdf
Meth Labs Make Way Through Kentucky. Associated Press (2002). New York Times on-line. July 17.
Azuma, R. (August 1997). A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 6,4. Retrieved January 23, 2002, from http://www.cs.unc.edu/~azuma/azuma_publications.html
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B. (2001, November/December). Recent Advances in Augmented Reality. IEEE Computer Graphics and Applications. Retrieved February 9, 2002, from http://www.computer.org/cga/cg2001/g6toc.htm
Bajura, M. (1997). Merging Real and Virtual Environments with Video See-Through Head-Mounted Displays. Doctoral dissertation, University of North Carolina, Chapel Hill. Retrieved February 1, 2002, from http://citeseer.nj.nec.com/cache/papers/cs/1615/ftp:zSzzSzftp.cs.unc. eduzSzpubzSzpublicationszSztechreportszSz98-036.pdf/bajura97merging.pdf
Body Scanner Sees Like Superman. (2001). Wired News Report. http://www.wired.com/news/technology/0,1282,48966,00.html. Wired.com. December 11.
Boeing Wearable Computer Workshop Breakout Session Summary. (1996, August) Human Computer Interaction Breakout Session. Retrieved on February 17, 2002, from http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/vuman/www/home.html
Bowen, T. (2002). Computers Sort Gender in a Binary World. Technology Research News. Retrieved February 10, 2002, from http://www.trnmag.com/ Stories/2002/013002/Computers_sort_gender_in_a_binary_world_013002.html
Broderick, D. (2001). The Spike: How Our Lives Are Being Transformed by Rapidly Advancing Technologies. New York: Tom Doherty Associates, LLC.
Broll, W., Schafer, L., Hollerer, T., Bowman, D. (2001). Interface with Angels: The Future of VR and AR Interfaces. IEEE Computer Graphics and Applications. November/December, 2001. Retrieved February 9, 2002, from http://www.computer.org/cga/cg2001/g6toc.htm
Brown, S. (2000, May). Law of Accelerating Returns. EE Times. Retrieved February 10, 2002, from http://www.eetimes.com/myf00/ceo_alpine.html
Callahan, Rick (2002). Camera May Be Able to Spot Liars. Associated Press, from FindLaw Legal News services. http://news.findlaw.com/ap/o/1501/1-2-2002/200201021009999506.html. January 2.
Clarke, A. (1984). The Profiles of the Future: An Inquiry Into the Limits of the Possible. New York: Henry Holt.
Colbridge, Thomas D. (2001). Kyllo vs. United States: Technology Versus Individual Privacy. Legal Digest. F.B.I. Law Enforcement Bulletin. October. Pp. 25-32.
Colgate, J., Wannasuphoprasit, W., Peshkin, M. (1996). Cobots: Robots for Collaboration with Human Operators. Proceedings of the International Mechanical Engineering Congress and Exhibition, Atlanta, GA, DSC-Vol. 58, pp. 433-39. Retrieved on February 15, 2002 from http://lims.mech.nwu.edu/ publications/jecolgate/IMECE96.Colgate.Wannasuphoprasit.Peshkin.html
Columbia University. MARS--Mobile Augmented Reality Systems. (n.d.) Retrieved February 2, 2002 from http://www.cs.columbia.edu/graphics/ projects/mars/mars.html
Defense Advanced Research Projects Agency. (2001). The Context Machine. Naval Post Graduate School. Monterey, CA. Retrieved February 4, 2002, from http://www.darpa.mil/ito/psum2001/m056-0.html
Defense Advanced Research Projects Agency. Synthetic Multi-Functional Materials (SMFM)--Thin Film Solid State Battery on Structural Fibers for Textiles and Composites with Integrated Power. Website. Retrieved February 10, 2002, from http://www.darpa.mil/dso/thrust/md/smfm/itn.html
Denning, P. (Ed.). (2002). The Invisible Future: The Seamless Integration of Technology Into Everyday Life. New York: McGraw-Hill.
Dertouzos, M. (2001). The Unfinished Revolution. New York: HarperCollins.
Everett, S., Wauchope, K., Perez-Quinones, M. (1998). Creating Natural Language Interfaces to VR Systems: Experiences, Observations and Lessons Learned. Future Fusion: Application realities for the virtual age, Proceedings of VSMM98, 4th International Conference on Virtual Systems and Multimedia, Vol. 2, pp. 469-474. IOS Press, Burke, VA. Retrieved February 10, 2002, from http://www.aic.nrl.navy.mil/~severett/VSMM98/VSMM98.html
Feiner, S. (1997). A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment. Proceedings ISWC '97 (International Symposium on Wearable Computing), Cambridge, MA. Retrieved February 10, 2002, from http://www.cs.columbia.edu/graphics/publications/iswc97.pdf
Feiner, S. (1999). The Importance of Being Mobile: Some Social Consequences of Wearable Augmented Reality Systems. Proceedings of IWAR '99 (International Workshop on Augmented Reality), San Francisco, CA, October 20-21, 1999, 145-148. Retrieved February 4, 2002, from http://www.cs.columbia.edu/graphics/publications/FEINERiwar99.pdf http://www.pnl.gov/microcats/fullmenu/minfuelcells.html
Harmon, Amy (2002). Good (or Unwitting) Neighbors Make for Good Internet Access. New York Times (March 4). Retrieved July 1, 2002, from: http://www.nytimes.com/2002/03/04/technology/04WIRE.html
Huxley, A. (1932). Brave New World. New York: Buccaneer Books, Inc. Interactive TV Makes Inroads. (2000). Reuters. New York Times December 1.
Jin, Q., Waibel, A. (2000). Applications of LDA to Speaker Recognition. International Conference on Speech and Language Processing, Beijing, China, October. 2000. Retrieved February 10, 2002, from http://www.is.cs.cmu.edu/mie/
Julier, S., Baillott, Y., Lanzagorta, M., Brown, D., Rosenblum, L. (2000). BARS: Battlefield Augmented Reality system. Naval Research Laboratory, Washington DC. Retrieved January 28, 2002, from http://www.ait.nrl.navy.mil/vrlab/papers/cp_NATO00.pdf
Kiser, K. (2000). Wearable Training. Inside Technology Training. April 2000. Retrieved February 10, 2002, from http://www. trainingsupersite.com/ittrain/pastissues/April00/apr00coverstory1.htm
Kramer, G., Walker, B., Bonebright, T., Cook, P., Flowers, J., Miner, N., Neuhoff, J., Bargar, R., Barrass, S., Berger, J., Evreinov, G., Fitch, W., Grohn, M., Handel, S., Kaper, H., Levkowitz, H., Lodha, S., Shinn-Cunningham, B., Simoni, M., Tipei, S. (1997). Sonification Report: Status of the Field and Research Agenda, Prepared for the National Science Foundation by members of the International Community for Auditory Display. Retrieved February 10, 2002, from http://www.icad.org/websiteV2.0/References/nsf.html
Kurzweil, R. (1999). The Age of Spiritual Machines: When Computers Exceed Human Intelligence. New York: Viking.
Kurzweil, R. (2001). The Law of Accelerating Returns. Retrieved February 10, 2002, from http://www.kurzweilai.net/meme/frame.html?main=/ articles/art0134.html
Magic Book: Project Description. Human Interface Technology Lab, University of Washington. Retrieved February 3, 2002, from http://www.hitl.washington.edu/magicbook/description.html
Mann, S. (1996). Wearable Tetherless Computer-Mediated Reality: WearCam as a Wearable face-recognizer and Other Applications for the Disabled. Presentation at the American Association of Artificial Intelligence, 1996 Symposium. Retrieved January 21, 2002 from http://wearcam.org/vmp.htm
Mann, S., Niedzviecki, H. (2001). Cyborg: Digital Destiny and Human Possibility in the Age of the Wearable Computer. Canada: Doubleday.
McAllister, D., Nyland, L., Popescu, V., Lastra, A., McCue, C. (1999). Real-Time Rendering of Real World Environments. Retrieved February 10, 2002, from ftp://ftp.cs.unc.edu/pub/publications/techreports/99-019.pdf
Meier, U., Stiefelhagen, R., Yang, J., Waibel, A. (2000) Towards Unrestricted Lipreading. International Journal of pattern Recognition and Artificial Intelligence, Vol. 14, No. 5, pp. 571-785, Retrieved February 16, 2002, from http://www.is.cs.cmu.edu/mie/
Merritt, T. (2001). Airport Privacy vs. Airport Security; Safer airports may come at the risk of embarrassing privacy invasions. December 11. http://www.techtv.com/cybercrime/story/0,23102,3349475,00.html
Micro Chemical and Thermal Systems. Compact Fuel Processors for Portable or Miniature Fuel Cells. Retrieved February 23, 2002, from http://www.pnl.gov/microcats/fullmenu/minfuelcells.html
Milgram, P., Kishino, F. (1994). A Taxonomy of Mixed Reality Visual Displays. IEICE Trans. Information Systems, Vol. E77-D, No. 12. Retrieved February 10, 2002, from http://vered.rose.utoronto.ca/people/paul_dir/IEICE94/ieice.html
National Institute of Justice. (2000). National Evaluation of the COPS Program--Title I of the 1994 Crime Act. (NCJ 183643) Washington, DC.
NYSTEC. (1997). Final Report for the New York State Police Communications Project--Phase I. Vol 3. Rome, NY: New York State Technology Enterprise Corporation. (Unpublished Report).
Office of Naval Research. Battlefield Augmented Reality System (BARS). Retrieved January 28, 2002, from http://www.ait.nrl.navy.mil/vrlab/projects/ BARS/BARS.html Patch, K. (2002). Hot Spots Give Away Lying Eyes. Technology Research News. Retrieved February 23, 2002, from http://www.trnmag.com/ Stories/2002/012302/Hot_spots_give_away_lying_eyes_012302.html
Peli, E. (2000). Visual Effects of Head Mounted Displays. The Schepens Eye Research Institute. Retrieved February 10, 2002, from http://www.microopticalcorp.com/Publications/Eli_Peli_Study.doc
Picard, R., Healey, J. (1997). Affective Wearables. Personal and Ubiquitous Computing, Vol 1. Retrieved February 3, 2002, from http://vismod.www.media.mit.edu/tech-reports/TR-467/index.html
Poulsen, Kevin (2002). "WiFi Honeypots A New Hacker Trap." SecurityFocus online (Monday, 29 July). Retrieved July 30, 2002, from http://online.securityfocus.com/news/552 Project Aura: Distraction-Free Ubiquitous Computing. Carnegie Mellon University. Retrieved February 10, 2002, from http://www-2.cs.cmu. edu/~aura/
Rauterberg, M. (1999). New Directions in User-System Interaction: Augmented Reality, Ubiquitous and Mobile Computing. IEEE Proceedings Symposium Human Interfacing: May 20, 1999. Retrieved February 9, 2002, from http://www.ipo.tue.nl/homepages/mrauterb/publications/IEEE99HI-paper.PDF
Robinett, W. (1992). Synthetic Experience: A Proposed Taxonomy. Presence: Teleoperators and Virtual Environments 1,2 (Spring 1992).
Robinson, Teri (2002). "Tracking People With Wireless Technology." Wireless NewsFactor, August 1. Retrieved August 1, 2002, from http://www.newsfactor.com/perl/story/18845.html
Roth, Jeffrey, Joseph P. Ryan, Stephen J. Gaffigan, Christopher S. Koper, Mark H. Moore, Janice A. Roehl, Calvin C. Johnson, Gretchen E. Moore, Ruth M. White, Michael E. Buerger, Elizabeth A. Langston and David Tacher (2002). National Evaluation of the COPS Program: Title I of the 1994 Crime Act. Washington: National Institute of Justice.
Shafer, S., Brumitt, B., Cadiz, J. (2001). Interaction Issues in Context-Aware Intelligent Environments. Retrieved February 10, 2002, from http://research.microsoft.com/easyliving/Documents/2001%2003%20Shafer.doc
Shim, Richard (2001). "TiVo Adds New Features to TV Recorder Service." New York Times (Saturday, January 6).
Smailagic, A., Siewiorek, D. (1999). User-Centered Interdisciplinary Design of Wearable computers. ACM Mobile Computing and Communications Review, Vol. 3, No. 3. pp 43-52. Retrieved February 10, 2002, from http://www-2.cs.cmu.edu/~asim/refs.html
Tristram, C. (2001, December). The Next Computer Interface. Technology Review, 53-59.
U.S. Department of Justice, Bureau of Justice Statistics. (2001). State and Local Law Enforcement Statistics. Retrieved February 3, 2002, from http://www.ojp.usdoj.gov/bjs/sandlle.htm#computers
U.S. Department of Justice, Bureau of Justice Statistics. (2001). Law Enforcement Management and Administrative Statistics. Local Police Departments 1999. Retrieved February 3, 2002, from http://www.ojp.usdoj.gov/bjs/pub/pdf/lpd99.pdf
Viire, E., Pryor, H., Nagata, S., Furness, T. (1998). The Virtual Retinal Display: A New Technology for Virtual Reality and Augmented Vision in Medicine. Proceedings of Medicine Meets Virtual Reality, San Diego, CA (pp. 252-257)
Wang, Z., Garlan, D. (2000). Task-Driven Computing. Project Aura, Carnegie Mellon University. Retrieved February 10, 2002, from http://www-2.cs.cmu.edu/~aura/docdir/wang00.pdf
Wrolstad, Jay (2002). "As WiFi Spreads, Insecurity Mounts." Wireless NewsFactor, Retrieved August 1, 2002, from http://www.newsfactor.com/perl/ story/18841.html
Wu, Y., Shan, Y., Zhengyou, Z., Shafer, S. (2000). Visual Panel: Toward a Vision-Based Mobile Input Interface for Anywhere. Retrieved January 28, 2002, from http://research.microsoft.com/easyliving/Documents/2000%2009%20Wu%20for%20CHI .pdf
Young, Jeffrey R. (2002). Self-Described 'Cyborg' Reveals Promise and Dangers of Wearable Computers; Engineering professor has been wired for 20 years. Chronicle of Higher Education. May 3. pg. A31. Retrieved June 10, 2002, from http://chronicle.com/free/v48/i34/34a03101.htm
Young, S. (2001). Statistical Modeling in Continuous Speech Recognition. Proceedings of the International Conference on Uncertainty in Artificial Intelligence, Seattle, WA, August 2001. Retrieved February 10, 2002, from http://svrwww.eng.cam.ac.uk/~sjy/publicats.htm
Thomas J. Cowper
New York State Police
Michael E. Buerger
Bowling Green State University
|Printer friendly Cite/link Email Feedback|
|Author:||Cowper. Thomas J.; Buerger, Michael E.|
|Publication:||Improving Our View of the World: Police and Augmented Reality Technology|
|Date:||Jan 1, 2004|