Animating evidence: computer game technology in the courtroom.

1. Introduction

Inevitably the future will be increasingly digital. The continuing digital revolution has had an enormous impact on the way forensic evidence is collected, analysed and interpreted and has even led to the defining of new types of digital evidence (for example, digital imagery and video, hard drives and digital storage devices). Much of this digital media will end up needing to be admitted into courtrooms as evidence. In most jurisdictions around the world technology can be slow to become legally accepted. It is fair to say that, in general, legislation for the admissibility of digital media usually lags behind the technological development (Schofield and Goodwin, 2007).

In a modern courtroom, the presentation of forensic evidence by an expert witness can bring about the need for arduous descriptions by lawyers and experts to get across the specifics of complicated scientific, spatial and temporal data. These technological advances have also meant that experts have had to develop new ways to present such complex evidence in court. Digital visual evidence presentation systems (including digital displays, computer-generated graphical presentations and three-dimension simulations) have already been used in many jurisdictions. These visual tools can be used to present evidence and illustrate hypotheses based on scientific data, or they may be used to depict the perception of a witness, such as what may have occurred (seen from a specific viewpoint) during a particular incident. Digital reconstruction technology may also be applied in a courtroom to explore and illustrate 'what if' scenarios and questions, testing competing hypotheses and possibly exposing any inconsistencies and discrepancies within the evidence (Burton et al, 2005).

It is important to realise that the use of such computer-generated presentations in a courtroom is only the current manifestation of evidence illustration and visualisation in a long history of evidential graphics used in litigation (Schofield and Goodwin, 2007). However, computer animations and interactive virtual simulations are unparalleled in their capabilities for presenting complex evidence. The use of such enabling visualisation technology can affect the manner in which evidence is assimilated and correlated by the viewer; in many instances, it can potentially help make the evidence more relevant and easier to understand (Tufte, 1985; Burton et al, 2005; Mervis, 1999).

At this point, it is perhaps worth defining and describing the technologies under discussion in this paper. Over the past ten years, visual evidence displays and digital courtroom presentation systems have developed to cover a wide variety of technologies (O'Flaherty, 1996; Schofield and Goodwin, 2007). This paper focuses on the evidential use of computer-generated imagery, particularly computer graphics.

Computer graphics in this context refers to a suite of software applications that can be used to produce outputs such as rendered images and animations(2). Computer graphics systems can utilise numerical three-dimensional models of real world objects to create artificial virtual environments. Based on scene survey data, objects such as equipment, vehicles, human figures, environment details, landscape features and other relevant evidence items can be accurately positioned and precisely scaled within the artificial three-dimensional environment. The scene objects can then be texture mapped with relevant photographic images to produce a credible lifelike appearance (Watt, 1999; Foley et al, 1995).

Computer technology can be employed to build an animation from one these virtual environments, this is usually rendered frame-by-frame (as a series of still images). These frames, when played back in quick succession, create an experience of space, motion and time. Popular cultural examples of this technique include animated films and movies such as Shrek by Dreamworks Animation <http://www.imdb.com/title/tt0126029> and Beowulf by Imagemovers <http://www.imdb.com/title/tt0442933/> (3). In a courtroom context the term often used to describe evidence presented in this format is forensic animation.

Virtual reality involves interactive, real-time, three-dimensional graphical environments that respond to user input and action, such as moving around in the virtual world or operating virtual equipment. An important aspect of such a virtual reality system is its underlying processes, simulations, behaviour and reactions, and the way a user can interact with objects within the virtual world. A virtual reality user could, for example, sit in a virtual vehicle and drive it. Popular cultural examples of this technique include modern three-dimensional computer games such as Unreal Tournament 3 <http://www.epicgames.com> and Grand Theft Auto: San Andreas <http://www.rockstargames.com> (4). In a courtroom context the term often used to describe evidence presented in this format is virtual simulation or virtual reconstruction (5).

At first glance, these graphical reconstructions may be seen as potentially useful in many courtroom situations, and they are often treated like any other form of digital evidence regarding their admissibility (Schofield and Goodwin, 2007). However, perhaps this specific form of digital media warrants special care and attention due to its inherently persuasive nature, and the undue reliance that the viewer may place on the evidence presented through a visualisation medium, this is often referred to as the 'seeing is believing' tendency (Galves, 2000; Girvan, 2001; Spiesel et al, 2005; Sherwin, 2007).

The need for a clear presentation of evidence by an expert (forensic) witness is summed up by Burns (2001) who states:

'The presentation typically takes the form of a report, and the scientist must be prepared to explain this report in such a way that a typically science-phobic judge and jury are able to comprehend it. Presentation is everything.'

2. Visual Evidence

A visualisation or graphic can potentially be a valuable aid to help construe and convey a large amount of complex information. An American judge, Rubin (Rubin, 1993) highlighted the problem of retaining the interest of the jurors when he stated:

'It isn't difficult to tell when jurors have lost interest ... Such wandering attention is much less likely in a paperless trial, because the evidence is presented in a format jurors are used to watching ... I have noticed repeatedly that when a document is displayed on the monitors, the jurors sit up and pay attention. Such attention is far greater than that given to a document which they cannot see as it is being discussed by the attorney and the witnesses ...'

The above comment illustrates the potential benefit in a courtroom environment of reducing lengthy verbal explanations and increasing the use of visual tools (this applies even to simple displays, such as presenting a text document on a screen). Visual displays can often act to improve the viewer's ability to retain the evidence, maintain an interest in the proceedings, and help them to more fully understand the nature of the evidence (Loftus and Loftus, 1980; Leader and Schofield, 2006). Backing up this claim, a survey by the American Bar Association found that members of a jury are often confused, bored, frustrated and overwhelmed by technical issues or complex facts (Kuehn, 1999). Other research has indicated that the attention span of the average member of a jury in a standard trial in court may be as little as seven minutes (Devine et al, 2001).

Historically, static images such as diagrams and charts have been used to explain the complex testimony of an expert witness. Forensic animations or virtual reconstructions are unique in their ability to visually manipulate, animate and illustrate the passing of time. This extra temporal dimension can be extremely useful when explaining a chronological sequence of events, such as in the reconstruction of a vehicle accident, where the dynamic movement of the vehicles involved may be dependent on complicated and difficult to explain engineering or mathematical principles (Schofield, 2006). These virtual environments can also be used to take advantage of their lack of physical restrictions, which may allow the viewer to be placed in a position where it is physically impossible to be with a normal camera (such as inside an engine or a human body), to show views of a crime or accident from previously unseen points of view, or to slow down or speed up time (Jones et al, 1991).

Research work, previously undertaken in the USA, has examined how members of a jury retain details in their memory from different forms of evidence:

* One American study showed that the average person retains 87 per cent of information presented visually, but only 10 per cent of information presented orally (Seltzer, 1990).

* Another study showed that the average person retains 65 per cent of information presented visually and 15 per cent of what is presented orally (Cobo, 1990).

* A further American survey showed that members of a jury will retain twice the amount of information when using a visual presentation, as distinct to an oral presentation (Krieger, 1992).

When the evidence is animated, the improvement in memory retention can be even more apparent: another survey revealed that members of a jury will retain an increase of 650 per cent of information when presented with presentations using a form of computer animation (Thomas, 1995).

However, the precise effect that visual imagery has on members of a jury, witnesses and other viewers in the court is not known, and concerns are beginning to be articulated that the use of modern computer-generated visualisation technology can distort perceptions, memories, attitudes and decision making in the court (Girvan, 2001; Spiesel et al, 2005; Bailenson et al, 2006; Schofield, 2007).

Recently, the Visual Persuasion Project, run by New York School of Law, (Sherwin, 2002) has identified a number of issues and problems with the use of visual technology. The goal of this project is to promote a better understanding of the practice, theory, and teaching of law in the current screen-dominated, pervasively visual, digital era <http://www.nyls.edu/pages/2734.asp>.

Australia currently has a couple of projects underway in this thematic area. In Western Australia, rare permission has been given by the Attorney General for a researcher to interview jurors after criminal trials in which a range of expert evidence was presented. Even though the data collection component of this study was only completed in 2007, some preliminary results have already been published (Fordham, 2006).

[FIGURE 1 OMITTED]

The author is currently involved in a large research project, the Juries and Visual Evidence Project (JIVE) which is also examining some of these issues (6). The project intends to measure the impact of interactive displays on the trial process; specifically whether forensic animation and virtual reconstruction technology better informs juries or potentially increases prejudice against defendants. In January 2008, the JIVE project team ran a number of mock trials in Sydney where a range of forensic animations and interactive reconstructions of evidence relating to a terrorist bombing were shown to a number of different groups of jurors <http://www.justiceenvironments.edu.au/project-1>. Images from the virtual reconstruction are shown in figure 1. The data from these trials is currently being collated and analysed, it is expected that the results of this study will be ready for publication in the near future.

Kassin and Dunn (1997) undertook two experiments to assess the effects of computer-animated displays on mock jurors. In both, participants watched a trial involving a dispute over whether a man who fell to his death had accidentally slipped or jumped in a suicide. They observed that when the plaintiff and defense used a forensic animation to depict their own partisan theories, participants increasingly made judgments that contradicted the physical evidence, suggesting that computer-animated displays have greater impact than oral testimony.

Other research by Selbak (1994) also examined the prejudicial effects of computer-generated animations. This research work led to the formulation of a number of suggested legal guidelines for animation to be used in American courtrooms.

At present, to the author's knowledge, there is little other current active research work in this area. The following sections contain some recent examples of where this technology has been applied and a number of comments and reflections on its application in a number of these cases.

3. Visual Evidence Examples: Forensic Animation

Computer-generated graphical evidence in the US has primarily been used in civil cases. One of the first major uses of forensic animation took place in the federal civil case for the Delta flight 191 crash. In August 1985 the Delta airplane with 163 people aboard was caught in a wind vortex and crashed while attempting to land at Dallas-Fort Worth Airport, a mile from the runway. In the subsequent litigation the US Government offered a 55-minute computer-generated presentation, including forensic animations to the court to explain details pertaining to each item of evidence (Marcotte, 1989).

There is an extensive precedent concerning the use of a range of computer-generated evidence in the United States, but very little in comparison in many other jurisdictions. Consequently, judges in other jurisdictions may look to the US for guidance in considering issues of admissibility. This has been particularly true for the introduction of computer-generated animations and virtual simulations in courtrooms in the UK and Australia (Schofield and Goodwin, 2007).

The case of R v. Ore is thought to have introduced the first forensic computer-generated animations to an English criminal trial. The Crash Investigation and Training Unit of the West Midlands Police Service produced the animation. The case involved a collision between two vehicles at a junction; one of the drivers was killed as he pulled out in front of an oncoming vehicle. The views of both drivers were partially obscured by large hedges and walls around the junction (Doyle, 1997).

Justice Tucker, who presided over this case, further stated in his ruling on 25 November 1998 (Schofield and Goodwin, 2007):

'I am told that this is the first time in which it has been suggested that a jury in a trial such as this should be shown a computer aided animation which pictorially represents a reconstruction of a road traffic accident. It may be that in years to come such displays will be commonplace and that lawyers will marvel that anyone should ever have questioned their admissibility.

... I am satisfied that it would be right to admit this evidence and, indeed, wrong to refuse so to do, provided, as I shall try to do, that I give the jury proper directions as to their approach to this evidence and provided I ensure, so far as I can, that they do not place disproportionate weight upon it. Accordingly, I rule that the evidence is admissible.'

A further influential example of the use of this technology, from Northern Ireland, is a reconstruction that has come to be regarded as highly relevant, unbiased and accurate. Computer-generated visual evidence has been used extensively at the very high profile, Bloody Sunday Inquiry <http://www.bloody-sunday-inquiry.org>. In 1972, thirteen people were killed in a peaceful demonstration that went awry. The original inquiry produced a report within 11 weeks of the incident, and acquitted the soldiers involved. In 1998, a Tribunal of Inquiry was established to reassess the events. Lord Saville, the chair of the tribunal, took full advantage of technology, and a computer-graphics based software system was designed especially for use in the Inquiry, to amplify the testimony of witnesses (Powell, 2008).

The computer-generated virtual models reconstructed a large area of Londonderry, which was extensively altered since 1972. The user was able to compare the same scene as it appeared both recently and in 1972. There were 80 locations stored in the system, which could be explored, with any point of view recalled when switching between the representations. The system could also store oral evidence about location and movement, and export scenes to a mark-up system so that witnesses could draw on top of images <http://www.bloody-sunday-inquiry.org>.

The following sections describe a number of specific examples involving the use of forensic animations from the author's own case portfolio. Each has been carefully selected to allow discussions around specific aspects relating to the application of this form of graphical technology in a courtroom environment.

3.1 Forensic Animation 1: Road Traffic Accident

Presenting data related to road traffic accidents in the courtroom is a prime example of the need to relate spatial and temporal data, for which the use of virtual environment technology has been extensively adopted (Schofield et al, 2001). In such cases, a computer-generated forensic reconstruction is built using a three-dimensional virtual environment of a scene created from actual measurements (which are usually taken by the police or investigators at the time of the incident). Dynamic vehicle movements are often then simulated using scientific calculations based on those measurements and the experience of the reconstruction engineer. This computer model can then be rendered to create a series of images and animations, which describe the scene or incident. These virtual environments, when viewed in court, must support and corroborate existing evidence to be admissible as substantive evidence in any courtroom (Noond and Schofield, 2002).

Such computer games technology has been used to recreate a number of road traffic accidents in both criminal and civil courtrooms in the UK. In these cases it has been possible to show views of these road traffic accidents from the viewpoints of the vehicles and pedestrians involved, and also from the viewpoint of a range of witnesses (Noond et al, 2002) (7).

[FIGURE 1 OMITTED]

The image shown in figure 1 is a view from evidence created regarding a road traffic accident where two motorcyclists were killed when they collided with the side of a car that had pulled across the road in front of them. This particular case involved the death of an off-duty motorcycle policeman and was one of the first times that a forensic animation was admitted in a UK courtroom. This case hence garnered a significant amount of national media attention (Tendler, 2000).

In this simulation the vehicle dynamics were based on the data and calculations from two independent accident reconstruction engineers. A number of collision scenarios were simulated based on the evidence, hypotheses and arguments from both legal teams. This allowed each of the lawyers to each demonstrate their expert's evidence in an efficient and easy to understand manner (Byass, 2000).

In this particular case the crucial piece of evidence under scrutiny in court related to the view of the driver of the car that pulled in front of the two bikes. It was seen as important to determine whether or not he had a clear view of the bikes as they approached. Virtual cameras were placed inside the car, replicating the view of the driver down the virtual road. These forensic animations showed that the street furniture in the area (street lights and traffic bollards) had potentially obstructed the view of the driver during the crucial moments as he decided to turn his vehicle into the side road.

This case reports a significant result, where it would have been difficult to generate such crucial evidence by other means. Most people would be positive about the benefits of the forensic animations used in this case and understand how the virtual reconstruction helped the jury to assess the burden of guilt in this case.

However, there are a number of issues and questions that appear when such a reconstruction is examined in further detail. For example :

* How can one correlate the lighting in the virtual world with that available at the scene at the time of the incident? Is an approximation good enough? One could argue that in this case this is not crucial as it is only the line of sight which is under investigation, not the illumination of the objects (Walter et al, 1997).

* How can one correlate the field of view of the driver of the vehicle with the field of view of the virtual camera? This also relates to the display mechanisms used, where a viewer watching a computer monitor does not have the same experience (depth of field, motion parallax, peripheral vision etc.) as a viewer watching a 'live' event (Kanade et al, 1997; Tromp and Schofield, 2004).

* How can one correlate the resolution of the virtual scene with that subjectively perceived by the real world viewer? In this instance resolution not only refers to the image dimensions (pixel count) but also to the level of photorealism of the virtual environment created (Brooks, 1999).

3.2 Forensic Animation 2: Pathology Visualisation

The image in figure 2 shows a pathology reconstruction used in a murder case to investigate the nature of a stabbing incident. In this case the autopsy report described the injuries sustained by a 30year-old male who had received a number of blunt force injuries to the face and chest, and a stab wound to the back measuring 3.4cm in length (March et al, 2004).

The cause of death was attributed to the extensive internal bleeding caused by the stab wound to the back which pierced the heart. It was also concluded that a large amount of force would be necessary to cause the incision to the eleventh thoracic vertebra and that the bruising to the victim's body suggested some degree of violent struggle prior to the fatal injury (Noond et al, 2002; March et al, 2004).

[FIGURE 2 OMITTED]

The left hand image in figure 2 shows the angle of the blade as it entered the body, cutting through the vertebra. The right hand image shows a hypothetical body dynamic produced to illustrate the position of the victim illustrating the position of the body so that the damage to the internal organs matches up with the angle of the knife entry.

One can see obvious benefits to the use of such forensic animation evidence, particularly as it is arguably less emotive and prejudicial than members of the general public being exposed to, often gruesome, post-mortem photographic evidence (March et al, 2004). However, as in the previous case, a number of potential issues and questions arise from the use of such technology for pathology visualisation:

* Unlike the previous road traffic accident scene reconstruction where exact measurements were available, pathology visualisations are often created based on descriptive post-mortem findings and approximate measurements. In this particular case the advice of the pathologist was seen as crucial in creating a forensic animation which accurately matched the medical opinion (8).

* The use of anatomical computer-models allows the recreation of the dynamic event in which the wound or damage occurred. However, such a reconstruction is, by its very nature, often dependant on the knowledge, expertise and opinion of medical experts. Of particular note in this particular case are the hypothetical limb positions of the victim (shown in the right hand image of figure 2). These must be viewed cautiously and the uncertainty associated with their exact position must be explained to the viewer (Noond et al, 2002; March et al, 2004).

3.3 Forensic Animation 3: Bullet Trajectories

The third forensic animation case involves a virtual reconstruction created for the Independent Police Complaints Commission (IPCC) in the UK. This reconstruction related to the fatal shooting of a civilian by police armed response unit in 2005 (Schofield, 2007).

[FIGURE 3 OMITTED]

The images in figure 3 demonstrate the level of photorealism it is possible to achieve using modern computer games development software. The image on the left of figure 3 is a photograph of the exterior of the building where the shooting incident occurred, the image to the right shows the virtual reconstruction of the building environment (9).

This particular case involved matching a possible bullet trajectory from the police marksman outside the front of the building to the victim, who was stood at a first floor window. To do this a transparent mannequin was used allowing the bullet trajectory (shown in the images in figure 4) to be visible throughout the animation.

[FIGURE 4 OMITTED]

This particular case also introduced another technology into UK courtrooms, where the animation was played alongside corresponding audio tracks taken from the police radio talkback and armed response units communication channels. Although, in this case, the forensic animation was able to explain a number of potential issues regarding the bullet trajectory, a number of other possible issue emerge:

* The photorealistic rendering of components of the virtual model, may possibly lull the viewer into a 'seeing-is-believing' attitude, causing a potential relaxation of their critical faculties (Girvan, 2000; Spiesel et al, 2005; Sherwin, 2002).

* The integration of real-world audio evidence with a forensic animation has been used in the United States for many years (10). Research suggests that adding audio to a computer-generated visual can have a major impact on the level of engagement of the viewer, and hence may potentially affect their understanding and interpretation of the evidence viewed (Hendrix and Barfield, 1996).

* Lastly the mixing of visual metaphors and modes may be potentially disorientating to some viewers. Combining abstract human representations in photo-realistic environments may provide an unnatural experience for the viewer. Fielder (2003) has commented on the way juries may be misled by the use of visual metaphors and abstract representations in forensic animations.

4. Visual Evidence Examples: Real-Time Simulation

Stephenson v. Honda Motors Ltd. of America (11) is generally accepted to be the first case to admit evidence using a computer game engine (real-time simulator). The attorney convinced a California Superior Court of the need to use the visual component of a virtual reality simulation to help a jury understand the nature of the terrain over which an accident victim chose to drive her Honda motorcycle <http://www.lectlaw.com/files/lit04.htm>. Honda argued that the terrain was obviously too treacherous for the safe operation of a motorcycle, and that, while two-dimensional photographs and videos would help provide the jury with some idea of what the terrain was like, a three-dimensional, interactive simulator was much more realistic. In allowing the evidence, the court determined that the three-dimensional view was more informative, relevant, and probative (12).

Since this initial success, the sporadic worldwide application of such computer game based, real-time technology in courtroom situations has (in most cases) offered a unique platform for the collation, interrogation, analysis and presentation of complex forensic data across a wide spectrum of crime-scene and accident scenarios. Three-dimensional reconstructions of incidents have allowed the user to interactive visualise views from multiple relevant positions within the virtual environment, something that can be beneficial within the dynamic, adversarial environment of the courtroom.

Again the US has a larger precedent for the admissibility of such technology into courtrooms. However, the following sections describe a number of specific examples involving the use of virtual simulations from the author's own case portfolio. Each has again been carefully selected to allow discussions around specific aspects relating to the application of this form of graphical technology in a courtroom environment.

4.1 Virtual Reconstruction 1: Debris Visualisation and Identification

The first case in the UK known to utilise an interactive three-dimensional real-time virtual simulator to present evidence in court was the case of the murder of a motorcycle policeman in Birmingham, UK. In October 2002, a vehicle was stolen at a petrol station when the driver had left it running while buying cigarettes. The car was stolen by three men, one of whom was the driver Nicholas Walters.

PC Malcolm Walker saw the car shoot over a nearby red light and followed the car but when Walters realised PC Walker was in pursuit, he stopped the vehicle, waited until the officer drew up alongside him and then deliberately drove into his motorbike forcing him off the road. PC Walker suffered multiple fatal injuries after being violently thrown from his bike with enough force to uproot a concrete bollard <http://news.bbc.co.uk/1/hi/england/2460325.stm>.

A large scale interactive virtual environment (shown in the left hand image of figure 5) was created reconstructing the area over which the incident occurred (over a kilometer of roadway was modelled). It was specifically designed to allow the viewer to examine and identify all debris found at the scene of the incident, by using context-sensitive links to the digital photographic evidence (collected by the 'scene of crime' officers) and to relevant forensic evidence (an example of a link to a scene photograph is shown in the right hand image of figure 5). Over 300 individual items of evidence could be identified in the virtual reconstruction of the scene.

This virtual simulation was used during the trial as a primary evidence display mechanism and helped to successfully convict the principal defendant and his associates of murder (West Midlands CPS, 2003).

[FIGURE 5 OMITTED]

One of the main advantages of the use of an interactive virtual simulation such as this is the ability to control the virtual camera movement within the environment (Ware and Osborne, 1990). What this means, in a courtroom setting, is that the user can 'interactively' potentially adjust the view of the digital evidence--for example, a witness could move a camera around until the virtual view matches their memory of their view of the incident.

The use of an interactive virtual simulator, such as the one described above, creates a number of particular issues and questions relevant to the use of this specific technology. For example :

* How can one correlate the viewpoint of a witness in a virtual environment with the view from their real-world position at the scene? It can be reasonably assumed that most people will be better able to correlate their spatial location from a three-dimensional simulation than they could on a two-dimensional plan. However, the ways in which humans position themselves and correlate spatial information between three-dimensional views of virtual and real worlds are not fully understood (Montello et al, 2004).

* The linking of 'real' evidence to spatially contextualised hotspots in the virtual environment potentially provides a potential mechanism for the viewer to easily understand the spatial relationship of the evidence. However, the ways humans organise and remember spatial relationships between objects in virtual and real world information are not fully understood (Arthur et al, 1997).

* It should never be forgotten that a virtual simulation, is by its very definition a 'simulation' of reality. In the courtroom context, there is a need to understand the nature of the simulation and the veracity of the representation, i.e. how close is it to the original evidence from which it was derived (Fielder, 2003 and Schofield, 2007). For example, the vehicle movement in a road traffic accident simulation may be based on the same equations as used by the accident reconstruction expert witness. However, does the simulation apply them in the same way? Does the simulation work to the same level of accuracy? Does the simulation make the same assumptions as the expert witness?

4.2 Virtual Reconstruction 2: Drive By Shooting Reconstruction 

For the high profile investigation of a drive by shooting in Birmingham, UK, the police commissioned a large-scale virtual reconstruction to simulate the events of the evening of the incident. Letisha Shakespeare and Charlene Ellis died when they were sprayed with bullets from a sub-machinegun as they stood with other revellers outside the back entrance of a hairdressing salon where they had been attending a New Year party. The dead girls, both aged 18, were caught in the crossfire when a gun battle broke out between two rival gangs. Charlene's twin sister Sophie and their friend Cheryl Shaw were injured in the incident, which police have attributed to a feud between rival gangs (Britten, 2003).

These well-publicised killings prompted a national outcry over the rise of gun crime in Britain. But police struggled to obtain eyewitness testimony; even though more than two dozen people were outside the salon when the shooting occurred. Most of those present fled and then refused to come forward, apparently out of fear of retribution from gang members. Due to the large numbers of hostile witnesses, many statements that were obtained from the police were contradictory about the spatial locations of key people around the time of the shootings.

An interactive crime scene briefing tool was created (figure 6), with all people and vehicles involved represented in the virtual environment (over sixty moving objects) over a two hour time window. Objects were positioned based on CCTV footage, physical evidence recovered from the scene and witness testimony. This allowed the user to view the crime scene and event chronology in an interactive way, updating the virtual evidence as and when new information came to light (Burton et al, 2005; Schofield and Goodwin, 2007; Schofield, 2007). The final version of this briefing tool was then used in court to aid in the conviction of four men for the shootings <http://news.bbc.co.uk/2/hi/uk_news/england/west_midlands/4366177.stm>.

[FIGURE 6 OMITTED]

A high degree of realism was created in virtual world to the extent that even the graffiti on the walls was photographed, measured and re-modelled accurately in the three-dimensional simulation. However, many of the objects in the virtual environment were modeled varying degrees of accuracy. For example, when a vehicle was known (owner, model, colour etc.) and could be accurately placed in the scene, a near photorealistic model was created, however, if a vehicle's details were unknown or there was conflicting evidence about the vehicle, it was modelled to a lower level of detail (as shown in figure 6).

The simulation contained a media control allowing the viewer to jump to, and play, the simulation from any point along the incident chronology. This media control contained play, stop and pause buttons, a display showing the time in the simulation and a sliding time scale, similar to the interface of Windows Media Player <http://www.microsoft.com/windows/windowsmedia/player/default.aspx>. This feature allowed investigators to view and assess the spatial position of objects in the world over time and correlate this movement with witness testimony.

Again, the use of an interactive briefing tool, such as the one described above, creates a number of particular issues and questions relevant to the use of virtual simulators. For example:

* The primary distinction between this and the previous forensic animations and virtual simulators are the way the technology was used. This reconstruction was created as an interactive briefing tool, allowing investigators to familiarise themselves with the evidence and test hypotheses. The simulation was even run on laptops by investigators who were walking the scene long after the incident (i.e. all transient evidence had been removed). The user was able to walk round the real scene environment while simultaneously moving through the virtual environment; jumping to points in the chronology, correlating the virtual evidence of the event spatially with their real-world view (Burton et al, 2005; Schofield, 2007).

* The mixing of levels of detail and degrees of photorealism may be potentially disorientating to some viewers. Combining different levels of detail and expecting the viewer to draw additional information from abstract representations in the virtual environment may overload the viewer and potentially add to the confusion (Fielder, 2003).

* The ability to jump through time, along the event chronology, in the virtual environment may be potentially disorientating to the viewer. Most viewers are used to linear narratives and may struggle to keep following specific narrative threads with such a non-linear approach (Craven et al, 2001).

5. Conclusion

Forensic science technology advances rapidly and the public, who regularly watch high-technology crime scene investigation on television, expect to see their TV experience duplicated in the real courtroom environment. The public expects professional visual representations illustrating complex forensic evidence, polished digital media displays demonstrating the location of spatially distributed evidence and dynamic animated graphics showing event chronologies. Many lawyers and expert witnesses now use, and have to confront in an adversarial manner, forensic animations, three-dimensional virtual reconstructions, real time interactive environments and computer-generated simulations.

Gerald Lefcourt, a criminal defence lawyer in New York, recently made the following comments about members of the public who attend court <http://www.abc.net.au/news/newsitems/200509/s1467632.htm>:

'These are people who by and large have grown up on television ... The day of the lawyers droning on is really gone. I think that jurors today, particularly the young ones, expect quickness and things they can see.'

Our culture is dominated with images whose value may be simultaneously over-determined and indeterminate, whose layers of significance can only be teased apart with difficulty. Different academic disciplines (including critical theory, psychology, education, media studies, art history, semiotics etc.) help explain how audiences interpret visual imagery (Tufte, 1985). However, the analysis of courtroom imagery and its interpretation by jurors (and other courtroom participants) is only just beginning (Spiesel, 2006).

Around the world a number of lawyers and forensic experts are already beginning to utilise 'slick' visuals to replace rhetoric and depend on their audience adapting a 'seeing is believing' attitude to persuade juries to believe in their arguments (Galves, 2000; Girvan, 2001; Spiesel, 2005; Bailenson, 2006). Many of these graphical displays are being created by contractors and consultants who operate externally to the organisations involved in the case (e.g. judiciary, legal team, police service or forensic laboratory) (13).

Whether one likes it or not, in the future the technology used to generate computer games is going to be increasingly used to generate advanced visual evidence presentations in a number of courtroom jurisdictions around the world. It is imperative that researchers and practitioners start to examine the implications of this technology, evaluate its potential advantages and disadvantages and assess its impact in the courtroom.

Those who present evidence in court (including lawyers, police officers and expert witnesses) may need different skills to collect and describe evidence relevant for animations. Those who view these animations in the court may also need a new skill set to allow them to interpret what they see. Potentially there is a need for specialised training for those who have to present graphical evidence and specific instructions for those who need to view and interpret that evidence (Selbak, 1994; Schofield, 2007).

This paper has, hopefully, been fairly positive about the future and the benefits that can arise through the introduction of this technology into our courtrooms. However, there are a number of issues and concerns that arise through the use of forensic animations and virtual reconstructions to present evidence. These are not reasons in themselves for abandoning the use of this technology, but rather aspects that need to be investigated further and safeguards and guidelines put in place to avoid any possible misuse of this technology.

Lastly, there is one possible benefit of this technology that is often overlooked. Some of the oft forgotten groups of people in the courtroom are the families and friends of the victim (or defendant) and other members of the general public. Many of these are unfamiliar with the courtroom environment and often find it intimidating. In the author's experience, when forensic animations or virtual reconstructions are appropriately used, these groups frequently express their gratitude regarding the clarity of the visual evidence. Often, their desire to appreciate the evidence and to understand what happened is every bit as keen as that of the other, more habitual, courtroom participants.

This is a refereed article published on 28 May 2009.

Citation: Schofield, D., 'Animating Evidence: Computer Game Technology in the Courtroom', 2009(1) Journal of Information Law & Technology (JILT), <http://go.warwick.ac.uk/jilt/2009_1/scofield>

6. References

Arthur, EJ, Hancock, PA and Crysler, ST (1997), 'The Perception of Spatial Layout in Real and Virtual Worlds', 40(1) Ergonomics, 69-77.

Bailenson, JN, Blaschovich, J, Beall, AC and Noveck, B (2006), 'Courtroom Applications of Virtual Environments, Immersive Virtual Environments and Collaborative Virtual Environments', 28(2) Law and Policy, 249-270.

Britten, N (2003), 'We Know Who Killed the New Year's Day Party Girls', The UK Daily Telegraph, 26th October 2003.

Brooks, FP (1999), 'What's Real About Virtual Reality?' 19(6) IEEE Computer Graphics and Applications, 16-27.

Bryce, J and Rutter, J (2002), 'Spectacle of the Deathmatch: Character and Narrative in First-Person Shooters', in King, G and Krzywinska, T (eds.) ScreenPlay: Cinema/Videogames/Interfaces (London: Wallflower), 66-80.

Burns, DC (2001), 'When Used in the Criminal Legal Process Forensic Science Shows a Bias in Favour of the Prosecution', 41(4) Science and Justice, The Journal of the UK Forensic Science Society, 271-277.

Burton, A, Schofield, D and Goodwin, L (2005), 'Gates of Global Perception: Forensic Graphics for Evidence Presentation', Proceedings of ACM Symposium on Virtual Reality Software and Technology, Singapore November 6th-12th 2005, 103-111.

Byass, R (2000), 'Animated Crash KO's Inspector', Gazette (Journal of the UK Law Society) October 2000.

Craven, M, Taylor, I, Drozd, A, Purbrick, J, Greenhalgh, C and Benford, S (2001), 'Exploiting Interactivity, Influence, Space and Time to Explore Non-Linear Drama in Virtual Worlds', Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Seattle, USA, 1st-5th April 2001, 30-37.

Cobo, R (1990), 'An Approach to Demonstrative Exhibits', 3 PLI/Lit 359.

Devine, DJ, Clayton, LD, Dunford, BB, Seying, R and Pryce J (2001), 'Jury Decision Making, 45 Years of Empirical Research on Deliberating Groups', 7(3) Psychology: Public Policy and Law, 622-727.

Doyle, M (1997), 'Working Model: Helping the Police with Their Enquiries', CAD User, October 1997, 62-63.

Fielder, B (2003), 'Are Your Eyes Deceiving You? The Evidential Crisis Regarding the Admissibility of Computer-Generated Evidence', 48(1&2) New York Law School Law Review, 295-321.

Foley, JD, van Dam, A, Feiner, SK and Hughes, JF (1995), 'Computer Graphics: Principles and Practice in C', (London: Pearson).

Fordham, JG (2006), 'Illuminating or Blurring the Truth: Jurors, Juries, and Expert Evidence' in Brooks-Gordon, B and Freeman, M (eds.), 9 Law and Psychology, 338-360.

Galves, F (2000), 'Where the Not So Wild Things Are: Computers in the Courtroom, the Federal Rules of Evidence, and the Need for Institutional Reform and More Judicial Acceptance', 13(2) Harvard Journal of Law and Technology, 161-302.

Girvan, R (2001), 'An Overview of the Use of Computer-Generated Displays in the Courtroom', 7(1) Web Journal of Current Legal Issues 

Heintz, ME (2002), 'The Digital Divide and Courtroom Technology: Can David Keep up With Goliath?', 54 Federal Communications Law Journal, 567-589.

Hormann, JS (1995), 'Virtual Reality: The Future of Law Enforcement Training', FBI Law Enforcement Bulletin, July 1995.

Hendrix, C and Barfield, W (1996), 'The Sense of Presence Within Auditory Virtual Environments', 5(3) Presence: Teleoperators and Virtual Environments, 290-301.

Kanade, T, Rander, P and Narayanan, PJ (1997), 'Virtualized Reality: Constructing Virtual Worlds From Scenes', 4(1) IEEE Multimedia, 34-47.

Kassin, S and Dunn, MA (1997), 'Computer-Animated Displays and the Jury: Facilitative and Prejudicial Effects', Law and Human Behaviour, 269-281.

Krieger, R (1992), 'Sophisticated Computer Graphics Come of Age--and Evidence Will Never Be the Same', Journal of the American Bar Association, December 1992.

Jones, IS, Muir, DW and Groo, SW (1991), 'Computer Animation--Admissibility in the Courtroom, Accident Reconstruction: Technology and Animation', 1 Society of Automotive Engineers, 143-151.

Kuehn, PF (1999), 'Maximizing Your Persuasiveness: Effective Computer Generated Exhibits', DCBA Brief: Journal of the DuPage County Bar Association, October 1999.

Leader, L and Schofield, D (2006), 'Madness in the Method? Potential Pitfalls in Handling Expert Evidence', 6(1) Journal of Personal Injury Law, 68-86.

Lederer, FI (2004), 'Courtroom Technology: For Trial Lawyers, The Future Is Now', 19(1) Criminal Justice, 14-21.

Loftus, ER and Loftus, GR (1980), 'On the Permanence of Stored Information in the Human Brain', 35(5) American Psychologist, 409-420.

MacDorman, KF (2006), 'Subjective Ratings of Robot Video Clips for Human Likeness, Familiarity, and Eeriness: an Exploration of the Uncanny Valley', Proceedings of the ICCS/CogSci2006 Long Symposium: Toward Social Mechanisms of Android Science, Vancouver, Canada, July 26th 2006, 26-29.

March, J, Schofield, D, Evison, M and Woodford, N (2004), 'Three-Dimensional Computer Visualisation of Forensic Pathology Data', 25(1) American Journal of Forensic Medicine and Pathology, 60-70.

Marcotte, P (1989), 'Animated Evidence: Delta 191 Crash Re-Created Through Computer Simulations at Trial', 75 Journal of the American Bar Association, 52-57.

Mervis, J (1999), 'Court Views Engineers as Scientists', 284(5411) Science, 21.

Montello, DR, Hegarty, M, Richardson, A and Waller D (2004), 'Spatial Memory of Real Environments, Virtual Environments and Maps' in Allen, G (ed.) Human Spatial Memory: Remembering Where, (US:Lawrence Earlbaum Associates), 251-286.

Noond, J and Schofield, D (2002), 'Visualising the Scene--Interactive Evidence: Litigation Graphics and Virtual Reality', Proceedings of Expert Evidence: Causation, Proof and Presentation Conference, Prato, Italy, 2nd-5th July 2002.

Noond, J, Schofield, D, March, J and Evison, M (2002), 'Visualising the Scene: Computer Graphics and Evidence Presentation', 42(2) Science and Justice, Journal of The UK Forensic Science Society, 89-96.

O'Donnel, C, Rotman, A, Collett, S and Woodford, N (2007), 'Current Status of Routine Post-Mortem CT in Melbourne Australia', 3(3) Forensic Science: Medicine and Pathology, 226-232.

O'Flaherty, D (1996), 'Computer-Generated Displays in the Courtroom: For Better or Worse?', 2(4) Web Journal of Current Legal Issues.

Olsen, ECB (1995), 'The Evolution of a Virtual Reality Law Enforcement Driving Simulator', Proceedings of the Third Annual International Conference: Virtual Reality and Persons with Disabilities, California State University, California, August 1995.

Powell, J (2008), Great Hatred, Little Room--Making Peace in Northern Ireland, (UK: Boodley Head).

Rubin, CB (1993), 'A Paperless Trial', 19(3) Litigation Magazine.

Schafer, B and Keppens, J (2007), 'Legal LEGO: Model Based Computer Assisted Teaching in Evidence Courses', 2007(1) Journal of Information Law and Technology (JILT).

Schofield, D, Noond, J, Goodwin, L and Fowle, K (2001), 'Accident Scenarios: Using Computer Generated Forensic Animations', 17(2) Journal of Occupational Health and Safety (Aus NZ), 163173.

Schofield, D (2006) The Future of Evidence: New Applications of Digital Technologies, Forensic Science: Classroom to Courtroom, Proceedings of 18th International Symposium of the Forensic Sciences, Fremantle, Western Australia, 2nd-7th April 2006.

Schofield, D (2007), 'Animating and Interacting with Graphical Evidence: Bringing Courtrooms to Life with Virtual Reconstructions', Proceedings of IEEE Conference on Computer Graphics, Imaging and Visualisation, Bangkok, Thailand, August 14th-16th 2007, 321-328.

Schofield, D and Goodwin L (2007), 'Using Graphical Technology to Present Evidence' in Mason S (ed.) Electronic Evidence: Disclosure, Discovery and Admissibility (London: LexisNexis), 101121.

Schofield, D, (2008), 'Do You Learn More When Your Life is in Danger?', Keynote Address, Proceedings of the 12th International Conference on Mechatronic Technology, Ontario, Canada, 14th--17th November 2008.

Schnabel, MA and Kvan, T (2003), 'Spatial Understanding in Immersive Environments', 1(4) International Journal of Architectural Computing, 435-448.

Selbak, J (1994) 'Digital Litigation: The Prejudicial Effects of Computer-Generated Animation in the Courtroom', 9 High Technology Law Journal 337.

Seltzer, RF (1990), 'Evidence and Exhibits at Trial', 387 PLI/Lit 371.

Seymour, GO, Stahl, JM, Levine, SL and Ingram, JL (1994), 'Modifying Law Enforcement Simulators', 26(2) Journal of Behaviour Research Methods, Instruments and Computers, 266-268.

Sherwin, RK (2002), When Law Goes Pop: The Vanishing Line between Law and Popular Culture (2 Ed.), (Chicago: University of Chicago Press).

Sherwin, RK (2007), 'Law in the Age of Images' in Elkins, J (ed.) Visual Literacy in Action, (Routledge) 179-194.

Spiesel CO, Sherwin RK, Feigenson N (2005) 'Law in the Age of Images: The Challenges of Visual Literacy', In: Wagner A, Summerfield T, Benavides F (eds.) 13 Contemporary Issues of the Semiotics of Law, Onati International Series in Law and Society, 231-255.

Spiesel, C. (2006), 'Reflections on Reading: Words and Pictures and Law', 2(3) International Journal of Law in Context, 305-320.

Tendler, S (2000), 'Inquest to See Crash Animations', The UK Times, 21st September 2001.

Thomas, RD (1995), 'Computer Reenactment: A New Form of Investigation Using Virtual Reality Technology', National Association of Investigation Specialists Newsletter, December 1995.

Tromp, J and Schofield, D (2004), 'Practical Experiences Of Building Virtual Reality Systems', Proceedings of Designing and Evaluating Virtual Reality Systems Symposium, Nottingham, UK, 22nd-23rd January 2004.

Tufte, ER (1985), 'The Visual Display of Quantitative Information', 53(11) American Journal of Physics, 1117-1118.

Walter, B, Alppay, G, Lafortune, E, Fernandez, S and Greenberg, DP (1997), 'Fitting Virtual Lights for Non-Diffuse Walkthroughs', Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, (New York: ACM Press/Addison Wesley Publishing), 45-48.

Ware, C and Osborne, S (1990), 'Exploration and Virtual Camera Control in Virtual Three Dimensional Environments', Proceedings of the 1990 Symposium on Interactive 3D Graphics, SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques, Utah, USA, 3rd-5th October 1990, 175-183.

Wilson, JR, Eastgate, RM and D'Cruz, M (2002), 'Structured Development of Virtual Environments', in Stanney, KM (ed.) Handbook of Virtual Environments: Design, Implementation and Applications, (New York: Lawrence Erlbaum Associates), 353-378.

Watt, F (1999), 3D Computer Graphics (3rd Ed.), (New York: Addison Wesley).

West Midlands CPS (2003), 'Murder of a Police Officer: R v Nicolas Waters', Annual Report 2002-2003, Crown Prosecution Service (West Midlands), 9.

Dr. Damian Schofield

Technical Director--Virtual Simulation

MIRARCO, Laurentian University 

[e-mail]

(1) The author acted as an expert witness in all the cases discussed, responsible for the preparation of the virtual reconstructions described in this paper. Most of the work was undertaken through the author's own company in the UK, Aims Solutions Ltd. <www.aims-solutions.co.uk>.

(2) Rendering is the process of generating a digital image from a virtual digital model, by means of computer software. The term may be thought of by analogy with an 'artist's rendering' of a scene.

(3) The two films mentioned demonstrate two distinct animations styles. The first, Shrek, relies on a cartoon-like, abstract approach to present its narrative. The second, Beowulf, relies on a more realistic representational form. A number of researchers have published on an interesting observable fact relating to the realism in animated imagery, where many viewers become 'unnerved' by images of humans which are close to, but not quite real. This phenomenon has become known as the uncanny valley, due to the sharp dip seen in a graph of familiarity vs. reality (MacDorman, 2006). In a forensic graphics context, many evidence presentations currently rely on fairly abstract representations, however as technology develops the possibility of increasingly photorealistic evidence presentations becomes an ever more possible reality.

(4) The two computer game titles listed illustrate two distinct game playing styles. Unreal Tournament 3 belongs to a genre known as the First Person Shooter (FPS), distinguished by a first person perspective (egocentric) that renders the game world from the visual perspective of the player character. Grand Theft Auto: San Andreas is a Third Person Shooter (TPS), this is a genre of video game in which an avatar of the player character is seen at a distance from a number of different possible perspective angles (exocentric). In any forensic reconstruction (as in any computer game), the choice of the viewing perspective can have an enormous impact on the way an image is interpreted by the viewer. Changing the viewing perspective can potentially alter which 'character' in an evidence presentation a viewer identifies with, or aligns themselves with (Bryce and Rutter, 2002).

(5) One of the more widespread uses of this technology is in training simulations. The discussion of the simulation of crime investigations for training purposes is beyond the scope of this paper. For more information see Seymour et al. (1994); Olsen (1995); Hormann (1995); Schafer and Keppens (2007); Schofield (2008).

(6) This project is funded by the Australian Research Council (ARC) and is a collaboration between multiple Australian universities, the Australian Federal Police, the Department of Public Prosecutions and the Australian Institute of Judicial Administration.

(7) It is interesting to note that research has shown that a significant proportion of the general public have problems relating and correlating two-dimensional (e.g. maps and plans) and three-dimensional (e.g. real and virtual) spatial information (Schnabel and Kvan, 2003). What this means in practice in a typical courtroom scenario is that a number of witnesses may find it easier to position and orientate themselves relative to the real world within a three-dimensional virtual environment rather than on a two-dimensional plan of the incident scene.

(8) In May 2005, a multidetector computed tomography (CT) scanner was installed into the mortuary of the Victorian Institute of Forensic Medicine (VIFM) in Melbourne. Since that time most biological material admitted to the institute has been scanned. This is one of the few mortuaries in the world to have access to such technology and provides a real opportunity for accurate visualisations of post-mortem evidence to begin to be introduced into Australian courtrooms (O'Donnell, 2007).

(9) It should be noted that this three-dimensional computer model of the house was built in approximately one day. As software and technology develops, the time to build three-dimension models and program dynamic interaction in these virtual worlds is continually reducing (Wilson et al, 2002).

(10) One of the first recorded applications of such a dual-modal forensic animation was the reconstruction of the Delta 191 plane crash in 1985. In the courtroom the animation was played alongside the audio from the cockpit voice recorder (Marcotte, 1989).

(11) Cal. Super. Case No. 81067

(12) Dunn <http://www.lectlaw.com/files/lit04.htm> also offers an intriguing take on the viewing of this evidence:

'A sixty year-old grandmother sits astride a Honda Interceptor, motoring at eighty miles-per-hour over wild, treacherous terrain, the wind whipping her face and hair, the heat of the noonday sun on her back, feeling the g-forces as she and her Interceptor lurch through hairpin turns and barely miss countless trees and rock formations. Finally, she fails to negotiate one last turn and flies out into a canyon. In mid-air, if she has not fainted, she contemplates whether it was foolhardy to attempt such a ride, and tries to fathom how much mental suffering is experienced by looking down at the canyon floor and confronting death.'

The emotive language in this statement quite obviously supports the hypothesis put forward by a number of academics and lawyers that one of the possible dangers of using computer-generated visual evidence is that they can be 'loaded' with emotive content that may have a prejudicial effect on the viewer (O'Flaherty, 1996; Girvan, 2001; Bailenson, 2006; Fielder, 2003). This process of adding emotive content has been called 'Disneying-up' the evidence (Galves, 2000).

(13) Although a few specialist forensic graphic service providers do exist in various jurisdictions around the world, mainstream animation houses and multimedia development companies tend to undertake a large proportion of the work. These companies may have very little experience with legal practice, evidential procedures, audit trails and the requirements for validation/verification of the evidence created.