How to challenge a black box report: automakers often point to data from a car's black box to 'prove' that a defect was not to blame in an accident. But this data can be highly unreliable - and is always worth close scrutiny.

If you handle automobile defect cases, sooner or later a defense attorney will wave around a page filled with rows of numbers from a black box download. The lawyer will tell you, the judge, and the jury that the numbers prove you have no case. Before that day comes, you can learn how to counter the attack, using the facts about this often misunderstood technology.

"Black box" is a popular term for an electronic device that monitors and stores information about events that take place immediately before, during, and after a collision. It resides in the vehicle's Electronic Control Unit (ECU), which controls the air bags. (1) Black boxes are formally referred to as "crash data recorders" (CDRs) or "event data recorders" (EDRs).

Defendants may say that their CDR data proves their case. But many factors can affect the validity of a black box download and call into question the accuracy of the data.

There are four basic CDR facts to bear in mind. First, not every vehicle has a CDR. Second, the devices differ wildly in the amount and quality of data they record. Third, there is no uniform downloading system: Most car manufacturers use proprietary systems, making it exceedingly difficult to download their CDR data.

Finally--and perhaps most important--no CDR issues a report. A CDR download produces a series of hexadecimal numbers representing values for various parameters. These numbers must be deciphered, either by the download software or by hand, by an analyst or technician, depending on the system.

We can thank the Bush administration's version of the National Highway Traffic Safety Administration (NHTSA) for this state of affairs. NHTSA bowed to the desires of the industry it regulates and declined to issue a regulation requiring standardization of the data recorded or the method of downloading and interpreting it.

The closest thing to a standardized system is the Vetronix Crash Data Retrieval system. Vetronix collaborated with General Motors (GM) in producing it. Since 2000, the instrumentation and software needed to read some of the data has been available to the general public for purchase. Vetronix equipment can read CDR data from most GM vehicles--and some built by affiliated companies. Since 2001, Ford has been making more of its CDRs downloadable with Vetronix gear. (2)

But aside from GM and Ford, most automakers treat CDR data as if it were a matter of national security. They have not licensed Vetronix or any other company to produce software that can work with their CDRs. To get data from a Toyota or a Honda CDR, for example, you must have the company's cooperation and participation in the download. Several manufacturers impose absurd conditions on the process.

These companies' motivations are suspect. They routinely claim that the download process is a "trade secret" and that they need to protect it from competitors. But it is unlikely that anyone would be interested in copying their systems, which are rudimentary at best.

It is far more likely that these companies want to control the download results as part of their risk-management strategy. Remember this whenever you need to perform a non-Vetronix download, and take the most extensive precautions possible to protect the data's integrity.

This is not idle speculation. One company used a single technician to perform all of its downloads. The technician invariably produced data that favored the auto company's position. Because the reports were at odds with known facts, suspicions were raised that the company had routinely substituted prerecorded data sets for the real data. Scrutiny of several downloads revealed one with a date that preceded the actual download date and two others with checksum errors. According to Geoffrey Mahon, a well-known expert on air bag sensors, the chances of a component-failure-induced checksum error is "less than 1 in 2 million hours of operation." (3)

Hits and misses

How much useful data does a CDR record? Here, too, General Motors surpasses its competitors. For example, a 1996 Sunfire CDR recorded velocity change every 10 milliseconds (ms) for a total of 300 ms, starting from the time its air bag sensor algorithm first identified the potential for a collision. It recorded significant information about air bag deployment timing, sensor overlap, which deployment criteria were met, ignition cycles, warning light status, and the driver's seat belt use. It also performed numerous diagnostic functions and recorded fault codes identifying defects in the system.

The capacity of GM's recorders has increased almost yearly. The CDR in a 2005 Envoy records engine speed, brake use, throttle position, passenger seat belt use, driver and passenger seat track position, automatic suppression status, rollover detection, first- and second-stage air bag deployment orders, driver- and passenger-side curtain deployment orders, front- and back-seat-occupant pretensioner deployment orders, fault codes, whether any doors were ajar before the accident, and the principal direction of force.

Other vehicles record much less information. For example, a download from a 2004 Ford comes with the following disclaimer:

   This Restraint Control Module (RCM)
   records longitudinal deceleration data for
   the purpose of understanding the input
   data the Restraint Control Module used to
   determine whether or not to deploy restraint
   devices. This module does not
   record vehicle speed, throttle position,
   brake on-off, and other data, which may be
   recorded in some 1999 model year and later
   General Motors modules.

Most car companies have yet to develop CDRs with capabilities that match what GM was doing 10 years ago. Many devices do not record information that might be important to a reconstruction, including the date and time of the incident; headlight status and use; brake light status; gear selection; steering input; lateral and vertical deceleration; use and status of turn signals; and vehicle tilt, rate of roll, and yaw.

And the reliability of the data they do collect is unimpressive. Hard- earned experience has revealed many flaws that cast doubt on the validity of the downloaded data from many CDRs. When confronting such data in court, you have several grounds for challenging it.

Admittedly unusable data. Many manufacturers admit that some CDR data is not fit for use in accident reconstruction. For example, the deceleration data recorded in many Toyota CDRs is raw, unfiltered output from an accelerometer. To be valid, the deceleration output must be filtered, usually by hardware and software, to screen out "noise"--fluctuation in the data caused by vibration in the sensor components. Noise masks and distorts the actual vehicle data.

If the output was sampled at the time of a spike or dip in the data, the resulting deceleration or acceleration figures could be grossly inaccurate. Toyota openly admits this, stating:

   ECU memory was designed for the purpose
   of Toyota's internal research and development.
   Any information which may be contained
   in an ECU's memory regarding G
   wave data [Toyota's term for accelerometer
   output] was not designed to be used as an
   accident reconstruction tool. Additionally,
   ECU memory of diagnostic codes was not
   designed as a means for determining the
   status or condition of the air bag system prior
   to an accident in which the air bags deployed.
   Data obtained during the readout
   of the ECU could be misleading, if used for
   those purposes. Accordingly, it is Toyota's
   policy to refuse requests to perform readouts
   of ECU memory. Similarly, Toyota
   does not rely on such data, nor has it ever relied
   on such data, in analyzing collision
   events and/or air bag deployments which
   are the subject of a claim or lawsuit. (4)

Such data can be used only by the most experienced, qualified experts, and then only when many samples are taken at extremely short intervals. Few accident reconstruction experts are aware of this problem, and fewer still have the ability to correct it.

But just because a company's CDR is unreliable does not mean the company won't use it against a plaintiff. In one case, BMW obligingly offered to help an insurance company defend a claim by providing it with a download. The plaintiff attorney demanded that an order be entered regulating the download. Among other things, it required BMW to produce the documents needed to interpret the hexadecimal data, as well as information about the download's reliability. BMW backtracked on its previous claims regarding the utility of its data and filed an affidavit stating, "There is very limited data available. The data is used to assess the performance of the vehicle and is not meant nor is it suitable for the reconstruction of an accident." (5)

False reporting of seat belt status. Reports claiming that the seat belt was unbuckled are most likely to affect your case. Don't assume these reports are correct. A major concern is that ECU seat belt sensors are not monitored.

The ECU tests other restraint system components--including the accelerometer, deployment wiring, and reserve power--at ignition and every few milliseconds thereafter. But if the belt switch sensor is malfunctioning, the system does not detect or record it. A defective seat belt switch sensor could cause a false reading that neither the system nor the download technician can detect.

Another problem: The definition of "unbuckled" is open to challenge. No actual switch is engaged to send a signal to the CDR. Instead, voltage is monitored across a resistor in the sensor; if it drops below a certain specification, it registers as not buckled. But many things can cause voltage to fluctuate, especially in a crash. And since the buckle sensor is still being monitored near the end of the crash, the recorder is likely to report the switch status at a time when many systems' electrical integrity has been compromised. This can impair the validity of the download data. (6)

Loss of electrical power has other adverse effects on data. Capacitors supply power to the air bag system, but the recording process takes a lot of energy and time and can often drain the capacitor. This can lead to mischief. In known systems, a reading of 00 hexadecimal in the ECU memory is interpreted as "unbuckled." But it could also be due to power loss.

Recorded-speed errors. Many CDR systems use powertrain sensors as their input source for vehicle- or engine-speed data. The CDR algorithm is calibrated for the factory-installed tire and original gear ratio. If different-size tires are used, or if the gear ratio has been changed for instance, people often buy fires that are different from factory size, and people who haul trailers sometimes change gear ratios--the algorithm's speed calculations will be incorrect. Also, a reconstruction based on "indicated speed" data is unreliable, because yaw and lateral decelerations vital to an accurate reconstruction of true speed are not taken into effect when recording "indicated speed"; nor are skidding and brake lockup. (7)

In an independent evaluation of the validity of crash data, testers performed several low-speed rear-end collisions. Results showed that the speed recorded by the CDR varied by 25 percent from the vehicle's true speed. (8)

Missing data. Not all the data in a download appears in the report. In a representative air bag injury case, the report encompassed data from only 34 of 512 data addresses (6 percent of the total data recorded). The missing data was significant to defect issues; specifically, the report did not identify what criteria, if any, the crash met to trigger the air bag deployment.

The automaker's attorney argued that the download proved the decrease in velocity was sufficient to justify deployment, but this was not true. When the unreported data was found and interpreted, it clearly showed that the velocity criteria had not been met. (9)

Incomplete information on fault codes. No download report should be accepted as valid without complete information on which, if any, fault codes were recorded. Most downloads contain unreported data about fault codes--numbers corresponding to specific component failures that have been detected by the ECU's diagnostic function. These codes can be critical to your case.

Consider an example: A client had an accident, but the air bag did not deploy. The defendant informs you that the download shows the air bag system was malfunctioning--but the air bag indicator light had been on for 250 ignition cycles. This means your client ignored the warning light for months, the automaker argues, and it is his fault that the air bag system was not repaired and functioning at the time of the crash.

In such a circumstance, wouldn't you like to know that the download reported a fault code that indicated the warning light was also malfunctioning? You would be at the defendant's mercy if you did not make sure to get this information.

Another example: A client says his air bag deployed while he was driving, causing him to lose control and have a serious collision. The download report says the air bag properly deployed at the time of the collision. But the defendants do not tell you that the hexadecimal data also includes a fault code indicating that the field effects transistor (FET) was malfunctioning. They also neglect to tell you that an FET failure is a known cause of air bag deployments without collisions.

Unreliable acceleration data. Acceleration data has been used in civil and criminal cases forvarious purposes (for instance, to claim that a driver was speeding or accelerating just before impact). Accelerometer data can be invalid for several reasons. In one case, a CDR report showed that a vehicle was braking and decelerating for four seconds before impact, but in the last second, it stopped decelerating even though the brake was still fully engaged. This meant that the accelerometer had failed, which explained why the air bags had not deployed.

Furthermore, floor-mounted accelerometers are susceptible to dramatically erratic operation because of outside factors, like gravel striking the undercarriage. The accelerometer can also be thrown off by vibrations from the ECU's mounting brackets or its own mounting inside the ECU. This vibration can result in inadvertent air bag deployments.

Misleading interpretations of air bag deployment timing data. Most air-bag-induced injuries are the result of late deployments, when the bag deploys after the occupant has moved forward into the deployment zone. Depending on how far he or she has moved, the injury could range from a bruised face or a blinded eye to quadriplegia, brain damage, or a burst heart. Accurate data about deployment timing is vital to understanding how the injury happened.

Some manufacturers say their systems can prove whether a deployment was late. These CDRs record the time elapsed between the closing (triggering) of the arming sensor (10) and the order to deploy the air bag. Defendants may argue that a short period of time between these events means the deployment was timely.

This is not always true, because many arming sensors are poorly designed and tend to "chatter"--that is, open and close numerous times during a collision. (11) If the CDR records only the last time the sensor closed, the time between closure and the order to deploy will seem shorter than it actually was. To be accurate, the system must calculate the time from the arming sensor's first closure, not the last.

Car companies know that calculations based on the last sensor closing are not valid, but this doesn't stop them from using them. Be prepared to rebut this misleading interpretation of the data.

Unnecessary off-vehicle tests. Some manufacturers will claim that they have to take the ECU to the "factory" to download it because the equipment needed to perform the download is not portable. In one case against DaimlerChrysler, the CDR was a TRW product. Both companies informed the court that it was physically impossible to download the CDR at the vehicle. The plaintiff persuaded the court otherwise, and after the remote download was performed, the plaintiff took the TRW technician's deposition. He testified that the remote download required no special equipment and that the necessary equipment could easily be transported from the engineering lab to the site where the car was in storage. (12) Despite this testimony, DaimlerChrysler still says that download equipment is not portable.

Had the companies prevailed and been allowed to transport the vehicle for the download, TRW had planned to perform a "thruster" test, which subjects the sensor to known forces, ostensibly to check its operation. But it would also record a new data set and potentially overwrite and spoliate the accident data.

Tools and tactics

Be aggressive in your efforts to ensure that a download is performed properly, and get whatever documents you'll need to support your challenge.

Regulating the download process. Before you let a download take place, get a court order to ensure that the process is open, transparent, and honest, with the goal of producing data that is usable by all parties. The download order should require that all documents needed to interpret the meaning of every item of data stored in the CDR be provided to you before the download.

Keep in mind that some data is double- and triple-encoded. Knowing that a hexadecimal is translated into a binary number does you no good until you know what the binary translates to be. Knowing that a number translates to "fault code 83" means nothing if you don't know what fault code 83 is. The order should also require

* that any potential for the download process to alter the data be disclosed at least two weeks before the download (to give you time to object)

* that the technician not alter, erase, or add data, and that he or she provide a true and complete copy of the data to you immediately after the download

* that a complete report detailing the interpretation of each item of data be provided to you within 10 days after the download

* that the whole process be photographed and videotaped, including images of computer screens

* that the technician's qualifications be disclosed before the download

* that the technician sign the order to acknowledge his or her receipt of it.

Seek this order even if you're dealing with a Vetronix-compatible CDR. Vetronix systems do not interpret all the data. And, of course, never allow the defendant to take the CDR "to the factory" for the download.

Discovery. Many of the documents you will need to establish doubts about the reliability of CDR data will be subject to a protective order. These include documents required to interpret the data's meaning, those describing the sensor and CDR system and how they work, and those that describe the failure modes of various components. Knowing that a fault code for the accelerometer was in the data is useless without knowing the effect the failure can have on the system. These documents will have different names depending on the company involved, so you need to provide a description of each document you request and list the possible titles. (13)

Daubert challenge. As more CDR systems are subjected to vigorous discovery and critical examination, more faults will be discovered. In some cases, you may be able to persuade the court to declare the defendant's CDR report unreliable and therefore inadmissible under Daubert v. Merrell Dow Pharmaceuticals, Inc. (14)

The defendant's accident reconstructionist may not be able to answer Daubert questions about the systems at issue. He or she may not know what fault codes exist, what they mean, or how they affect the data's reliability. The reconstructionist may not know the processes that affect the validity of the deployment timing data, the method used to determine brake use or belt use, or factors that can affect that data. He or she may not be ready to address accelerometer issues or component deterioration. Identify, if you can, the specific area of the expert's weakness, and do your homework to mount an effective challenge.

Ownership of data. You can challenge a download that took place without the car owner's consent. NHTSA takes the position that the person who owns the vehicle owns the data. (15) Since information is property, this seems logical.

But what happens if the vehicle is transferred after an accident--for example, if the insurance company takes tide? Under California law, the data belongs to the person who owned the vehicle when the data was recorded, and it is illegal to download a CDR without the owner's permission or a court order. Several states have laws regarding data, but most do not. You need to be aware of your local statutes. If you can, consider challenging the data on ownership grounds.

Spoliation trap for plaintiffs

There is a popular misconception about CDRs that needs to be addressed: "Experts" often claim that performing a download will not alter the downloaded data.

This is not true. In a restraint defect case, significant material alteration can occur if the CDR is not downloaded through the vehicle's diagnostic link connector (DLC), which service personnel use to read trouble codes when performing ordinary maintenance and repairs. The CDR can also be read through the DLC. However, many CDRs, including those that are compatible with Vetronix gear, can be read directly, through connectors on the ECU itself.

The only reliable way to download data is through the DLC. Ford includes the following warning in its Vetronix download reports:

   If there is any question that the restraint system
   did not perform as it was designed to
   perform, please read the system only
   through the diagnostic link connector....
   Since the vehicle restraint control module is
   constantly monitoring air bag system readiness
   (when powered), it will detect that the
   sensors and air bags are not connected [during
   the download]. The restraint control
   module may record a new diagnostic trouble
   code in to memory for each device that is
   not connected. These new diagnostic trouble
   codes may record over previously written
   diagnostic trouble codes present prior to the
   accident and spoil evidence necessary to determine
   if the restraint system performed in
   the accident as it was designed to perform.
   Not only could this prevent Ford from being able
   to determine if the system performed as it was
   designed to perform, but, regardless of innocent
   inadvertence, you could raise issues of evidence
   spoliation in any litigation that may arise out
   of the accident. (16)

An adverse "black box" report is not the end of your case. Taking steps early in the litigation to regulate the download and get the necessary documents for interpreting the data will put you in the best position to expose invalid conclusions that the defense tries to assert.

Notes

(1.) Automakers use different names for their ECUs. General Motors designated its device the Sensing and Diagnostic Module (SDM), Ford uses the term Restraints Control Module (RCM), and DaimlerChrysler uses Airbag Electronic Control Unit (AECU).

(2.) The former Vetronix Corp. merged with Bosch and is now known as Vetronix Bosch Group. See www.vetronix.com. A list of Vetronix-compatible vehicles is at www.sbcommerce. vetronix.com/CDR/CDRVehicleList.pdf (last accessed Dec. 15, 2006).

(3.) According to Mahon, this is a conservative figure, and the more realistic number would be 1 failure in 10,000 years. This estimate was calculated according to the U.S. government standard for computing the reliability of electrical components outlined in MIL-HDBK-217. See U.S. Dept. of Defense, Military Handbook: Reliability Prediction of Electronic Equipment (Feb. 28,1995).

(4.) Toyota Corp., Operation Manual for Toyota's SRS Airbag G Data Recorder Readout Tool (Version 2.01).

(5.) Aff. of Mark Yeldham, Morales v. Averitt Express, No. 05-01-10133-DCVAJA (Tex., Dimmit Co. Dist., Jan. 6, 2006) (on file with John B. Scofield Jr., Caddell & Chapman, Houston, Tex.).

(6.) The disclaimer in a Vetronix download report states, "If power to the SDM is lost during a crash event, all or part of the crash record may not be recorded." Vetronix, CDR Crash Data Retrieval System, VIN: IG6KS54Y2YU211, at 1 (Sept. 14, 2004), www-nrd.nhtsa.dot.gov/database/aspx/ getmedia.aspx?tstno=4955&index=4&database= V&type=R (last accessed Dec. 15, 2006).

(7.) See Armin Kast, Kast GmbH Accident Research, Comment Regarding Notice of Proposed Rulemaking on Event Data Recorders (EDR), NHTSA 2004-18029-60 (Aug. 9, 2004), 60 Fed. Reg. 32932 (June 14, 2004), http://dmses.dot.gov/docimages/ pdtB9/291713_web.pdf, at 3 (last accessed Dec. 15, 2006).

(8.) Joe T. Correia et al., Utilizing Data from Automotive Event Data Recorders, Proceedings of the Canadian Multidisciplinary Road Safety Conference XII, at 12 (London, Ontario, June 10-13, 2001), www-nrd.nhtsa.dot.gov/edr-site/uploads/Utillizing_ Data_from_Automotive Event_Data_Recorders. pdf (last accessed Dec. 15, 2006).

(9.) The actual case occurred during a mediation, records of which are confidential.

(10.) An arming sensor or sating sensor operates in conjunction with the primary sensor to deploy the air bag. The sensors must close simultaneously; this reduces inadvertent deployments.

(11.) You can prove this by reviewing the sensor graphs in crash testing produced in discovery.

(12.) Personal communication with Leon R. Russell, Law Offices of Leon R. Russell, P.C., Dallas, Tex. Russell warns that "companies know far more about the airbag systems they manufacture than plaintiff's counsel ever will. Documentation describing air bag systems is usually cryptic and difficult to decipher. It is essential then that plaintiff's counsel retain an expert on black boxes not only to help counsel understand the air bag system specifications but also to test the critical components of the system to assure that they functioned as designed."

(13.) Some common tides include "software definition document," "SDM product definition document," "hexadecimal download tool," and "failure mode and effects analysis" (FMEA). Request an FMEA for the CDR, air bag sensors, air bag inflators, buckle sensors, passenger detection system, pretensioners, and warning indicator light.

(14.) 509 U.S. 579 (1993).

(15.) Nat. Highway Transp. Safety Admin., Final Regulatory Evaluation: Event Data Recorders 1-4 (July 2006), www.nhsta.dot.gov/staticfiles/DOT/ NHTSA/Rulemaking/Rules/Associated%20Files/ EDRFRIA.pdf (last accessed Dec. 15, 2006).

(16.) Vetronix, Crash Data Retrieval System, VIN: 1FMYU70E31UB29688, at 1 (Nov. 16, 2005) (emphasis added), www.bloombergconsulting.com/ Ford.PDF (last accessed Dec. 15, 2006).

STEPHEN E. VAN GAASBECK practices law in San Antonio, Texas.