Defining accuracy and precision.
As an example of the importance of accuracy and precision, in January of this year, the U.S. Food and Drug Administration (FDA) sent out over 1,000 letters to drug-product owners regarding a "lack of assay reproducibility between original and repeat results," and in a related warning letter stated that because of lack of reproducibility, "the reported concentration results cannot be considered accurate." The laboratories involved have spent many thousands of hours and tens of millions of dollars responding to this situation. These sorts of events grab headlines and create excitement, but most of us prefer a less thrilling work experience.(1)
What are accuracy and precision?
No measurement is perfect, and all measurements have some error associated with them. In the laboratory, two terms often used somewhat interchangeably to describe measurement errors are accuracy and precision. While general usage may be lax at times, in statistical calculations and in regulations, these two terms have different and very distinct meanings. Knowing the "precise" definitions and importance of each are critical to the development of a sound laboratory quality-control program.
Accuracy refers to the deviation of a measurement from a standard or true value of the quantity being measured. We can talk about the accuracy of a single measurement. For example, if a pipette is set to dispense 100 microliters but actually delivers 99 microliters, the accuracy of that particular dispense is off (or the pipette is inaccurate) by -1 microliter. Notice that in this case we know what happened during that last dispense (it was 1 microliter too low), but we do not have much knowledge about what is likely to happen the next time this pipette is used.
We can also talk about the accuracy of a group of repeated (replicate) measurements. In these cases, we must first determine the mean of the group, and then compare that average value with the standard or true value. Accuracy for a group of measurements refers to the deviation of the group's mean value from the standard or true value. But knowing accuracy alone is of limited use. For example, if we know that three replicate measurements averaged exactly 100 microliters, we still cannot predict how likely it is that the next dispense will be within some limits. One pipette might deliver 99, 100, and 101 microliters (pretty good), while a second delivers 80, 100, and 120 (pretty bad). The averages of both sets of data are exactly 100 and both are perfectly accurate, but which one would you use if your life depended on the next volume measurement being very close to 100?
Precision tells us how close a group of measurements are to one another. The closer the data replicates, the more likely the results will be similar in the future. For this reason, good precision has predictive value, it gives us confidence in future results. Precision is usually calculated and discussed in terms of standard deviation (SD) and coefficient of variation (CV). A precise or closely clustered data set has a smaller CV and is generally more reliable than one that is widely scattered.
Because precision is concerned with the closeness of two or more measurements to each other rather than to a standard value, it is possible for a group of values to be precise without being accurate, or to be accurate without being precise (see Figure 1).
Tech A Tech B Tech C Tech D 5.3 5.3 5.0 5.0 [micro]L [micro]L [micro]L [micro]L 4.7 5.35 5.05 5.3 [micro]L [micro]L [micro]L [micro]L 5.0 5.25 4.95 5.5 [micro]L [micro]L [micro]L [micro]L Tech A's Tech B's Tech C's Tech D's values are values are values are values are accurate accurate accurate accurate but but but but imprecise imprecise imprecise imprecise 4.5 4.5 4.5 4.5 5.0 5.0 5.0 5.0 5.5 5.5 5.5 5.5
When should accuracy and precision be investigated?
A laboratory should investigate the accuracy and precision of a method when the method is new, the method is questioned because of external quality-control (QC) data, or the validity of the results is questionable. It is also a common practice to check the accuracy and precision of laboratory equipment whenever a new instrument is brought into the lab, when equipment is suspected of being damaged, and on a periodic schedule thereafter. In addition, accuracy and precision checks are frequently used as part of the qualification process of a new laboratory technician. Table 1 shows some examples of errors that influence accuracy and precision.
How should accuracy and precision requirements be determined?
To determine the importance of accuracy and precision of liquid handling and other equipment in a method, you will have to take a close look at the intended use of results, sources of error, and how those errors will affect results. A three-step analysis is involved:
1. Establish the limits of acceptable error in final results;
2. Determine each predominant source of error in the method; and
3. Do a statistical analysis of the impact of these errors on your results.
This analysis will help you to focus efforts on controlling and reducing the largest sources of error in your results.
Why is it necessary to test for both accuracy and precision in liquid delivery?
Both accuracy and precision are necessary in order to ensure that results are valid. In general, failure to achieve either accuracy requirements or precision requirements is sufficient to constitute a failed test or calibration.
What is required when equipment accuracy and/or precision requirements are not met?
The U.S. Food and Drug Administration sets high standards for ensuring that calibrated equipment meets pre-established accuracy and precision limits:
"When accuracy and precision limits are not met, there shall be provisions for remedial action to evaluate whether there was any adverse effect on product quality." 21 CFR 820.72 (b)
Type of error Example Corrective action Personal error Excessive thumb Training and pressure or pipetting competence testing too quickly for accuracy and precision Method error Poor precision from Use a pipette sized using a variable volume so the set volume is pipette at the bottom in the top half of of its range its range Instrumental error Delivering inaccurate Clean and check volume because of a pipettes regularly correded pipette piston Table 1. Some errors influencing accuracy and/or precision
Remedial-action requirements include repair and/or recalibration of the equipment and consideration of potential quality impacts in three particular areas. According to FDA Quality System Requirements Manual Part 7: Equipment and Calibration, this includes repair and/or recalibration:
* on the product design or process validation parameters or data;
* on the quality of existing components, in-process, or finished products; and
* appropriate corrective action.
This regulation means that something as simple as a failed pipette could call into question and place at risk any work done with that device since the last prior calibration, and both validation data and products (in-process or finished) can be held suspect. What constitutes appropriate corrective action can vary depending on the situation. An isolated mechanical failure might be corrected with a simple repair, while a pattern of recurring failures in a particular laboratory may require process re-engineering to prevent recurrences or to mitigate the associated risks. Either way, it is desirable to avoid remedial action. Therefore, regular verification of accuracy and precision are essential.
Lack of accuracy or precision is like a roller coaster ride. Results can be up one minute and down the next. Attention to the accuracy and precision of pipettes and other liquid-handling devices improves confidence and quality in laboratory-analytical testing. When tests are correct and reliable, production is smooth and laboratories are less likely to be troubled by regulatory-compliance concerns. A precise and accurate QC laboratory is a pleasant place to work--and a smooth running operation leaves us more time for thrill-seeking outside of work.
1. FDA notifies pharmaceutical companies to confirm or repeat analytic studies used in the approval of a number of drug products. Food and Drug Administration website. Available at: http://www.fda.gov/cder/news/pharmaco_studies/default.htm. Accessed January 2007.
Editor's Note: This article was first published in the February 2007 issue of Pharmaceutical Processing and is reprinted here with permission. It also appeared in Laboratory Equipment.
By George Rodrigues, PhD
George Rodrigues, PhD, is senior scientific manager at ARTEL, responsible for developing and delivering communications and consulting programs designed to maximize laboratory quality and productivity through science-based management of liquid delivery. He can be contacted at 207-854-0860 or email@example.com.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||CLINICAL ISSUES|
|Publication:||Medical Laboratory Observer|
|Date:||Aug 1, 2007|
|Previous Article:||CE test on ethics: a code for the laboratory, plus ethos and the laboratory leader.|
|Next Article:||Health and safety issues in lab design.|