Printer Friendly

Should I repeat my 1:2s QC rejection?

It is considered good laboratory practice to design QC strategies--number of QC examinations, QC rules, and frequency of QC evaluations--to ensure that patient results meet the quality required for their intended use (1). Many laboratories, however, continue to use a 1:2s QC rule for all analytes without considering the relationship of analytical performance to quality requirements. A 1:2s QC rule rejects results when any QC examination is either 2 SDs higher or lower than the QC target concentration.

One of the undesirable characteristics of the 1:2s QC rule is its high false-rejection rate (4.6% for 1 QC examination, 8.9% for 2 QC examinations, 13.0% for 3 QC examinations). A laboratory that examines 2 QC samples for 20 different analytes and applies a 1:2s QC rule should expect 1 or more analytes to give a QC rule rejection at every QC evaluation. To deal with this unacceptable frequency of false rejections, a common practice is to repeat the QC examination for any failing analytes and again apply the 1:2s rule (2).

Clearly the continued repetition of quality control testing until the desired result is achieved is a misguided practice (3), but what about a single planned repeat evaluation for a failing 1:2s QC rule? The QC performance characteristics of this practice have not been formally investigated. Therefore, we considered a number of alternative 1:2s repeat-sampling strategies and objectively compared their performance characteristics.

Materials and Methods

We considered 4 different repeat-sampling strategies. In all cases, N QC concentration levels (N = 2 or 3) were initially examined. Depending on the initial QC results, 1 or more QC examinations were repeated. The 4 repeat-sampling strategies are described in Table 1. All 4 repeat-sampling strategies begin by comparing the results at each concentration level of control material to within 2 SD limits. The differences between the 4 repeat-sampling strategies are defined by what decisions can be made on the basis of the first sample of QC results and, if a repeat sample is performed, how many samples are repeated. For repeat-sampling strategies 1 and 3, there are only 2 decisions that can be made on the basis of the first sample of QC results: either accept the results or obtain a second sample. For repeat-sampling strategies 2 and 4, there are 3 decisions that can be made on the basis of the first sample QC results: accept, reject, or obtain a second sample. For repeat-sampling strategies 1 and 2, if a second sample is obtained, only the sample concentrations that initially exceeded their limits are repeated. For repeat-sampling strategies 3 and 4, if a second sample is obtained, all sample concentrations are repeated.

For the case N = 2, we compared the 4 repeat-sampling strategies to the 1:2s rule, 1:3s rule, and the 1:3s/2:2s/R:4s multirule. For the case N = 3, we compared the 4 repeat-sampling strategies to the 1:2s rule, 1:3s rule, and the 1:3s/2of3:2s/R:4s multirule. These QC rules, along with many others, have been described in the clinical laboratory literature (4). The 1:2s QC rule rejects if any QC measurement is more than 2 SDs from its target value. The 1:3s QC rule rejects if any QC measurement is more than 3 SDs from its target value. With 2 QC concentration levels, the 1:3s/2:2s/R:4s multirule rejects if any QC measurement is more than 3 SDs from its target value, both QC measurements are more than 2 SDs above target, both QC measurements are more than 2 SDs below target, or the range of the deviation of each QC measurement from its target exceeds 4 SDs. With 3 QC concentration levels, the 1:3s/ 2of3:2s/R:4s multirule rejects if any QC measurement is more than 3 SDs from its target value, any 2 of 3 QC measurements are more than 2 SDs above target, any2 of 3 QC measurements are more than 2 SDs below target, or the range of the deviation of each QC measurement from its target exceeds 4 SDs.

We compared QC rules on the basis of their false-rejection and error-detection rates (power curves) for detecting systematic error (SE) [2] out-of-control conditions. For the repeat-sampling strategies, the total number of QC examinations can vary each time the QC rules are evaluated, so we also computed the in-control average number of QC examinations per QC evaluation.

Probability of error detection ([P.sub.ed]) and average number of QC examinations per QC rule evaluation ([N.sub.Q]) were derived mathematically for the repeat-sampling QC rules and validated by simulating 1 000 000 QC rule evaluations. [P.sub.ed] was derived mathematically for the 1:2s and 1:3s QC rules and was computed by simulating 100 000 000 rule evaluations for the 1:3s/2:2s/R:4s and 1:3s/2of3:2s/R:4s multirules. Performing 100 000 000 simulations ensured that the 95% CIs for the computed probabilities were always <0.0001, matching the number of digits reported in the tables.

Results

The mathematical formulas to compute [P.sub.ed] and [N.sub.Q] for each of the 4 repeat-sampling QC strategies are given in Table 2. More detail regarding the derivations of [P.sub.ed] and [N.sub.Q] is given in the Appendix in the Data Supplement, which accompanies the online version of this article at http://www.clinchem.org/content/vol58/ issue 5. The online Data Supplement also gives plots showing probabilities of error detection on the basis of 1 million simulated QC rule evaluations that confirm the validity of the mathematical derivations for each ofthe 4 repeat-sampling strategies when N = 2 and N = 3.

The probabilities of error detection with 2 concentration levels of control material are given in Table 3. We evaluated SE conditions ranging from 0 to 5 SD and compared the 4 repeat-sampling strategies as well as the 1:2s and 1:3s QC rules and the 1:3s/2:2s/R:4s multirule. The first row of Table 3 (SE = 0) gives the false-rejection rate for each QC rule; the last row of Table 3 gives the average number of QC examinations per QC evaluation for the in-control process.

All of the repeat-sampling strategies slightly increase the overall utilization rate of control material. The increase in utilization rate is determined by the probability that the QC rule will trigger a repeat sample and the number of control samples that are examined if a repeat sample is obtained. If only the control sample concentrations that initially exceeded their limits are repeated (repeat strategies 1 and 2), then the increase in utilization rate is approximately equal to the probability that a QC result will be more than 2 SDs from its target value when the process is in control (about 4.5%). If all control sample concentrations are repeated when a repeat sample is obtained (repeat strategies 3 and 4), then the increase in utilization rate is approximately the number of control levels multiplied by the probability that a QC result will be more than 2 SDs from its target value when the process is in control (2 X 4.5% = 9%).

Fig. 1 displays power function curves for each of the QC rules listed in Table 3. The repeat-sampling strategies are seen to behave more like the 1:3s QC rule for small out-of-control conditions and more like the 1:2s QC rule for large out-of-control conditions. Of the 4 repeat-sampling strategies, strategy 4 has the highest power and strategy 1 has the lowest power. All 4 repeat-sampling strategies have higher power than the 1:3s/2:2s/R:4s multirule.

[FIGURE 1 OMITTED]

Table 4 gives the probability of error detection and the average number of QC examinations per QC evaluation for the 4repeat-sampling strategies as well as the 1:2s and 1:3s QC rules and the 1:3s/2of3:2s/R:4s multirule when 3 concentration levels of control material are examined. Fig. 2 displays power function curves for each of the QC rules listed in Table 4. As in the case with 2 concentration levels of control material, repeat-sampling strategy 4 has the highest power and repeat-sampling strategy 1 has the lowest power among the repeat-sampling strategies. In this case, however, the 1:3s/2of3:2s/R:4s multirule has greater power than repeat-sampling strategy 1 but less power than repeat-sampling strategies 2-4. Again, the repeat-sampling strategies result in a slight increase in control material utilization rate (4%-13%).

Discussion

The tables and figures demonstrate that the repeat-sampling strategies provide an effective tactic that takes advantage of desirable properties of both the 1:2s and 1:3s QC rules. The false-rejection rates of the repeat-sampling strategies are low, like those of the 1:3s QC rule, yet the error detection rates for the repeat-sampling strategies approach the error detection rate of the 1:2s QC rule as the magnitude of SE increases. Repeating all control samples when repeat testing is performed (repeat strategies 3 and 4) provides greater power than only repeating the controls that exceeded their limits in the initial sample (repeat strategies 1 and 2), but at a slight increase in control material utilization. Additionally, allowing an immediate QC rule rejection if more than 1 QC sample result exceeds its limit in the initial sample increases the error detection rate and slightly reduces the control material utilization of the repeat strategy (compare repeat strategy 2 to 1 and repeat strategy 4 to 3).

[FIGURE 2 OMITTED]

We also computed power curves for the repeat-sampling strategies to detect out-of-control conditions that increase the analytic imprecision of the process (random error out-of-control conditions) (see online Data Supplement). For random error out-of-control conditions, only repeat-sampling strategies 3 and 4--which repeat all QC concentration levels when repeat testing is performed--are effective. Repeat sampling strategies 1 and 2--which repeat only the QC concentration level that exceeds its limit in the initial sample--are no more effective than a 1:3s QC rule. Repeat sampling strategy 3 has approximately the same power as the 2 multirules evaluated here. Repeat sampling strategy 4 is still superior to the other repeat-sampling strategies and the multirules. Thus, for detecting out-of-control conditions that increase analytic imprecision, it is important that a repeat-sampling strategy-repeat all the QC concentration levels initially examined.

The 4 repeat-sampling strategies investigated here were chosen because our experience suggested they were similar to the approaches taken by laboratories that will repeat an initial QC rule violation. However, they are only a small subset of the many different repeat-sampling strategies that could be envisioned (5). In general, a repeat-sampling QC strategy can be defined by 4 design parameters: the number of control samples tested in the initial sample, the QC rules applied to the initial QC sample results, the number of control samples tested if a repeat sample is obtained, and the QC rules applied to the repeated QC sample results to decide acceptance or rejection. Nothing requires the number of QC samples tested in the initial and repeat samples be the same, or that the QC rules applied to the initial QC sample results be the same QC rules applied to the repeated sample results. Investigating the probability of error detection and control sample utilization for other possible repeat-sampling strategies might prove valuable.

Contrary to the notion that repeating an initial QC rule violation is a misguided practice, we show that a QC strategy that permits a planned repeat sample to be tested in the event of an initial QC rule violation can improve out-of-control error detection ability with only a slight increase in control material utilization rate. The secret to the repeat-sampling strategy's performance characteristics is that it is more likely to require extra QC examinations when the testing process is truly out of control (improving power), but less likely to require the extra QC examinations when the process is in control (reducing cost). For example, when 2 QC concentration levels are initially examined, there is an 8.7% chance that repeat-sampling strategy 4 will obtain a repeat sample if the process is in control. If a 2 SD systematic error out-of-control condition exists, however, there is a 50% chance that a repeat sample will be obtained. A repeat testing strategy provides a data-driven tactic to decide when to examine additional QC samples.

In summary, laboratories should design their QC strategies on the basis of the performance characteristics of their test methods and the allowable total error requirements for each analyte. A repeat-sampling QC strategy is not the best approach for all circumstances; it simply adds another group of candidate QC rules to the list of possibilities a laboratory should consider. Our work suggests that laboratories that currently use 1:2s, 1:3s, 1:3s/2:2s/R:4s, or 1:3s/2of3:2s/R:4s QC rules might want to consider the merits of adopting a repeat-sampling strategy.

Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revisingthe article for intellectual content; and (c) final approval of the published article.

Authors' Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the Disclosures of Potential Conflict of Interest form. Potential conflicts of interest:

Employment or Leadership: C.A. Parvin, Bio-Rad Laboratories; L. Kuchipudi, Bio-Rad Laboratories; J.C. Yundt-Pacheco, Bio-Rad Laboratories.

Consultant or Advisory Role: None declared.

Stock Ownership: J.C. Yundt-Pacheco, Bio-Rad Laboratories.

Honoraria: None declared.

Research Funding: None declared.

Expert Testimony: None declared.

Role of Sponsor: No sponsor was declared.

References

(1.) International Organization for Standardization (ISO). Medical laboratories: particular requirements for quality and competence. Geneva: ISO; 2007. ISO 15189.

(2.) Housley D, Kearney E, English E, Smith N, Teal T. Audit of internal quality control practice and processes in the south-east of England and suggested regional standards. Ann Clin Biochem 2008;45:135-9.

(3.) Westgard JO. Assuring the right quality right. Madison (WI): Westgard QC; 2007.

(4.) Cembrowski GS, Carey RN. Laboratory quality management: QC & QA. Chicago: ASCP Press; 1989.

(5.) Daudin JJ. Double sampling Xcharts. J Qual Technol 1992;24:78-87.

Curtis A. Parvin, [1] * Lakshmi Kuchipudi, [1] and John C. Yundt-Pacheco [1]

[1] Bio-Rad Laboratories, Quality Systems Division, Plano, TX.

[2] Nonstandard abbreviations: SE, systematic error; [P.sub.ed], probability of error detection; [N.sub.Q], average number of QC examinations per QC rule evaluation.

* Address correspondence to this author at: Bio-Rad Laboratories, Quality Systems Division, Plano, TX, 75074. Fax 972-423-4395; e-mail curtis_parvin@bio-rad.com.

Received December 29, 2011; accepted January 30, 2012.

Previously published online at DOI: 10.1373/clinchem.2011.181818
Table 1. Four repeat-sampling QC strategies.

 Repeat
strategy                        Description

 Rule 1    If all initial QC results are within mean [+ or -] 2s
             control limits, then accept.
           Else, repeat any control observation that exceeded
             its control limit.
           If all repeated QC results are within control limits,
             then accept.
           Else, reject and begin troubleshooting.
 Rule 2    If all initial QC results are within mean [+ or -] 2s
             control limits, then accept.
           If >1 QC results exceed control limits, then reject.
           Else, repeat the control that exceeded its control
             limit.
           If the repeated result is within control limits, then
             accept.
           Else, reject and begin troubleshooting.
 Rule 3    If all initial QC results are within mean [+ or -] 2s
             control limits, then accept.
           Else, repeat all control observations.
           If all repeated QC results are within control limits,
             then accept.
           Else, reject and begin troubleshooting.
 Rule 4    If all initial QC results are within mean [+ or -] 2s
             control limits, then accept.
           If >1 QC results exceed control limits, then reject.
           Else, repeat all control observations.
           If all repeated QC results are within control limits,
             then accept.
           Else, reject and begin troubleshooting.

Table 2. [P.sub.ed] and [N.sub.Q] for 4 repeat-sampling
QC strategies.

 Repeat           [P.sub.ed] and [N.sub.Q] computational
strategy          formulas (a)

 Rule 1           [P.sub.ed](SE) = 1 - [1 - [P.sub.1][(SE).sup.2]]N
                  [N.sub.Q] = N[1 + [P.sub.1](0)]
 Rule 2           [P.sub.ed](SE) = 1 - [[1 - [P.sub.1](SE)].sup.N]
                    [1 + [NP.sub.1](SE)]
                  [N.sub.Q] = N{1 + [[1 - [P.sub.1](0)].sup.N-1]
                    [P.sub.1](0)}
 Rule 3           [P.sub.ed](SE) = [{1 - [[1 - [P.sub.1]
                    (SE)].sup.N]}.sup.2]
                  [N.sub.Q] = N {2 - [[1 - [P.sub.1](0)].sup.N]}
 Rule 4           [P.sub.ed](SE) = 1 - [[1 - [P.sub.1](SE)].sup.N]
                    {1 + [NP.sub.1](SE)[[1 - [P..sub.1]
                    (SE)].sup.N-1]}
                  [N.sub.Q] = N{1 + N[[1 - [P.sub.1] (0)].sup.N-1]
                    [P.sub.1](0)}

(a) [P.sub.1](SE), probability that a single QC examination is
outside its control limit given that a systematic error of magnitude
SE exists; N, number of QC concentration levels.

Table 3. [P.sub.ed] and average number of QC examinations per QC
Evaluation ([N.sub.Q]) for repeat sampling strategies with 2
initial control examinations compared with some standard QC rules.

                    Repeat sampling strategy

 SE (a)     Rule 1     Rule 2     Rule 3    Rule 4    1:2s     1:3s

   0.0      0.0041     0.0060     0.0079    0.0098   0.0889   0.0054
   0.5      0.0106     0.0152     0.0198    0.0244   0.1407   0.0128
   1.0      0.0505     0.0686     0.0867    0.1047   0.2944   0.0450
   1.5      0.1816     0.2271     0.2727    0.3182   0.5222   0.1292
   2.0      0.4375     0.5000     0.5625    0.6250   0.7500   0.2921
   2.5      0.7276     0.7732     0.8187    0.8642   0.9048   0.5219
   3.0      0.9147     0.9325     0.9503    0.9681   0.9748   0.7500
   3.5      0.9833     0.9872     0.9911    0.9950   0.9955   0.9048
   4.0      0.9980     0.9985     0.9990    0.9995   0.9995   0.9748
   4.5      0.9998     0.9999     0.9999    1.0000   1.0000   0.9955
   5.0      1.0000     1.0000     1.0000    1.0000   1.0000   0.9995
[N.sub.Q]    2.09       2.09       2.18      2.17      2        2

 SE (a)     1:3s/2:2s/R:4s

   0.0          0.0097
   0.5          0.0195
   1.0          0.0653
   1.5          0.1884
   2.0          0.4089
   2.5          0.6686
   3.0          0.8665
   3.5          0.9632
   4.0          0.9933
   4.5          0.9992
   5.0          0.9999
[N.sub.Q]         2

(a) Systematic error condition (multiples of analytic SD).

Table 4. [P.sub.ed] and [N.sub.Q] for repeat sampling strategies
with 3 initial control examinations compared with some
standard QC rules.

                 Repeat sampling strategies

 SE (a)     Rule 1   Rule 2   Rule 3   Rule 4    1:2s     1:3s

   0.0      0.0062   0.0117   0.0170   0.0222   0.1304   0.0081
   0.5      0.0159   0.0290   0.0414   0.0535   0.2034   0.0192
   1.0      0.0749   0.1228   0.1659   0.2066   0.4073   0.0668
   1.5      0.2596   0.3638   0.4485   0.5236   0.6697   0.1873
   2.0      0.5782   0.6875   0.7657   0.8282   0.8750   0.4044
   2.5      0.8579   0.9097   0.9421   0.9648   0.9706   0.6694
   3.0      0.9751   0.9859   0.9920   0.9958   0.9960   0.8750
   3.5      0.9978   0.9989   0.9994   0.9997   0.9997   0.9706
   4.0      0.9999   1.0000   1.0000   1.0000   1.0000   0.9960
   4.5      1.0000   1.0000   1.0000   1.0000   1.0000   0.9997
   5.0      1.0000   1.0000   1.0000   1.0000   1.0000   1.0000
[N.sub.Q]    3.14     3.12     3.39     3.37      3        3

 SE (a)     1:3s/2of3:2s/R:4s

   0.0           0.0202
   0.5           0.0376
   1.0           0.1204
   1.5           0.3243
   2.0           0.6193
   2.5           0.8613
   3.0           0.9702
   3.5           0.9965
   4.0           0.9998
   4.5           1.0000
   5.0           1.0000
[N.sub.Q]           3

(a) Systematic error condition (multiples of analytic SD).
COPYRIGHT 2012 American Association for Clinical Chemistry, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2012 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Laboratory Management
Author:Parvin, Curtis A.; Kuchipudi, Lakshmi; Yundt-Pacheco, John C.
Publication:Clinical Chemistry
Article Type:Report
Geographic Code:1USA
Date:May 1, 2012
Words:3392
Previous Article:How safe is the outpatient management of patients with acute chest pain and mildly increased cardiac troponin concentrations?
Next Article:Increased cardiac troponin I as measured by a high-sensitivity assay is associated with high odds of cardiovascular death: The Minnesota heart survey.
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters