Evaluation of imprecision for cardiac troponin assays at low-range concentrations.
Consistent data on cardiac troponin assay imprecision are often lacking in the peer-reviewed literature. Many reports state the imprecision only for increased concentrations of cardiac troponins and do not address the low range, which is of greatest importance for clinical decision making. Furthermore, the imprecision frequently mentioned in published studies usually reflects assay performance under ideal, nonroutine laboratory conditions. No study has simultaneously determined the total imprecision for a large number of cardiac troponin assays based on the same samples and the same experimental protocol. The aims of the present study were (a) to construct clinically relevant imprecision profiles for commercially available cardiac troponin assays by assessing samples containing cardiac troponin concentrations covering the range around the assay-dependent MI decision limit; (b) to determine the lowest concentrations giving a 10% CV (total imprecision) in the various assays; and (c) to determine whether the concentration associated with a 10% CV corresponds to the manufacturer's stated 99th percentile reference limit.
Materials and Methods
Eight serum pools (A-H) were prepared in the laboratory of one of the authors (K.T.J.Y.) by selectively pooling fresh sera (leftover) from patients to target different concentrations of cardiac troponin I (cTnI) and T (cTnT) spanning the 99th percentile reference limits to ROC curve-based MI cutoffs. The Bayer ACS:180 was used to preset pool concentrations [range, 0.10-3.15 [micro]g/L (12)]. Institutional Review Board approval was not necessary because samples were stripped of patient identifiers, and no medical records or patient-specific information were obtained. Before aliquoting and freezing, each pool was filtered and centrifuged to carefully remove particulate debris and fibrin strands. Pools were then stored (as 0.5-mL aliquots) at -70 [degrees]C until they were shipped (on dry ice) to participating companies for analysis.
All companies manufacturing platforms/ assays for cTnI and/or cTnT were invited to participate. The imprecision experiments were performed directly by the manufacturers. A total of 13 companies agreed to participate, but 2 of these companies did not return their results: Innotrac Diagnostics did not explain why the data were not sent, and Biosite Diagnostics specified that their system (Triage) was designed only for plasma and whole-blood samples. Eleven manufacturers submitted results; 4 of them used two or three different analytical systems (Table 1).
On receipt, the eight pools were stored at -70 [degrees]C until measurement, which occurred within 30 days. Vials of each aliquot were thawed, allowed to equilibrate to room temperature, and centrifuged to eliminate possible fibrin strands. Manufacturers were directed to analyze two replicates of each specimen in each run and to perform one run per day for 20 working days with their respective analytical systems. Two reagent lots (10 working days per lot) and at least three calibrations were to be used. Participants were instructed to randomly analyze pools to reflect any carryover effect. The manufacturer's recommended controls, assayed at the beginning and at the end of each run, were used to validate measurements.
Measurements were reported on a special electronic form prepared by the IFCC Committee and sent to the Committee's chairman (M.P.). The original instrument printouts with raw data were also submitted. The total imprecision of troponin assays at different concentrations was estimated using the ANOVA method described in NCCLS EP5-A guideline (13). The CV values reported for the pools were used to construct imprecision profiles for each method. The troponin concentration associated with a 10% CV was determined from the intercept of the total CV (y axis) equal to 10% on the imprecision profile curve.
The mean cardiac troponin concentrations for pools A-H, with the exclusion of pool F, are shown in Table 2. Results for this pool were excluded from all analyses for technical reasons; i.e., out-of-proportion imprecision data and atypical behavior in several assays. Marked differences were detected between the results obtained with the cTnI and cTnT assays, as well as among different assays measuring cTnI. The analytical response for each pool varied >20-fold for the 15 participating cTnI assays, underscoring the need for standardization. The data also indicated that the different methods had different sensitivities for measurement of cTnI in the pools with the lowest concentrations of this biomarker. Seven methods were unable to measure cTnI in pool A, and two methods were unable to measure cTnI in pool B. An extreme case was represented by the Alpha Dx method, which was unable to measure cTnI in all but one tested pool.
The imprecision profiles (total CV vs pools A-H, which had increasing cTnI concentrations) obtained for each cTnI method are shown in Fig. 1. Because it was impossible to directly compare cTnT with cTnI concentrations in the evaluated pools, the CV data for the two Roche platforms are reported in Fig. 2. Differences in imprecision were clear, which allowed us to categorize assays according to performance as good, moderate, and less than ideal.
The 10% CV concentrations together with the corresponding upper reference limits for each evaluated assay are reported in Table 3. There were no cardiac troponin assays that can achieve an imprecision (CV) of 10% at the 99th percentile reference limit. Six systems achieved a 10% CV at concentrations that were approximately twofold higher than the concentration corresponding to the 99th percentile of the value distribution in a reference population. A second group of nine platforms achieved a 10% CV at concentrations approximately fourfold higher than the 99th percentile cutoff. In two cases, the concentration was very high (ninefold higher than the 99th percentile cutoff; Opus) or even impossible to calculate (Alpha Dx).
Despite the overt clinical advantages, important analytical and preanalytical obstacles to cardiac troponin analysis and interpretation remain, such as assay standardization, antibody specificity, interferences, preanalytical factors, and assay imprecision (9,14). The 99th percentile reference limit for MI diagnosis proposed by the European Society of Cardiology /American College of Cardiology Committee for the Redefinition of Myocardial Infarction was largely driven by the demonstration that any amount of detectable cardiac troponin release is associated with an increased risk of new adverse cardiac events (3). Currently available clinical data demonstrate no threshold below which increased cardiac troponin concentrations do not pose an increased risk and do not have negative implications for prognosis (15-17). However, to avoid diagnostic misclassification arising from assay imprecision in clinical practice, the decision limit should be measured with a total imprecision (CV) [greater than or equal to]10% (4,9,10). A recent study showed that only use of cardiac troponin concentrations that meet the goal of 10% CV as an MI cutoff provide a misclassification rate <1% in a population of patients with suspected acute coronary syndrome (18).
[FIGURE 1 OMITTED]
With regard to this point, many cardiac troponin assays have not been appropriately validated, and the absolute performance of some commercially available assays is not easily discerned (19). To date, clinical users have frequently had to rely on the manufacturers' claims and package inserts, even if data provided by manufacturers often portray better precision than is achieved in clinical practice (9). The current study is unique in that it demonstrates the imprecision at the low concentration ranges of the commercially available assays for cTnI and cTnT based on the use of the same experimental protocol performed over a clinically relevant timespan (4 weeks), use of the same human sera analyzed randomly with more than one lot of reagents and calibrations, and direct comparisons among assays independent of the commercial information provided with each assay. Pools of human sera were used. Because the imprecision for cardiac troponin measurements in modified protein matrices may be significantly different from that for measurements in human serum pools, a clinically realistic precision estimate is obtained only when unmodified human sera are tested.
[FIGURE 2 OMITTED]
Our data confirm the large diversity among cardiac troponin assays with respect to total imprecision and underscore the need for introduction of improved cardiac troponin assays that comply with the redefined MI definition and imprecision requirements. There currently are no commercial assays that can achieve a 10% CV at the 99th percentile reference limit, which would allow accurate differentiation between "minor" myocardial injury and analytical noise. The demand for very precise cardiac troponin assays undoubtedly presents a difficult challenge, but results obtained with more recently released next-generation assays show that there has been substantial improvement in the precision and sensitivity offered by the newer assays. This type of low-end improvement is considered by manufacturers as the main goal in the design of new assays (14).
According to previous suggestions (9,20,21), in the context of clinical practice a predetermined cardiac troponin concentration that meets the imprecision goal should be used as the cutoff for MI until the assays are improved. The results from the present study may help laboratories by identifying the lowest cardiac troponin concentrations that give the 10% CV required for implementation of the recommended clinical strategies in acute coronary syndrome patients. The use of the actual 10% CV concentration instead of the 99th percentile reference limit as a decision cutoff could slightly decrease the clinical sensitivity of the biochemical criterion used for the MI diagnosis, but should permit physicians to reduce the occasional increase in serum cardiac troponin in the absence of myocardial damage. Although higher precision at lower cardiac troponin concentrations does not automatically equate with higher clinical sensitivity, it has been shown that the use of a high-sensitivity cardiac troponin assay would allow identification of a substantial and additional proportion of patients with MI compared with a less sensitive cardiac troponin assay (22,23). Use of the new diagnostic criteria for MI could lead to a mean increase in the number of infarcts in patients with acute coronary syndrome from 20% to 30%, but the percentage of patients recategorized from angina to MI may also be critically dependent on the performance of the cardiac troponin assay used.
Decision limits other than the 99th percentile and 10% CV have been clinically defined for some of the evaluated methods and used for risk stratification of patients with acute coronary syndrome (15-17). Although the data from these clinical trials are compelling, the use of cardiac troponin for MI diagnosis is different from its use for risk stratification. As discussed previously (24), it is important to consider differences in the prevalence of acute coronary syndrome in different populations. If the purpose of measuring cardiac troponin is only to risk-stratify patients with acute coronary syndrome for adverse events, consideration should be given to lowering the cardiac troponin cutoff below the 10% CV value. However, these low cardiac troponin cutoffs are not likely appropriate for the diagnosis of MI in a cohort of patients with chest pain and a low prevalence of disease, where false-positive results produced by a cardiac troponin assay as a result of analytical imprecision could have a much larger negative impact (18).
The limitations in our study should be noted. One limitation is that because each method was assessed in only one (manufacturer) site, it was not possible to definitively determine the effect, if any, on the imprecision of an assay by type of clinical laboratory (emergency or centralized) or by type of healthcare worker (laboratory technician or ward nurse) performing measurements. Furthermore, although human serum pools are better than artificial protein matrices, pooling may mask problems related to the performance for individual samples because interfering compounds in individual sera, e.g., heterophilic antibodies, are diluted and their effects may escape attention. Another limitation is that it could be difficult to accurately estimate the 10% CV concentration based on an interpolation between relatively few points. However, there was no alternative way to derive this information from our findings. Furthermore, looking at the peer-reviewed literature, imprecision studies on individual assays using robust experimental approaches have obtained results very similar to ours (18,25). Finally, use of the 99th percentile limits information obtained from the manufacturers for calculation of the ratio of the 10% CV and 99th percentile concentrations, and the ranking of tests could influence the results, if aspects such as statistical data analysis, sample sizes, or criteria for individual selection were not correctly considered in the production of data by each company. However, for fairness to all manufacturers, we decided to use only what their claim.
We thank the following people and companies, who participated in this study: Robert Hruska (Abbott Laboratories, Abbott Park, IL), E. Glenn Armstrong (Bayer Diagnostics, Tarrytown, NY), Dennis Smith (Beckman Coulter, Inc., Chaska, MN), Patrick Gradon (BioMerieux, Marcy l'Etoile, France); Michael Oed (Byk-Sangtec Diagnostica, Dietzenbach, Germany); Roger Bauer (Dade Behring, Newark, DE), Kathy Maugh (Diagnostic Products Corp., Los Angeles, CA), Judith Davis (First Medical, Inc., Mountain View, CA), Sheryl Sullivan (Ortho Clinical Diagnostics, Rochester, NY), Klaus Hallermayer (Roche Diagnostics GmbH, Penzberg, Germany), and Masuo Inoue (Tosoh Corp., Tokyo, Japan). We also acknowledge the technical assistance provided by Julie Martel (Bayer Diagnostics), Gerard Baudino (BioMerieux), Todd Jensen (Diagnostic Products Corp.), and Koji Igarashi (Tosoh Corp.). Authors of this study have been consultants or research grantees of companies that produce cardiac troponin assays.
Received September 1, 2003; accepted November 7, 2003.
Previously published online at DOI: 10.1373/clinchem.2003.026815
(1.) Wu AHB, Apple FS, Gibler WB, Jesse RL, Warchaw MM, Valdes R. National Academy of Clinical Biochemistry Standards of Laboratory Practice: recommendations for the use of cardiac markers in coronary artery diseases. Clin Chem 1999;45:1104-21.
(2.) Panteghini M, Apple FS, Christenson RH, Dati F, Mair J, Wu AH. Use of biochemical markers in acute coronary syndromes. Clin Chem Lab Med 1999;37:687-93.
(3.) Alpert J, Thygesen K, for the Joint European Society of Cardiology/ American College of Cardiology Committee. Myocardial infarction redefined-a consensus document of the Joint European Society of Cardiology/American College of Cardiology Committee for the Redefinition of Myocardial Infarction. J Am Coll Cardiol 2000;36: 959-69.
(4.) Jaffe AS, Ravkilde J, Roberts R, Naslund U, Apple FS, Galvani M, Katus H. It's time for a change to a troponin standard. Circulation 2000;102:1216-20.
(5.) Apple FS, Murakami M, Panteghini M, Christenson RH, Dati F, Mair J, et al., on behalf of the IFCC Committee on Standardization of Markers of Cardiac Damage. International survey on the use of cardiac markers [Technical Brief]. Clin Chem 2001;47:587-8.
(6.) Murakami MM, Chmielewski LR, Sykora RT, Bardales R, Apple FS. International survey of cardiac troponin users: 2001 vs. 1999 [Abstract]. Clin Chem 2002;48(Suppl o):A97.
(7.) Ottani F, Galvani M, Dolci A, Plebani M, Tubaro M, Zaninotto M, et al. Markers of myocardial damage in the diagnosis of acute myocardial infarction: the Italian reality in the year 2000. Ital Heart J 2002;3(9 Suppl):933-42.
(8.) Tsang WM, Owen AM, Collinson P0, Barth JH. Surveys on the use of cardiac markers in the UK. Ann Clin Biochem 2003;40:13842.
(9.) Apple FS, Wu AH, Jaffe AS. European Society of Cardiology and American College of Cardiology guidelines for redefinition of myocardial infarction: how to use existing assays clinically and for clinical trials. Am Heart J 2002;144:981-6.
(10.) Panteghini M, Gerhardt W, Apple FS, Dati F, Ravkilde J, Wu AH. Quality specifications for cardiac troponin assays. Clin Chem Lab Med 2001;39:174-8.
(11.) Panteghini M. The measurement of cardiac markers. Where should we focus? [Review]. Am J Clin Pathol 2002;118:354-61.
(12.) Pagani F, Bonetti G, Stefini F, Cuccia C, Panteghini M. Determination of decision limits for ACS:Systems cardiac troponin I. Clin Chem Lab Med 2000;38:1155-7.
(13.) National Committee for Clinical Laboratory Standards. Evaluation of precision performance of clinical chemistry devices. NCCLS approved guideline EP5-A. Wayne, PA: NCCLS, 1999.
(14.) Panteghini M. Performance of today's cardiac troponin assays and tomorrow's [Editorial]. Clin Chem 2002;48:809-10.
(15.) Lindahl B, Diderholm E, Lagerqvist B, Venge P, Wallentin L. Mechanisms behind the prognostic value of troponin T in unstable coronary artery disease: a FRISC II substudy. J Am Coll Cardiol 2001;38:979-86.
(16.) Venge P, Lagerqvist B, Diderholm E, Lindahl B, Wallentin L. Clinical performance of three cardiac troponin assays in patients with unstable coronary artery disease. Am J Cardiol 2002;89: 1035-41.
(17.) Morrow DA, Cannon CP, Rifai N, Frey MJ, Vicari R, Lakkis N, et al. Ability of minor elevations of troponins I and T to predict benefit from an early invasive strategy in patients with unstable angina and non-ST elevation myocardial infarction: results from a randomized trial. JAMA 2001;286:2405-12.
(18.) Sheehan P, Blennerhassett J, Vasikaran SD. Decision limit for troponin I and assay performance. Ann Clin Biochem 2002;39: 231-6.
(19.) Yeo KTJ, Quinn-Hall KS, Bateman SW, Fisher GA, Wieczorek S, Wu AHB. Functional sensitivity of cardiac troponin assays and its implications for risk stratification of patients with acute coronary syndromes. In: Adams JE III, Apple FS, Jaffe AS, Wu AHB, eds. Markers in cardiology: current and future clinical applications. Armonk, NY: Futura Publishing Company, 2001:23-9.
(20.) Apple FS, Wu AHB. Myocardial infarction redefined: role of cardiac troponin testing [Editorial]. Clin Chem 2001;47:377-9.
(21.) Galvani M, Panteghini M, Ottani F, Cappelletti P, Chiarella F, Chiariello M, et al. The new definition of myocardial infarction: analysis of the ESC/ACC consensus document and reflections on its applicability to the Italian Health System. Ital Heart J 2002;3: 543-57.
(22.) Ferguson JL, Beckett GJ, Stoddart M, Walker SW, Fox KAA. Myocardial infarction redefined: the new ACC/ESC definition, based on cardiac troponin, increases the apparent incidence of infarction. Heart 2002;88:343-7.
(23.) Agewall S. Evaluation of point-of-care test systems using the new definition of myocardial infarction. Clin Biochem 2003;36:27-30.
(24.) Tate JR, Badrick T, Koumantakis G, Potter JM, Hickman PE, Wu AHB, et al. Reporting of cardiac troponin concentrations [Letter]. Clin Chem 2002;48:2077-82.
(25.) Uettwiller-Geiger D, Wu AHB, Apple FS, Jevans AW, Venge P, Olson MD, et al. Multicenter evaluation of an automated assay for troponin I. Clin Chem 2002;48:869-76.
(9) Nonstandard abbreviations: MI, myocardial infarction; and cTnI and cTnT, cardiac troponin I and T, respectively.
MAURO PANTEGHINI,  * FRANCA PAGANI, KIANG-TECK J. YEO,  FRED S. APPLE ,  ROBERT H. CHRISTENSON,  FRANCESCO DATI  JOHANNES MAIR,  JAN RAVKILDE,  and ALAN H.B. WU,  on behalf of the COMMITTEE ON STANDARDIZATION OF MARKERS OF CARDIAC DAMAGE OF THE IFCC
 Laboratorio Analisi Chimico Cliniche 1, Azienda Ospedaliera 'Spedali Civili, 25125 Brescia, Italy.
 Department of Pathology, Dartmouth-Hitchcock Medical Center and Dartmouth Medical School, Lebanon, NH 03756.
 Department of Laboratory Medicine and Pathology, Clinical Laboratories, Hennepin County Medical Center and the University of Minnesota School of Medicine, Minneapolis, MN 55415.
 Department of Pathology, University of Maryland Medical School of Medicine, Baltimore, MD 21201.
 IVD Consulting, D-35041 Marburg, Germany.
 Department of Internal Medicine, Division of Cardiology, University of Innsbruck School of Medicine, 6020 Innsbruck, Austria.
 Department of Cardiology, Aalborg University Hospital, DK-9100 Aalborg, Denmark.
 Clinical Chemistry Laboratory, Hartford Hospital, Hartford, CT 06102.
* Author for correspondence. Fax 39-030-3995369; e-mail email@example.com.
Presented in part at the 55th AACC National Meeting, July 20-24, 2003, Philadelphia, PA (Clip Chem 2003;49:A34-5).
Table 1. Participating cardiac troponin assay manufacturers and systems. Manufacturer Platform Abbott Diagnostics, Inc. AxSYM Bayer Diagnostics ACS:180 Centaur Immuno 1 Beckman Coulter, Inc. Access, second generation Access 2, second generation BioMerieux Vidas Byk-Sangtec Diagnostica Liaison Dade Behring, Inc. Dimension RxL, second generation Opus, second generation Stratus CS Diagnostic Products Corporation Immulite One First Medical Inc. Alpha Dx Ortho Clinical Diagnostics Vitros ECi Roche Diagnostics (a) E170 Elecsys 1010, third generation Tosoh Corp. AIA 21, (b) second generation (a) The Roche assays are the only cTnT assays on the market; all other assays are for cTnI. (b) AIA Nex IA on US market. Table 2. Mean cardiac troponin concentrations ([micro]g/L) obtained from the evaluated pools by different measurement systems. (a) Bayer Abbott Pool AxSYM ACS:180 Centaur Immuno 1 A <0.3 (c) 0.085 0.182 <0.05 (c) B 0.785 0.151 0.301 <0.05 (c) C 0.948 0.198 0.368 0.107 D 1.708 0.383 0.608 0.242 E 2.088 0.462 0.713 0.370 G 3.720 1.164 1.558 0.691 H 8.080 2.673 3.512 1.299 Beckman BioMerieux Byk-Sangtec Pool Access Access 2 Vidas Liaison A 0.052 0.051 <0.1 (c) 0.019 B 0.164 0.163 0.140 0.036 C 0.166 0.165 0.204 0.052 D 0.293 0.295 0.521 0.076 E 0.422 0.418 0.742 0.097 G 0.761 0.747 0.956 0.145 H 1.793 1.709 1.862 0.360 Dade Behring Dimension Stratus Pool RxL CS Opus A <0.04 (c) 0.047 <0.1 (c) B 0.131 0.212 0.309 C 0.149 0.210 0.313 D 0.293 0.393 0.538 E 0.457 0.614 0.706 G 0.788 0.956 1.160 H 2.328 2.715 2.784 First DPC (b) Medical Ortho Pool Immulite Alpha Dx ECi A 0.126 <0.09 (c) <0.02 (c) B 0.319 <0.09 (c) 0.137 C 0.454 <0.09 (c) 0.197 D 0.703 <0.09 (c) 0.359 E 0.719 <0.09 (c) 0.438 G 1.656 <0.09 (c) 0.982 H 2.278 0.399 2.041 Roche Tosoh Pool Elecsys E170 AIA 21 A 0.028 0.030 0.100 B 0.026 0.031 0.219 C 0.023 0.028 0.277 D 0.058 0.064 0.526 E 0.052 0.061 0.675 G 0.134 0.147 1.168 H 0.256 0.290 2.131 (a) All data are cTnI values except Roche (cTnT). (b) DPC, Diagnostic Products Corporation. (c) Result lower than the detection limit of the assay. Table 3. Concentrations corresponding to 10% CV imprecision and 99th percentile reference limit for the evaluated troponin assays. Ratio of 10% 99th percentile 10% total CV CV concentration limit, (a) concentration, to 99th Platform [micro]g/L [micro]g/L percentile limit AxSYM 0.30 1.22 4.1 ACS:180 0.10 0.37 3.7 Centaur 0.10 0.33 3.3 Immuno 1 0.10 0.34 3.4 Access 0.04 0.06 1.5 Access 2 0.04 0.09 2.3 Vidas 0.10 0.36 3.6 Liaison 0.03 0.065 2.2 Dimension RxL 0.07 0.26 3.7 Opus 0.10 0.90 9.0 Stratus CS 0.07 0.10 1.4 Immulite One 0.20 0.32 1.6 Alpha Dx 0.15 ND (b) Vitros ECi 0.10 0.44 4.4 E170 0.01 0.04 4.0 Elecsys 1010 0.01 0.04 4.0 AIA 21 0.06 0.09 1.5 (a) Data obtained from manufacturer's package insert or through personal communications with manufacturers. (b) ND, not determined.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Proteomics and Protein Markers|
|Author:||Panteghini, Mauro; Pagani, Franca; Yeo, Kiang-Teck J.; Apple, Fred S.; Christenson, Robert H.; Dati,|
|Date:||Feb 1, 2004|
|Previous Article:||Effect of a controlled feedback intervention on laboratory test ordering by community physicians.|
|Next Article:||Rates of positive cardiac troponin I and creatine kinase MB mass among patients hospitalized for suspected acute coronary syndromes.|