Printer Friendly

ROC analysis comparison of three assays for the detection of antibodies against double-stranded DNA in serum for the diagnosis of systemic lupus erythematosus.

Antibodies targeted against double-stranded DNA (dsDNA) were first described in 1957 (1, 2). Detection of anti-ds-DNA antibody (ds-DNA-Ab) is an integral part of the definition of systemic lupus erythematosus (SLE) as proposed by the American College of Rheumatology (ACR) criteria (3). Two of 11 criteria set by the ACR are related to anti-nuclear antibody detection (anti-nuclear antibodies; anti-ds-DNA or anti-sm or anti-phospholipid antibodies). Because the diagnosis of SLE is established when 4 of the 11 criteria are fulfilled, the correct identification of antibodies is extremely important. Because dsDNA-Ab may occur in a wide spectrum of diseases and even in healthy volunteers, diagnostic tests must be highly specific. The Farr RIA is assumed to meet this need best (4, 5). To improve the diagnostic accuracy of dsDNA-Ab tests, new tests have been developed, including ELISAs with different preparations of ds-DNA antigens as well as an automated enzyme fluoroimmunoassay using plasmid DNA as antigen. However, clinical data to assess the diagnostic accuracy of these recently released test systems are limited at present.

To clarify the clinical usefulness of different dsDNA-Ab tests in the diagnosis of SLE, we measured ds-DNA-Ab in 152 Caucasian patients for whom analysis of anti-nuclear antibodies was requested between November 1999 and September 2000 in the Laboratory of Immunology at the University of Bonn. To take into account differences in diagnostic standards, the patient records were reevaluated by the study personnel, who used a unified protocol to verify the presence of diagnostic criteria according to the current revision of the ACR (3). In 12 of 152 patients, a definite decision on the presence of SLE-associated manifestations could not be made because the records were incomplete. These 12 patients were excluded from further analysis. All study procedures were performed in accordance with the current revision of the Helsinki Declaration of 1975, and the study design was based on the 2000 guidelines for evaluation of diagnostic accuracy as suggested by Bruns et al. (6).

For quantitative determination of anti-nuclear antibodies, the following assays were used: Farr RIA (Amersham; cutoff, 7 kIU/L; interassay CV, 3.8-11.8%); EliA[TM] anti-dsDNA IgG test (Pharmacia Diagnostics; cutoff, 10-15 k1U/L; CV, 3.9-4.7%); ds-DNA-ELISA ORG 604 (Orgentec; cutoff, 20 kIU/L; CV, 5.2-12.4%). Serum concentrations of IgG anti-nuclear antibody were measured by an indirect immunofluorescence test (IFT) using Hep-2 cells (The Binding Site). All assays were used according to the manufacturers' instructions following standard laboratory procedures. The laboratory staff were blinded with respect to the clinical presentation of the patients.

To characterize each assay with respect to its diagnostic accuracy, we performed ROC analysis as described elsewhere (7). The widely accepted ACR criteria (3) were used as the external criterion standard for the diagnosis of SLE. Because the presence of ds-DNA-Ab is an integral part of the ACR definition, the analysis was repeated three times with the results of the three evaluated dsDNA-Ab tests taken as the basis for diagnosis. To exclude possible bias attributable to the inclusion of any dsDNA-Ab test in the previous SLE diagnosis, we repeated the ROC analysis a fourth time. In this analysis, the presence of ds-DNA-Ab was excluded from the SLE criteria and instead three ACR criteria were regarded as sufficient to establish the diagnosis of SLE.

Serum samples from 140 well-characterized patients with clinical suspicion of SLE were entered into the analysis [43 males and 97 females; mean (SD) age, 45 (20) years]. Depending on the results of the anti-ds-DNA-Ab tests used, 25-35 of these patients were assigned to have SLE as assessed by the criteria of the ACR, whereas 105-115 patients did not fulfill these criteria (Table 1). Test results for the 10 patients whose definite diagnosis of SLE was dependent on the detection of ds-DNA-Ab are summarized in the table in the Data Supplement that accompanies the online version of this article at http://www. clinchem.org/content/vol50/issue11/. Seventy-four patients fulfilled at least one of the ACR criteria, whereas 31 patients did not meet any of the ACR criteria. Thirty-two of the 35 patients with established SLE according to the ACR criteria received treatment.

The ROC characteristics of all investigated tests are summarized in Table 1. The Farr RIA, which has been proposed to be the gold standard for ds-DNA-Ab testing, showed the greatest area under the ROC curve in all analyses. However, the other two tests under investigation were able to detect patients with SLE with accuracy similar to that reported for the Farr RIA, although the ORG 604 assay had a slightly greater area under the ROC curve than the EliA in all types of analysis. Statistical comparison of the areas under the ROC curves failed to reveal significant differences among the test results. In line with this finding, there was a high degree of correlation among the test results obtained with the different tests [Pearson correlation coefficient: Farr RIA-EliA, r = 0.934 (P <0.001); Farr RIA-ORG 604, r = 0.862 (P <0.001); ORG 604-EliA, r = 0.771 (P <0.001)]. In 10 of 35 patients, the diagnosis of SLE was based solely on the presence of ds-DNA-Ab as the fourth ACR criterion; therefore, a fourth ROC analysis was based on the remaining 25 patients with four ACR criteria different from ds-DNA-Ab. Again, the Farr RIA had the greatest area under the ROC curve without any statistically significant difference among the investigated assays.

When we used the cutoff values provided by the manufacturers, the specificities were as follows: Farr RIA, 95%; ORG 604, 71%; EliA, 86%. Although the sensitivities of the Farr RIA (44%) and the EliA assay (51%) were similarly low, the sensitivity for the ORG 604 assay was 74%. On the other hand, the ORG 604 assay detected ds-DNA-Ab in the sera of 29 patients with other autoimmune diseases, although a diagnosis of SLE could not be made. In contrast, false-positive ds-DNA-Ab results were obtained for only 12 and 4 patients with autoimmune diseases other than SLE when their sera were tested with the EliA assay and Farr RIA, respectively. ROC analysis allowed the calculation of "optimal" cutoff values for each analyzed test. These cutoff values were found to be close to the cutoff values given by the manufacturers (data not shown).

Anti-nuclear antibody testing with IFT on Hep-2 cells had a specificity of only 54%, whereas its sensitivity was 91%, thereby outperforming the sensitivities of the solid-phase ds-DNA-Ab assays (area under the ROC curve, 0.737; 95% confidence interval, 0.656-0.808). Accordingly, the correlation between test results obtained with IFT on Hep-2 cells and the ds-DNA-Ab assays was low [Pearson correlation coefficient: IFT-Farr RIA, r = 0.361 (P <0.001); IFT-EIiA, r = 0.318 (P <0.001); IFT-ORG 604, r = 0.285 (P <0.001)]. These results did not improve when repeated for anti-nuclear antibodies with a homogeneous staining pattern, which is considered to be the most characteristic fluorescence staining of ds-DNA-Ab (area under the ROC curve, 0.718; 95% confidence interval, 0.636-0.791). The combination of any two tests for the detection of dsDNA-Ab did not provide improved diagnostic accuracy (data not shown).

In this study, the Farr RIA emerged as the assay with the greatest diagnostic accuracy in all analyses, although there was no statistically significant difference with respect to the other two tests. However, the Farr RIA has several drawbacks, including the need for radioactive material, a sophisticated technical analysis, and the associated higher costs. Our analysis demonstrates that the other test systems performed comparably to the Farr RIA and, therefore, may represent easier-to-use alternatives for routine practice. In particular, the specificity of the EIiA system for the diagnosis of SLE was comparable to that of the Farr RIA, in agreement with the studies of Hernando et al. (8) and Villalta et al. (9). The authors of the latter studies proposed the EIiA system as a substitute for the Farr RIA, but their recommendations were not based on ROC analysis. In contrast, we found that the ORG 604 assay showed a greater sensitivity than the Farr RIA, but a rather low specificity of 71%.

It should be emphasized that our results can be generalized only with great caution because our sample size was rather small and a substantial number of the patients studied had received active treatment against SLE at the time of serum sampling. Testing for ds-DNA-Ab was performed at a single point of time. Nevertheless, cross-sectional comparison of the assays should not be influenced by these limitations. To exclude a potential bias attributable to the inclusion of any ds-DNA-Ab test results in the original SLE diagnosis, we repeated the analysis with each assay taken as the basis for SLE diagnosis. Nevertheless, in each analysis, the percentage of correctly identified patients was rather similar. Therefore, the potential bias attributable to the test design seems to be negligible. Selection bias was excluded by choosing unselected consecutive samples from clinical routine, and observer bias was excluded by blinding the laboratory staff with regard to clinical characteristics and diagnosis of the patients.

In conclusion, our study suggests that modern ELISAs and automated enzyme immunofluorescence assays are possible alternatives to the Farr RIA with similar diagnostic accuracies.

References

(1.) Holborow J, Weir DM, Johnson GD. A serum factor in lupus erythematosus with affinity for tissue nuclei. BMJ 1957;2:732-4.

(2.) Ceppellini R, Polli E, Celada F. A DNA-reacting factor in serum of a patient with lupus erythematosus diffusus. Proc Soc Exp Biol Med 1957;96:572-4.

(3.) Tan EM, Cohen AS, Fries JF, Masi AT, McShane DJ, Rothfield NF, et al. The 1982 revised criteria for the classification of systemic lupus erythematosus. Arthritis Rheum 1982;25:1271-7.

(4.) Smeenk RJ, van den Brink HG, Brinkman K, Termaat RM, Berden JH, Swaak AJ. Anti-dsDNA: choice of assay in relation to clinical value. Rheumatol Int 1991;11:101-7.

(5.) ter Borg EJ, Horst G, Hummel EJ, Limburg PC, Kallenberg CGM. Measurement of increases in anti-double-stranded DNA antibody levels as a predictor of disease exacerbation in systemic lupus erythematosus: a long-term, prospective study. Arthritis Rheum 1990;25:634-43.

(6.) Bruns DE, Huth EJ, Magid E, Young DS. Toward a checklist for reporting of studies of diagnostic accuracy of medical tests. Clin Chem 2000;46:893-5.

(7.) Wasmuth JC, Oliver y Minarro D, Homrighausen A, Leifeld L, Rockstroh JK, Sauerbruch T, et al. Phospholipid autoantibodies and the antiphospholipid antibody syndrome: diagnostic accuracy of 23 methods studied by variation in ROC curves with number of clinical manifestations. Clin Chem 2002;48: 1004-10.

(8.) Hernando M, Gonzalez C, Sanchez A, Guevara P, Navajo JA, Papisch W, et al. Clinical evaluation of a new automated anti-dsDNA fluorescent immunoassay. Clin Chem Lab Med 2002;40:1056-60.

(9.) Villalta D, Bizzaro N, Corazza D, Tozzoli R, Tonutti E. Evaluation of a new automated enzyme fluoroimmunoassay using recombinant plasmid dsDNA for the detection of anti-dsDNA antibodies in SLE. J Clin Lab Anal 2002;16: 227-32.

DOI : 10.1373/clinchem.2004.037960

Jan-Christian Wasmuth, * Barbara Grun, Birgit Terjung, Angela Homrighausen, and Ulrich Spengler (Department of Internal Medicine I, University of Bonn, Bonn, Germany; * address correspondence to this author at: University of Bonn, Department of Internal Medicine I, Sigmund-Freud-Strasse 25, 53105 Bonn, Germany; fax 49-228-287-5034, e-mail j-c.wasmuth@uni-bonn.de)
Table 1. Areas under the ROC curves and the corresponding
95% confidence intervals.

 Patients
 positive
 for four
 or more
 ACR Area under
ACR criterion criteria, the ROC
based on (a) n Assay curve (b) 95% CI (c)

ORG 604 35 Farr RIA 0.790 0.711-0.855
 ORG 604 0.780 0.702-0.845
 EliA dsDNA 0.732 0.651-0.803
Farr RIA 25 Farr RIA 0.757 0.676-0.827
 ORG 604 0.734 0.652-0.806
 EliA dsDNA 0.707 0.623-0.782
EliA dsDNA 31 Farr-RIA 0.763 0.683-0.832
 ORG 604 0.759 0.679-0.827
 EliA dsDNA 0.746 0.666-0.816
Without 37 Farr RIA 0.760 0.679-0.829
 anti-ds-DNA-Ab
 (three ACR
 criteria)
 ORG 604 0.743 0.662-0.813
 EliA dsDNA 0.702 0.619-0.776

(a) The presence of ds-DNA-Ab determined the diagnosis of SLE in 10
patients, i.e., 10 of 35 patients fulfilled four ACR criteria,
including the presence of ds-DNA-Ab.

(b) There was no statistical difference between any of the assays
as determined by comparison of the areas under the ROC curves
within any of the categories.

(c) CI, confidence interval. The 95% confidence interval indicates
whether the area under the ROC curve is significantly different
from 0.5, the area under the straight line (sensitivity = 1 -
specificity), which represents no discrimination between normal and
abnormal. If the 95% confidence interval is 0.5, the area under the
ROC curve is significantly different (see description of
statistical analysis).
COPYRIGHT 2004 American Association for Clinical Chemistry, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2004 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Technical Briefs
Author:Wasmuth, Jan-Christian; Grun, Barbara; Terjung, Birgit; Homrighausen, Angela; Spengler, Ulrich
Publication:Clinical Chemistry
Date:Nov 1, 2004
Words:2120
Previous Article:Simultaneous measurement of kynurenine and tryptophan in human plasma and supernatants of cultured human cells by HPLC with coulometric detection.
Next Article:Analysis of methylated genes in peritoneal fluids of ovarian cancer patients: a new prognostic tool.
Topics:

Terms of use | Copyright © 2017 Farlex, Inc. | Feedback | For webmasters