Printer Friendly

Clinical evaluation of an algorithm for short sample detection on a multi-analyte panel using a point-of-care analyzer.

Advances in instrument technology have led to extensive automation of the testing process. Use of information systems enables accurate and reliable tracking of laboratory test data and has facilitated the implementation of sophisticated informatics-based quality-control and quality-assurance programs. Application of quality-control systems using commercially available control samples has enabled the detection of analytical errors. In addition, use of laboratory data from patients has been helpful in detecting analytical errors as well as pre- and postanalytical errors. Examples of error detection schemes that rely on use of patient-derived data include checking of patient test results against predetermined limits or against previous values generated for the same patient (1, 2). These detection schemes, referred to as limit checks and delta checks, respectively, are in common use. Other error detection schemes that have been devised include arithmetic checks and "interchecks" on analytes that are physiologically linked, such as the anion gap, and relationships between other analytes, such as albumin and globulin or aspartate and alanine aminotransferase (3). More recently, an automated expert system that can perform thousands of rule-based checks and hundreds of statistical checks of patient test results has been introduced (4).

Among the potential errors that can occur during the analytical process is the addition of insufficient sample to the reaction mixture, either by itself or because of improper dilution of the sample with reagent diluent. This type of error may be caused by a variety of factors, including insufficient sample for analysis, insufficient reagent or sample diluent, obstruction of the sample probe by fibrin or other material, or increased viscosity of the sample. This type of error usually leads to abnormally low recovery of analytes and is often referred to as a "short sample". Many instruments use pressure transducers to verify system pressure and ensure sample integrity. However, systems that rely on pressure transducers for short sample detection do not measure the actual volume of fluid dispensed and are more likely to detect only catastrophic events in which the volume of delivered sample is significantly altered.

Errors attributable to short samples can also be identified by use of some of the previously mentioned data-checking algorithms. However, algorithms for this type of error detection usually require that the instrument performing the analysis be interfaced to a computer system capable of performing statistical analysis and accessing previous data that may not be available for all patients. To our knowledge, no algorithm that resides within the software of the analyzer itself has been specifically developed to analyze data resulting from "short" samples.

We developed an algorithm for the detection of short samples that resides completely within the instrument software on the Piccolo[R] chemistry analyzer. This point-of-care analyzer can perform analyses on various test panels by use of self-contained, disposable reagent discs. After the introduction of 90 [micro]L of whole blood, serum, or plasma into the disc and insertion of the disc into the analyzer, the testing system performs all functions necessary for analysis. These functions include separating plasma from other cellular components (for whole blood samples), measuring the appropriate volume of plasma or serum and diluent needed for analysis, mixing of sample and diluent, delivering the sample/diluent mixture to a reaction cuvette where all reagents are present in lyophilized form, and performing internal quality-control checks. The algorithm has been incorporated into the instrument software to identify short samples by use of reagent discs measuring analytes present in the Centers for Medicare and Medicaid Services (CMS) comprehensive metabolic panel (CMP), basic metabolic panel (BMP), and renal function panel (RFP) and, in addition, a specially designated MetLyte 8 panel; the latter is identical to the BMP except that creatine kinase is substituted in place of calcium.

A database of patient test results was obtained from a large population of hospital patients representing a wide variety of disease states. We analyzed the results of laboratory tests from more than 16 000 unique patient samples collected over a 60-day period. All samples were measured with an Olympus AU640 (Olympus America) and the manufacturer's reagents and procedures. The Olympus data were used to establish the frequency distribution of each analyte, from which we calculated the probability of obtaining a particular value. The probability of a short sample was determined by use of the best fit equation to the frequency distribution. The distribution is normalized to 100% for the mean expected value of the analyte. When the analyte value is greater than the mean value, there is no discrimination value for the probability of a short sample and the analyte value is assigned a probability of 1.0. The probability of a short sample for analyte values below the expected mean value is a function of the expected mean value and SD for the analyte. Analytes under stringent homeostatic control, such as electrolytes and other ions, provide greater discrimination for detection of short samples compared with other analytes, such as enzymes, that can show significant variability. Using the probability as determined by the distribution of analyte value, we defined a mathematical function describing the probability of a sample in a panel of results (e.g., BMP or CMP) having a particular set of results. The probability of a short sample was defined as:

P (short sample) = {n - [P(a) + P(b) + ... P(x)]} / n

where P = probability; a, b, ... x are analyte results; and n is the number of analytes measured in a reagent disc.

After establishing the frequency distribution of analytes measured with the Olympus analyzer, we applied the mathematical function to a large number of samples from a diverse hospital population and determined the cumulative distribution of the probability of detecting short samples. The cumulative distribution of a particular test panel allows the selection of a particular probability to be used as the threshold value for identification of a possible short sample. The number of acceptable false-positive and false-negative test results can be selected based on the individual laboratories' acceptance of this type of error.

The mathematical model was tested with the ABAXIS Piccolo and MetLyte 8 reagent discs specifically manufactured to produce short sample situations in ~10% of reagent discs. This was accomplished by incorporating a defect within the sample transport siphon in the reagent disc. The plastic surface of the siphon was made more hydrophobic than normal, which led to fluidics errors. The analytes present in the MetLyte 8 included glucose, urea nitrogen, creatinine, sodium, potassium, chloride, total bicarbonate, and creatine kinase. We obtained heparinized whole blood and serum samples from 10 individuals. Each of the whole blood and serum samples was analyzed 10 times, and the short sample algorithm was applied to the data. The accuracy of the algorithm was assessed for each series of whole blood or serum replicates by comparing the expected serum or whole blood values (mean [+ or -] 2.0 SD) with the algorithm.

Of a possible 200 sample runs with MetLyte 8 reagent discs, 2 runs were aborted by the Piccolo instrument because of failure of the internal quality-control checks of the reagent disc during sample analysis. Using the mean [+ or -] 2.0 SD as the acceptable target range for replicate results, we identified 24 sample runs that were considered to be short samples. The incidence of short samples, 12.1%, was close to the anticipated incidence of 10%. At a probability threshold of 0.50, the algorithm correctly identified 19 short samples. There were five samples with up to four analytes below the expected measured values that were not identified as short samples by the algorithm. Finally, one sample was incorrectly classified as being a short sample. Thus, when we used an algorithm threshold of 0.50, the sensitivity, specificity, and positive and negative predictive values of the algorithm were 0.79, 0.99, 0.95, and 0.98, respectively. Decreasing the probability threshold of the algorithm to 0.45 would have led to the correct identification of 21 of 24 short samples; however, 1 additional sample would have been incorrectly classified as being a short sample. The sensitivity, specificity, and positive and negative predictive values obtained with the algorithm at a threshold of 0.45 were 0.88, 0.99, 0.91, and 0.99, respectively.

As expected, analytes with narrow probability distribution ranges were more useful in the prediction of short samples. The inclusion of enzymes present in the various CMS panels that we evaluated was found not to be useful for detection of short samples because of the magnitude of increase that might be observed in some patients.

We also tested the algorithm by serially diluting samples. This procedure allowed us to evaluate the sensitivity of the algorithm for the identification of short samples. Using this technique, we found that samples short by as little of 5% of their required volume for analysis could be detected by the algorithm. However, the sensitivity for detection of short samples is dependent on several factors, including the combination of analytes in the test panel and the analyte concentrations used in the calculations.

In summary, we developed an algorithm to detect short samples by evaluating results from numerous patient samples and determining the population distributions for these analytes. The algorithm makes use of the probability distributions for most of the analytes present in the panel. Analytes, such as enzymes, that may show extreme variability are not useful in the algorithm. The greatest challenge for developing such an algorithm is to adjust the cutoff threshold that permits detection of true short samples and can differentiate these from patient samples for which the results are correct. One drawback of our model is that the initial probability distribution was based on results obtained with the Olympus analyzer for a hospital population. However, correlation studies performed between the Olympus and Piccolo analyzer enabled us to account for any differences in analyte recovery between instruments. In addition, although the algorithm was developed and tested with samples from hospital patients, the algorithm should perform equally well in the outpatient setting, where the incidence of abnormal results is less than that seen in hospital patients. Further development and refinement of the algorithm, especially in abnormal populations such as oncology and dialysis patients, is being conducted. Although the algorithm was developed for use with the ABAXIS Piccolo, the algorithm is applicable to any instrument that performs analysis of test panels.

DOI : 10.1373/clinchem.2004.037044

Editor's Note: This poster received a "Critical and Point-of-Care Testing" Poster Award at the Oak Ridge Conference.


(1.) Iizuka Y, Kume H, Kitamura M. Multivariate delta check method for detecting specimen mix-up. Clin Chem 1982;28:2244-8.

(2.) Whitehurst P, DiSilvio TV, Boyadjian G. Evaluation of discrepancies of patients' results: an aspect of computer-assisted quality control. Clin Chem 1975;21:87-92.

(3.) Witte DL, VanNess SA, Angstadt DS, Pennell BJ. Errors, mistakes, blunders, outliers, or unacceptable results: how many? Clin Chem 1997;43:1352-6.

(4.) Valdiguie PM, Rogari E, Corberand JX, Boneu B. The performance of the knowledge-based system VALAB revisited: an evaluation after five years. Eur J Clin Chem Clin Biochem 1996;34:371-6.

Steven C. Kazmierczak, [1] * Vladimir Ostoich, [2] Ken Aron, [2] Audie Hickey, [2] Diana E. Kazmierczak, [1] and Dennis M. Bleile [2] ([1] Department of Pathology, Oregon Health & Science University, Portland, OR; [2] ABAXIS, Union City, CA; * address correspondence to this author at: Department of Pathology, Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Mailcode: L-471, Portland, OR 97239; e-mail
COPYRIGHT 2004 American Association for Clinical Chemistry, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2004 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Abstracts of Oak Ridge Posters
Author:Kazmierczak, Steven C.; Ostoich, Vladimir; Aron, Ken; Hickey, Audie; Kazmierczak, Diana E.; Bleile,
Publication:Clinical Chemistry
Date:Oct 1, 2004
Previous Article:Homogeneous time-resolved fluorescence quenching assay (TruPoint) for nucleic acid detection.
Next Article:Sensitive assay for identification of methicillin-resistant staphylococcus aureus, based on direct detection of genomic DNA by use of gold...

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters