How to check costs and quality of point-of-care testing.

Traditional QC techniques are ineffective in safeguarding the quality of POCT. What are the system's weaknesses and how do we improve quality assurance programs in a practical, cost-effective manner?

Testing of Quality control specimens generally is viewed as a critical component in the quality assurance programs of clinical laboratories.[1] For most routine laboratory tests, QC analyses check multiple steps in the testing process: reagent preparation and stability, calibrator preparation and stability, calibration, sample and reagent delivery, mechanical systems, cuvette consistency, photometers, and electronic components. Typically, reagents are prepared in liquid form and have a working stability ranging from hours to weeks, and many systems of analyzers such as pipettors, electrodes, and cuvettes are reused. These complicated mechanical and fluidic systems should be checked periodically to monitor change in performance.

Based on experience and recognition of the importance of QC in clinical laboratories, most laboratorians stress that QC should have a similar central role in quality assurance for point-of-care testing (POCT). Currently, performance and documentation of the analysis of liquid quality control materials on a daily or per shift basis are major components of most quality assurance programs.[2] Furthermore, the regulations of CLIA '88 and accrediting agencies such as CAP and JCAHO legally require frequent QC testing.[3] Unfortunately, the traditional QC testing appropriate to central laboratories does not work effectively to prevent or detect most problems in POCT. Consequently, QC practice for POCT often becomes an effort to satisfy regulators and inspectors rather than to improve the quality of testing. While at the October 1996 meeting of the Clinical Chemistry Forum, which was devoted to quality assurance issues, Ron Laessig, PhD and professor of pathology at University of Wisconsin, summarized the problem by saying "traditional quality control is not very good or not very effective, and so we are looking for alternatives at the present time."[4]

Why traditional QC doesn't work for POCT

Traditional QC does not work as well for POCT as it does for central lab testing because of a variety of fundamental differences between testing in the central laboratory and at the point of care (see Table 1). In general, the factors that QC testing is best suited to control - such as changes in test performance caused by recalibration, rapid reagent deterioration, or failures of complex mechanical and fluidic systems - are less critical issues for POCT. For most POCT, there is no reagent preparation, reagents have long stability, and there is no recalibration of an analyzer; therefore, QC performed on a daily or more frequent basis is not needed to detect reagent problems. Point-of-care tests usually have no sample or reagent measurement system and few moving parts to go out of alignment. Also, most POCT is performed with single-unit-use devices in which a new test cartridge or test strip is used for each test. QC testing is poorly suited to detect sporadic defects in unit-dose cartridges caused by manufacturing defects or damage before use. Valid QC testing of cartridges requires extreme consistency of their production and performance.

Table 1

Different characteristics of central laboratory QC versus POCT QC

Central Lab                                    POCT

Frequent recalibration                         Infrequent
                                               recalibration

Reagent preparation required                   No reagent
                                               preparation

Short reagent stability                        Long reagent
                                               stability

Many reusable test components                  Few reusable test
(e.g., electrodes, pipettes,                   components (unit-use,
cuvettes)                                      replaced each test)

Same test components for QC and patient test   Different test units
                                               for QC and patient
                                               test

Complex mechanical systems                     No or few moving
                                               parts

Constant operating environment                 Variable operating
                                               environment

Few operators of instrument                    Many operators of
                                               instrument

Highly trained operators                       Operators with little
                                               laboratory experience

Aged specimen                                  Fresh specimen

Centrifuged specimen (serum or plasma)         Unprocessed whole
                                               blood or urine

High precision goals                           Lower precision goals

Limited data on patient condition              Direct access to
                                               patient condition

Linked to central data systems                 No link to central
                                               data systems

Few analyzers                                  Large number of
                                               analyzers

Low reagent cost per test                      High reagent cost per
                                               test

With unit-use devices, the most critical element of reagent QC has shifted from the laboratory to the manufacturer. Implementation of sophisticated production, inspection, and QC systems by manufacturers has been necessary to ensure constant performance of unit-dose reagents. More extensive QC is performed by the manufacturer than would be possible for any laboratory, and it is performed. on reagents with prolonged stability in the exact form used for patients. The only reagent problem that should be detected through QC by the end user is damage during transport or storage. Even then, traditional QC testing will not detect deterioration of reagents from problems such as an uncapped vial of glucose or urinalysis test strips unless that specific vial is tested.

Factors not controlled by QC

Traditional QC testing provides little control over some of the most critical factors for POCT: operator variability, manual test technique, and changes in operating environment. First, because there are many operators who use the point-of-care equipment, QC is usually performed by only a few of these operators on any given day. Traditional QC programs are poorly suited for testing operator performance and technique for POCT because operators are assessed infrequently. Traditional QC is performed most frequently by the most experienced and skilled operators, even if there are efforts to rotate performance of QC among staff members.

Second, the most critical and technique-dependent step in POCT may be the correct application of sample material to test cartridges or test strips. This step often is not validly checked with QC testing because a different technique is used for the addition of control versus patient samples.

Finally, QC testing usually is performed at a central location and does not control for the changes in operating conditions of portable equipment, such as glucose meters. The device may be placed on a surface that is not level, used at a site with a different temperature and humidity, or used in a room with either dim lighting or direct sunlight that may affect stray light detected by photometers.

Issues of sample integrity or interferences generally are not addressed by QC testing in either the central laboratory or a point-of-care setting. These issues are addressed in a central laboratory by examining serum or plasma for unusual appearance caused by problems such as hemolysis, lipemia, or icterus. Potential interferences are less likely to be identified in a point-of-care setting, because whole blood samples are used for testing; and in whole blood, red cells obscure other colored components.

High cost of traditional QC for POCT

Not only is traditional QC ineffective in assuring the quality of POCT, it is also costly compared with QC for central laboratories. POCT usually employs a larger number of analyzers that require testing, and the reagent costs per test are higher. Consequently, costs for reagents and labor to perform QC testing are usually many times greater than for similar programs in centralized laboratories.

Traditional QC practices can be cost prohibitive and are often a major factor limiting the availability of POCT. As an example, Paula Santrach, MD, at the Mayo Clinic in Rochester, Minn., estimates that the performance of QC every 8 hours for activated clotting time testing at the point of care, as mandated by CLIA '88 regulations for coagulation testing, would cost $289,200 annually and require 3.9 full-time equivalents to perform the testing.[4] The dialysis unit at the University of Alabama discontinued testing of activated clotting times because the cost of controls was excessive. Now, no on-site monitoring of coagulation is available for the dialysis patients. From the viewpoint of some laboratorians who feel threatened by competition from POCT, this is a desirable outcome; highly restrictive regulations make it difficult for POCT to compete with the central laboratory. However, this may not promote the ultimate goal of laboratory testing - improved patient care.

An example of current QC experience

For three years, the Office of Bedside Testing at the University of Alabama Hospital has coordinated the operation of approximately 40 glucose meters operated by 1,000 staff members on nursing units to perform more than 100,000 patient tests annually. The program initially required QC testing twice per day with three levels of QC material. This entailed approximately 70,000 control analyses per year with a materials cost of about $35,000, plus 2,000 hours of labor to perform the tests. Control values outside of acceptable limits were almost always caused by use of the wrong control level, contamination or degradation of control materials, or improper addition of control material to test strips.

Table 3

What affects the usefulness of frequent analyses of liquid QC

Makes QC more useful                Makes QC less useful

Brief reagent stability             Extended reagent stability

Frequent calibration                Infrequent calibration

High precision goals                Low precision goals

Low technique dependence            High technique dependence

Analyzer with little internal       Analyzer with high internal
diagnostic capability               diagnostic capability

Constant operating environment      Changing operating environment

Many moving parts                   Few moving parts

Multiple-use components             Single-use components

Failures observed during QC rarely indicated a problem with the meter or test strips. Review of more than 200 instances in which meters required service or replacement determined that only about 10% of cases were identified by QC testing. Most problems were identified by visual inspection of damaged meters or by the internal quality checks of meters. These measures, which have virtually no direct cost, detected approximately 10 times as many problems as an expensive QC program. Some operators with technique problems were detected by QC and proficiency testing, but this benefit largely disappeared when meters were switched to less technique-dependent, no-wipe test systems. Currently, there are few apparent benefits to performance of QC twice per day or even daily - it can, albeit rarely, detect meter problems, and it provides additional training experience for personnel. The number of QC analyses at the Office of Beside Testing has been decreased to the minimum required by current regulations - two levels per day for each meter. Performance of controls is guaranteed by programing the devices to prohibit users from testing unless controls are analyzed daily.

Improving quality assurance for POCT

There are several practical lessons from the experience described above. First, despite large investments in reagents and time, traditional QC has low yield for glucose meters. Second, observant operators and internal diagnostics of meters were more effective quality assurance tools. Third, even a simple device such as a glucose meter requires periodic service or replacement. There is a need for effective service support and rapid availability of replacement instruments. Fourth, selection of equipment with minimal technique dependence is important for consistent testing by large numbers of operators.

Because of the more challenging operating environment and larger number of operators, it is desirable to have some measure of quality control testing performed with every patient analysis rather than relying on traditional daily QC testing. Paradoxically, the solution to the ineffectiveness and inefficiency of QC for POCT may be to perform QC more frequently, but to use different forms of QC. It must be assured that an analyzer was not dropped and damaged in transit from the last analysis, that it is in a suitable operating environment, that the operator performs the test correctly, and that an adequate sample was used. A quality system must be designed that will control for variables such as meter operation, test environment, operator variability, sample integrity, and clinical use of results for each test that is performed.

Performance of these evaluations for every analysis will more effectively identify problems and will decrease the needed frequency and cost for analyses of liquid QC material. A variety of ways to achieve this end are listed in Table 2 (p. 34), one point being to use devices with internal diagnostics such as electronic checks, photometer checks, and operating temperature checks.

Important components of a quality system for POCT requires not just assurance of proper meter operation; inventory control to maintain proper storage of reagents is also essential. In addition, appropriate transmission and clinical correlation of results is an important check of the quality of results. Moving testing into the hands of clinical staff may lower the level of laboratory expertise, but results should be provided directly to staff who can assess whether results are consistent with clinical condition and previous values. Questionable results can be repeated immediately. The transfer of test results into central laboratory information systems can contribute to ongoing comparison with results from the central laboratory.[5] There is continuing progress with efforts at integration of POCT data.[6]

There is also room for improvement in the error detection capability of many point-of-care devices. It is desirable to have internal controls where possible, and these have been incorporated in many tests such as tests for pregnancy, drugs of abuse, and occult blood. Analyzers should detect inadequate specimen volume, clot, bubbles, or other specimen problems. For electrode measurements, statistical signal analysis of multiple measurements can be useful. It is possible to incorporate indicators of improper storage or reagent degradation in the packaging for many point-of-care reagents, but these are not generally available. Progressive miniaturization of test elements may make it possible to perform duplicate testing within a test cartridge with little additional cost.

Controversies about quality assurance for POCT

How to safeguard the quality of POCT in the most effective manner currently is a highly controversial issue. Many laboratorians remain convinced that daily QC is a sine qua non for assuring test quality. Others view that use of alternative approaches such as testing of analyzers with electronic simulators is a less costly and equally effective approach.

This is more than an academic argument. In many cases, the conclusions reached by regulatory and accrediting agencies will determine whether specific point-of-care tests are economically viable. It is an argument that will determine the expenditures of many millions of healthcare dollars. There has been active debate on this issue, and CLIAC has requested recommendations from the Centers for Disease Control and Prevention about possible changes in Federal regulations for QC practices.[4,7] The National Committee for Clinical Laboratory Standards Subcommittee on Unit-Use Test Devices is also working to develop guidelines for quality systems for unit-use devices.

Part of the difficulty lies in developing regulations that are appropriate for the entire range of testing performed in the point-of-care setting. A host of factors listed in Table 3 determine what affects the usefulness of frequent analyses of liquid QC. It may be essential for some devices that have low reagent stability or that have multiple-use electrodes. Also, the manner in which a device is used and the precision requirements for specific clinical application may impact the needs for QC analysis. Difficulty in identifying a single set of rules that apply to every device and clinical application may lead to greater flexibility and responsibility for each testing site to have a quality assurance program suited for its own needs.

Table 2

Ways to assure quality of POCT for each test performed

* Train operators to identify problems

* Devices with minimal operator dependence

* Devices with internal diagnostics

electronic checks photometer checks operating temperature

* Effective service support

* Effective inventory control

* Internal controls

* Analysis of sample integrity

correct specimen volume clots, bubbles, interferences

* Signal analysis of multiple measurements

* Indicators of reagent expiration

* Duplicate testing

* Clinical correlation of test results

testing by clinical staff test results in information systems

References

1. Westgard JO, Bawa N, Ross JW, Lawson NS. Laboratory precision performance: State of the art versus operating specifications that assure the analytical quality required by Clinical Laboratory Improvement Amendments proficiency testing. Arch Pathol Lab Med. 1996; 120 (7):621-625.

2. Jones BA, Howanitz PJ. Bedside glucose monitoring quality control practices: A College of American Pathologists Q-Probes study of program quality control documentation, program characteristics, and accuracy performance in 544 institutions. Arch Pathol Lab Med. 1996; 120(4):339-345.

3. Ehrmeyer SS, Laessig RH. Regulatory requirements (CLIA '88, JCAHO, CAP) for decentralized testing. Am J Clin Pathol. 1995; 104 (suppl 1):S40-S49.

4. Auxter S. Looking at laboratory quality control in a new light. Clin Lab News. 1996;22(12):5.

5. Hortin GL, Utz C, Gibson C. Managing information from bedside testing. MLO. 1995;27(1):28-32.

6. Skjei E. POC data-handling efforts more than lip service. CAP Today. 1996;10(9):22-28.

7. Stine V. CDC searches for answers to QC issues. Clin Lab News. 1996;22(11):11.

Glen L. Hortin is an associate professor of pathology and head of the Section of Clinical Chemistry, University of Alabama at Birmingham, and medical consultant for the Office of Bedside Testing at University of Alabama Hospital.