Quality assessment activities in Latin America and the Caribbean.
Latin America and the Caribbean encompass a geographic region made up of South america, Central America, and the islands in the Caribbean Sea. This vast area consists of 43 countries, 13 in South America, eight in Meso America (including Central America and Mexico), and 22 (islands) in the Caribbean. Unfortunately, due to the size and diversity of testing practices among diagnostic laboratories in this region, implementing quality assessment activities and standardizing lab work in each of these countries as well as in the entire area are no easy tasks.
An estimated 150,000 diagnostic laboratories operate in Latin American and the Caribbean. Most of these facilities are privately owned and function independently, while some labs are government-supported clinical diagnostic centers associated with hospitals. In many instances, the latter group participates in the epidemiologic surveillance programs of their local Ministries of Health. Labs located in countries where a Social Security system is operational function autonomously. Further complicating matters, some laboratorians with different titles and academic backgrounds (e.g., lab directors, clinical chemists, pathologists) direct both a clinical diagnostic lab and a public health lab. Consequently, efforts are being made in certain countries to develop laboratory accreditation and personnel certification programs.
Regardless of the various structured, legal approaches that exist in these facilities, the establishment of technical standards is imperative - not only to address the protection of the lab environment and its staff from potential hazards but also to ensure test results are accurate and produced in a timely fashion.
External performance evaluation
Mechanisms designed to alert employees of local healthcare systems to the need for the standards noted above are provided through various external performance evaluation programs. Combined with internal quality control (QC) and periodic audits, weaknesses can be identified and strategies can be devised to overcome deficiencies.
The World Health Organization (WHO), through its Regional Office for the Americas, the Pan American Health Organization (PAHO), supports external performance evaluation programs in clinical chemistry, hematology, parasitology, microbiology, human immunodeficiency virus (HIV) serology, and syphilis serology. National reference labs are encouraged to participate in these programs and expected to replicate these activities in their own countries.
Additionally, partnerships are being promoted with professional associations, non-governmental organizations, and academic centers in North America, Latin America, and the Caribbean to identify the regional needs and technical resources necessary to cooperate with national authorities to address key lab testing issues. Among the current strategies used to promote PAHO objectives are local situation assessment, the dissemination of information, and direct support of lab personnel.
Several voluntary quality assessment activities are available to labs in Latin America and the Caribbean. Two such programs, conducted by the U.S. Centers for Disease Control and Prevention (CDC), are the Model Performance Evaluation Program (MPEP), designed to evaluate the performance of labs that conduct HIV Type 1 antibody testing, and the Syphilis Serology Proficiency Testing (SSPT) program. Labs in this region also can participate voluntarily in an international external quality assessment program for hematology sponsored by WHO.
MPEP. The MPEP was developed in 1986 and performance evaluation (PE) shipments (i.e., biannual panels of samples sent to participants) implemented in 1988. Initially, 800 labs were enrolled in the MPEP, including the participation of 10 Latin American labs. The number of latin American participants increased to 40 in 1995, with 10 participants from the Caribbean subregion, 21 from South America, and nine from Central America. While two participants from two different Latin American countries were dropped from the program in 1995, four new facilities were added that same year. There has been a steady increase in the number of Latin American labs enrolled in the MPEP since 1988.
Unfortunately, not all labs registers in the MPEP report the test results of their PE shipments to the CDC. In 1995, for instance, only 26 (65%) of the 40 enrolled labs reported PE results, although this number did increase to 31 (77.5%) of 40 labs in 1996. Participation is recorded provided a lab reports results for at least one biannual shipment.
SSPT program. Laboratories' enrollment and participation in the CDC's SSPT program is similar to that of the MPEP. The SSPT began in 1988 with 14 participating Latin American labs. This number increased to 21 participants with the addition of three new centers in 1995; four more labs enrolled in 1996. In 1994, 15 (83.3%) of the 18 enrolled labs indicated they could participate in SSPT shipments. Of these 15 labs, nine (60%) reported complete results for their three yearly panels. The number of participants active in SSPT shipments during 1995 increased to 16 (76.2%) of 21 enrolled labs, and among these, 13 (81.25%) reported complete test results.
Monitoring test accuracy
Increasing the number of labs that participate in quality assessment programs is only half the battle. It is just as important for test results to be monitored for accuracy. Case in point: Of the 21 facilities that reported EIA results in a 1994 MPEP performance survey, four false-positive results were reported for 69 analyses performed on well-characterized HIV antibody-negative samples, while no false-negative results were reported for 161 determinations performed on HIV antibody-positive samples.
In contrast, of the 25 labs reporting EIA results during a 1995 MPEP performance survey, no false-positive results were reported for 51 determinations performed on HIV antibody-negative samples. And yet, on 102 analyses performed on HIV antibody-positive samples, three false-negative results were reported. These samples were confirmed by Western Blot in 18 separate labs, where 13 results were reported "indeterminate" and two were reported "negative." What's more, the percentage of labs with acceptable SSPT program test scores during 1995 ranged from 71% to 93%.
Twenty laboratories from Latin America and the Caribbean are registered in a program sponsored by WHO called the International External Quality Assessment Scheme for Hematology. Eleven of these facilities responded to the first survey conducted in 1996. In determinations for hemoglobin using cyanmethemoglobin methodology, two of the 11 labs reported values higher than the upper limit of the acceptable range. Using lysate methodology, four of seven labs reported unacceptable results.
To encourage development of programs to evaluate lab performance and promote standardization at the national level, various strategies are being promoted, one of them through the pyramid of national networks of public sector labs. The pyramid structure functions with the municipal labs forming the largest bottom layer, they work with the state labs; the state labs, in turn, work with one national reference lab. In one country, 14 state labs participated in a preliminary evaluation in clinical chemistry. Twenty-three analyses were to be determined, yet only glucose results were reported by all 14 facilities, and determinations of acid phosphatase GGT, CK, and sodium were performed by only one facility. Glucose levels are reported in the figure, left. No reference values or acceptable ranges are included because they were not known at the time of this survey. Still, the variability of these values indicates the need for standardization and control procedures.
In another country, 230 clinical microbiology labs participated voluntarily during a 3-year period (1993-1995) in surveys designed to test for bacterial resistance to antimicrobials. Following College of American Pathologists (CAP) guidelines, these labs were rated either excellent, good, fair, or poor, depending on their scores for accuracy in determining zones of inhibition seen in the Kirby-Bauer disk diffusion test, the interpretation of those zones, and the manner in which results were reported. In 1993, about 80% of participants were categorized as being poor; this percentage dropped to 52% in 1994 and 1995.
More work to be done
The above examples illustrate the need for continued monitoring of laboratory performance and external technical support of national reference laboratories in the Region of the Americas. Of course, encouraging general laboratory usage of QC procedures will improve the validity of lab results as well.
To strengthen the labs' testing performance in this region, PAHO's Regional Program on Laboratory and Blood Services has identified two main areas that still need to be addressed:
1. The establishment of norms, guidelines, and technical standards that can be used as the basis for assessing labs in each of these countries.
2. The integration of these labs into specific health programs and surveillance systems.
The authors would like to thank Drs. Jose Ignacio Santos and Carlos F. Amabile for providing lab performance data related to tests that determine antimicrobial susceptibility, and Dr. Victoria Pope for providing syphilis serology proficiency testing data.
Jose Ramiro Cruz is the regional laboratory advisor, Pan American Health Organization, in Washington, D.C. William O. Schalla is the chief for the Model Performance Evaluation Program, Laboratory Practice Assessment Branch, Division of Laboratory Systems, Public Health Practice Program Office, Centers for Disease Control and Prevention (CDC), in Atlanta, Ga. Sandra A. Larsen is the chief, Treponemal Research Branch, Division of AIDS, Sexually Transmitted Diseases, and Tuberculosis Laboratory Research, National Center for Infectious Diseases, CDC.