Beyond looking glass: why proficiency testing is crucial to laboratory quality control.

Beyond the looking glass: Why proficiency testing is crucial to laboratory quality control

Before you buy a statistical quality control package or adopt some other costly quality assurance/quality control program, ask yourself one question: How good are my laboratory measurements? The answer to this question may well determine the effectiveness of your quality control efforts.

Over the years, labs have utilized what might be called a self-comparison or "looking glass" approach. Calibrating instruments to in-house standards was assumed to guarantee that the equipment was operating properly and that lab personnel were correctly following an established test method. Two problems with this approach are immediately apparent. First, the majority of rubber tests lack absolute standards. Second, there is no indication how well the lab's results correlate with other labs performing the same test. In-house standards, where available, serve an important function, but cannot represent the sole means of assuring quality in the testing lab.

Traditional round robin testing, in which the same sample is tested and passed from lab to lab, is impractical when dealing with a large number of labs and is inappropriate for destructive tests and many materials, particularly those affected by environmental conditions. Its obvious advantage is that each participating lab tests the same sample(s). The process, however, is time-consuming and subject to delays if one lab fails to pass on the sample in a timely fashion. The supervisor of such a test must acknowledge the possibility of accidental or intentional tampering of the sample by one or more of the labs. Historically, round robins have been offered on an informal basis, so that labs failed to receive consistent feedback regarding performance.

The National Bureau of Standards (now known as the National Institute of Standards and Technology) developed a concept of formalized, large-scale interlaboratory testing in the late 1960s and applied it to rubber testing in 1970. NBS provided standard reference materials (SRMs) to various industries, but were unsure how individual labs used these standards to improve the quality of their measurements. NBS also desired to know what steps were taken when such standards were not available. As a result, NBS hoped to be able to assist labs in improving test measurements. The first interlaboratory program for rubber focused on tensile properties of rubber. The program later was extended to include tests for other characteristics of vulcanized and raw rubber. Responsibility for the program's operation was transferred to the private sector in the mid 1970s. Although there are several types of interlaboratory studies, the majority of which concentrate on validation of a test method, NBS initiated a proficiency testing program which focuses mainly on overall lab performance.

Though each of the three types of testing (self comparison, round robin and proficiency testing) discussed above serve important functions in the quality assurance process, proficiency testing permits a lab to assess the level and uniformity of its testing by comparing its results to those of other labs. As a by-product it also provides the industry with an assessment of the state of testing capabilities, particularly where true values are lacking for test results.

A number of other major benefits can be attributed to proficiency testing, including:

* Enables a lab to evaluate both instrument calibration and operator technique. Instrument calibration procedures check only the instrument, and yet all tests can be affected by operator techniques. In addition, the analysis can be extended to an examination of laboratory/material interaction problems, often overlooked by labs, which can result in inconsistent test results.

* Results in a savings of production costs. A consistently low or high test result, or any uncertainty regarding the accuracy of a test result, could increase manufacturing costs by requiring additional processing and/or quantities of expensive raw materials to assure meeting production specifications. Participation in a proficiency program allows a lab to track results over a period of time, conduct trend analyses and make corrections as needed on an ongoing basis.

* Provides documentation of a lab's testing capabilities. Independent testing labs can detail their testing capabilities for prospective clients, and manufacturers can reassure customers (both current and prospective) of their ability to test for compliance with specified requirements.

The final aspect of proficiency testing is not so much a benefit as it is need-based: for those labs seeking accreditation for rubber testing through the American Association for Laboratory Accreditation (A2LA), participation in a proficiency program is a requirement. And this requirement does not stop at the borders of the United States. In many cases, A2LA accreditation has achieved cross-recognition with European and other non-U.S. accrediting bodies, eliminating the need to become re-accredited when entering the international arena. For those labs seeking to assist their organizations in competing in a global marketplace, the need to be accredited - and thus participating in a proficiency testing program - will be a part of normal operations.

The rubber testing program started at NBS continues today. Confidentiality of test results is assured, as program participants are identified only by lab code numbers. As an added benefit, the Rubber Division of the American Chemical Society created an interlaboratory testing committee that provides on-going guidance for existing programs. Committee members are available to assist proficiency testing program participants experiencing technical problems. In addition, many committee members are involved with the American Society for Testing and Materials (ASTM), so that continual feedback to appropriate standards writing bodies occurs.

Proficiency testing is conducted using an enhanced Youden plot. The term Youden plot describes not only the graphical technique but also the design procedures used in evaluating the performance of each laboratory through the results obtained on paired test samples. After analyzing the results for each individual sample, a simultaneous analysis of test results for both samples is performed.

The results for all labs are presented in a table and a plot (see figures 1 and table 1). The table includes lab means, deviations from the grand mean and comparative performance values (CPVs) for each sample. The CPV is the "h statistic" per ASTM E691-87 and is a ratio indicating the number of standard deviations from the grand mean. The critical value for each CPV will vary depending on the number of labs participating in a test. Instrument models are identified by codes so that labs can compare results obtained on similar instruments. Many instrument manufacturers monitor test results so that they can better serve their customers who experience problems. Finally, grand means and between-lab standard deviations for both samples are given at the end of each table.

In the enhanced Youden method, if a lab exceeds the critical limit on either of the samples, it will fail the simultaneous analysis. Also, a lab that looks fine for individual sample CPV results can still receive a data flag (* = caution or X = stop work) if the results are inconsistent. An example of this might be a +1.5 CPV for sample A and a -2.2 CPV for sample B. The CPVs can also be used to identify systematic error. Both of these errors are discussed below.

For each laboratory, the lab mean for the first sample is plotted against the lab mean for the second sample with each point representing a laboratory. The horizontal and vertical axes are the grand means for each sample. When 20 or more laboratories are in the statistics, an ellipse is also drawn so that 95% of the time a randomly selected laboratory will be included inside the ellipse. Labs receiving an asterisk (*) fall within the 95% - 99% ellipse, which is not shown on the plot. Labs receiving an "X" fall outside the 99% ellipse.

When a lab's data point falls on the major axis of the ellipse, the lab is consistent in its measurements between the two samples but exhibits an offset from the grand mean. This offset is labeled systematic error. Causes of systematic errors include improper calibration, an idiosyncracy of the particular make/model of equipment, or a modification to the testing procedure.

If the lab's plotted point falls to the side of ellipse, it indicates that there are differences in the way the lab tested the two samples. This type of error may be attributed to the analyst deviating from the procedure when testing one of the samples or a material interaction occurrence with the instrument or room conditions. The inconsistency is reflected in the CPVs for the two samples, such as a +1.5 CPV for sample A and a -2.2 CPV for sample B. It is appropriate to note here that a single data flag is not cause for alarm. Since proficiency testing is offered at a regular interval, labs are encouraged to look for trends in test results.

By searching for these trends, a lab creates an on-going process that works to validate the quality found in the lab and, by extension, the products and services offered by its company. The economic ramifications of these efforts are obvious. And as such, the need to utilize such techniques as proficiency testing as part of an overall quality control/quality assurance program becomes a critical component in competitive strategy. In today's global marketplace, companies must look beyond themselves to attain the pinnacle of quality. [Figure 1 Omitted] [Tabular Data Omitted]

Janine L. Leete, who has worked extensively with the Youden plot, is program manager for Rubber & Plastics at Collaborative Testing Services, Inc.