Quality assessment of interpretative commenting in clinical chemistry.
The following factors favor the continuation, or even expansion, of the practice of adding comments to clinical pathology reports (2):
* Introduction of new and complex tests, including the availability of genetic testing, will increase the complexity of medical management. This may influence the interpretation and clinical applicability of existing tests.
* Increased electronic data communication will require clinicians to cope with this traffic and increase the desirability of interpretative filters.
* Because of competition among pathology laboratories, especially but not exclusively in the private sector, interpretative comments add value to the reports that customers receive.
* Given the increasing number of clinical and regulatory guidelines, the laboratory can provide guidance on appropriate test ordering, correct interpretation, and relevant follow-up of results.
* The capability of current computer networks has enabled the use of expert systems and the building of interpretative algorithms to provide relevant clinical interpretation.
Although the analytical activities in the clinical biochemistry laboratory are rigorously quality controlled and most pre- and postanalytical activities are monitored for errors, interpretative commenting in clinical chemistry has until recently escaped this scrutiny. Gordon Challand (3), through the Internet-based "Cases for Comments" program, highlighted the ability to objectively analyze and assess the quality of interpretative comments in clinical biochemistry results. The need for such formal assessment has been recognized by Clinical Pathology Accreditation Limited in the United Kingdom through their funding of a national external quality assessment scheme (NEQAS) for interpretative comments in clinical chemistry. There is now some degree of acceptance of quality assurance (QA) of interpretative commenting within the clinical biochemistry profession (4,5). Recognizing these developments, the Royal College of Pathologists of Australasia /Australasian Association of Clinical Biochemists (RCPA-AACB) Chemical Pathology Group of RCPA Quality Assurance Programs (QAP) Pty Ltd initiated a quality assessment program in patient report commenting, which was piloted in 2000. This program has now gained acceptance among the chemical pathology profession in Australia as a tool for continuing professional development. The original design of the program, its assessment process, and a review of individual case reports have been discussed previously in the pilot program as well as in the subsequent year's program (2, 6). Substantial changes have been made to the design of the program and method of analysis and feedback to participants as a result of the experience gained from previous cycles as well as regular feedback and review sessions with participants at scientific meetings. We present here the case reports distributed in the Patient Report Comments Program in 2002 and discuss the responses from and the feedback to the participants, the assessment of these responses by the program expert panel, and the format of the summary report.
Materials and Methods
Ten cases were distributed in 2002 to all participants of the Chemical Serum Chemistry program, which includes ~600 laboratories. Two of these cases (cases 1 and 3) are shown in Table 1, and the rest are available in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/content/vol50/issuel/. The 10 cases were selected from a routine clinical laboratory attached to a teaching hospital that also serves general practitioners as well as smaller district hospitals. The cases were representative of the spectrum of results routinely encountered in clinical biochemistry practice. With each of the case reports distributed, a standard format was followed with the provision of (a) age and gender of the patient, (b) location where the tests were ordered, (c) the set of biochemistry results for commenting, (d) clinical notes that would typically appear on the request form, and (e) additional relevant information, past or present, available to the laboratory. Space was provided to allow participants to write their interpretative comments.
Each laboratory could send an "official" response, which was generated by the pathologist or senior scientist who would customarily add comments on reports in routine practice. In addition, "unofficial" responses were also accepted from any other interested laboratory personnel because the program was aimed at individuals rather than laboratories. Computer-generated comments were accepted for submission, but these comments were not identified in the program. Currently, participation in this program attracts no subscription charges.
All comments received were broken down into components, and the components were translated/summarized into common keywords or phrases (Table 2). More than one key phrase could be generated from each submitted response. Responses were further subdivided as official or unofficial. An expert panel was selected that consisted of the Program Convenor, the chairman of the Chemical Pathology QAP Group, and the chief examiners of RCPA and AACB. One of the roles of the expert panel was to independently assess the appropriateness of the key phrases from all the comments received. The panel did not see the whole comments from individual participants, only the list of key phrases extracted from the comments submitted by all participants.
Each key phrase was classified as "preferred", "less relevant", or "unacceptable" by the expert panel (Table 2). The preferred key phrases were those judged to be appropriate for that report and that were useful to the attending clinician who would receive the report. Unacceptable comments were inappropriate and could potentially lead to mismanagement of patients. Comments that were classified as less relevant were seen as not useful to the requesting doctors and offering no guidance for patient management, although not erroneous or misleading. A summary report of each case was returned to the participants that included the classification of all key phrases, a frequency bar chart of the commonly used key phrases, a "suggested" comment as recommended by the expert panel, and a brief discussion of the case and one or two key literature references (cases 1 and 3 are shown in Table 3 and the rest of the cases are available in the online Data Supplement). No attempt was made to score the comments. A summary report of each case was returned to the participants within 2 weeks of receipt of their submission.
The participation rate varied from case to case during the year (Table 4). The total number of key phrases extracted from comments of all participants varied depending on the spectrum and complexity of the case report. The mean number of key phrases for the 10 cases was 51 (range, 29-71; see Table 4). The break down into official and unofficial participants was, on average, relatively evenly distributed: 52% official and 48% unofficial (Table 4). Table 5 shows the total number of times the three categories of key phrases were used for each case by official and unofficial participants. Although the number of unacceptable key phrases for most cases was higher than the number of preferred key phrases, each unacceptable key phrase in general was used by only a few participants or, in many cases, by one participant only, whereas each preferred key phrase was used by a high proportion of participants. For example, for case 1, there were 8 preferred key phrases used a total of 200 times, whereas there were 12 unacceptable key phrases used a total of 19 times (Tables 4 and 5). The proportion of inappropriate key phrases in the interpretations submitted by the official and the unofficial participants were similar (Table 5).
The full list of key phrases used in comments received for case 1 and their classification according to appropriateness by the expert panel is shown in Table 2. The most commonly used preferred key phrases were "suggest serum/urine osmolality" in 45 (46%) responses, "?secondary to medications" in 39 (40%) responses, "suggest urine electrolytes" in 34 (35%) responses, "severe hyponatremia" in 27 (28%) responses, "?hyponatremia cause of fit" in 16 (16%) responses, and "?psychogenic polydipsia" in 11 (11%) responses.
The full list of key phrases used in comments received for case report 3, which is an example of a more specialized case, are also shown in Table 2. Sixty-two (75%) of the responses suggested "HFE genotype", and 50 (60%) raised the possibility of "?Hemochromatosis". These key phrases were classified as less relevant (7, 8). Key phrases such as "hemochromatosis unlikely" or "iron overload" were classified as unacceptable. Only 5 (6%) participants stated "diabetes not secondary to hemochromatosis", a preferred key phrase, in their comments. "Repeat iron studies", a preferred key phrase, was included in 24 (29%) of the responses, and in fact, this patient had returned a normal transferrin saturation on repeat testing.
Interpretation of laboratory test results is a postanalytical activity. It requires an understanding of the analytical processes involved in generating the results, recognition of potential analytical and preanalytical errors, and correlation of results with the patient's clinical status. Ideally, because no one individual can have full knowledge and expertise in all of the above aspects of the testing cycle, result interpretation should be a collaborative activity with input from the pathologist and the clinician. In reality this is not possible for every test result generated because of the sheer number of tests performed, nor is it necessary in the vast majority of cases. However, in a large proportion of results, attaching a comment adds value to the report and is appreciated by junior hospital doctors and general practitioners. Even specialists may be helped by interpretative comments, especially on test results outside their specialty.
The practice of attaching interpretative comments to clinical chemistry reports varies widely among countries and types of practices as well as the degree of specialization and complexity of the particular test. There are guidelines published by the Royal College of Pathologists in the UK for clinical authorization and reporting of results in clinical biochemistry laboratories (9). It is uncommon for individualized interpretative comments to be attached to general chemistry reports such as liver function tests, urea, and electrolyte profiles. This may be difficult because of the large number of such tests performed by a laboratory as well as the lack of clinical information available to the pathologist to make useful comments (10). However, recent developments in information technology mean that these hurdles are not insurmountable. The development of laboratory expert systems might help overcome the former problem, whereas availability of patient information by means of a unified information system may assist with the latter (11). Most laboratories have a roster of on-duty biochemists whereby pathologists and senior scientists offer interpretation over the phone or in person. The QA program described here may be directed at the activity of adding interpretative comments on printed reports as well as the provision of such verbal advice.
QA of interpretative commenting is still in its infancy, and the methods used need to be developed further and fine-tuned. What has become clear is that it is well accepted by the profession as a useful tool for continuing professional development in the countries in which it is being developed. The proponents hope that the development and administration of, and participation in, such programs would help the profession to scrutinize and improve the practice of interpretative commenting. The key lessons learned by the organizers of this program is that both participation and education could be enhanced by providing expert analysis of responses and an assessment of what is preferred and what is unacceptable (2). In addition, a rationale and key reference(s) are also now included in the summary report on the request of participants.
The analysis of comments into components and the translation into key phrases is subjective, and this step might introduce errors, thus contributing to the diversity and complexity of the interpretation. The key phrases are extracted from a sentence, but attempts were made to preserve the meaning and context of the interpretation. An independent review of this step has been introduced to improve consistency of key-phrase translation. In the NEQAS (UK) program for interpretative comments, each comment as a whole is scored instead of being translated into key phrases. We feel that classifying key phrases is more objective and of more educational value than scoring whole comments. The latter would also be more time-consuming to the panel. The panel of assessors in this program are "experts" holding designated positions in professional bodies, whereas the NEQAS program uses "peers". The assessor panel may not necessarily always get it right; e.g., in the assessment of one case, a key phrase classified as inappropriate appeared in the expert-panel's suggested comment. The anomaly may be because the phrase is appropriate only if used in context (i.e., it is unacceptable if suggesting adequate treatment but preferred if suggesting possible overtreatment). However, an internal quality check has been introduced to confirm consistency between the suggested comment and the key phrase classification. The pros and cons of the various methods of analysis are at this stage unclear and need to be studied further.
Inappropriate commenting is an area of concern because the laboratories are seen as experts and are providing guidance to the requesting clinicians. Inappropriate comments appearing on patient reports with the frequencies listed in Table 5 could lead to a significant number of misdiagnoses and could be dangerous to patient care if acted on by the clinicians. The considerable number of inappropriate key phrases in the comments received could in part be explained by the various assumptions made by participants for cases in which sufficient clinical information was lacking. In reality, the pathologist would contact the requesting doctor by phone and obtain details of the case before commenting. The number of inappropriate key phrases listed may also relate to the fact that this is perceived as an educational program by chemical pathology trainees and junior scientists and that such comments may not in fact be used in real life. However, perhaps surprisingly, the distribution of inappropriate key phrases in the interpretations submitted by the official and unofficial participants were similar (Table 5), suggesting that, on average, professionals who offer interpretation of results in practice do not perform substantially better than other participants in this program. The number of inappropriate key phrases may also reflect recent changes in laboratories, which have led to a decrease in subspecialization with the merging of specialist laboratories into general chemistry laboratories.
These findings do indicate that interpretation provided by laboratory professionals with inadequate expertise can be clinically dangerous and highlight the need for improvement in the standards of interpretation currently provided. The profession needs to address the reasons for the current level of performance of the interpretative service it offers and take steps to greatly improve its quality if it is to contribute to the improvement of patient care and minimize medical errors. QA programs in interpretative commenting, such as that described here, may help in education and continuing professional development in this regard and in monitoring performance in the future.
(1.) Fraser CG, Fogerty V. Interpreting laboratory results. BMJ 1989; 298:1659-60.
(2.) Vasikaran SD, Penberthy L, Gill J, Scott S, Sikaris KA. Review of a pilot quality assurance program for interpretative commenting. Ann Clin Biochem 2002;39:250-60.
(3.) Challand GS. Assessing the quality of comments on reports: a retrospective study. Ann Clin Biochem 1999;36:316-22.
(4.) Marshall WJ, Challand GS. Provision of interpretative comments on biochemical report forms. Ann Clin Biochem 2000;37:758-63.
(5.) Sciacovelli L, Zardo L, Secchiero S, Zanintonotto M., Plebani M. Interpretive comments and reference ranges in EQA programs as a tool for improving laboratory appropriateness and effectiveness. Clin Chim Acta 2003;333:209-19.
(6.) Lim EM, Vasikaran SD, Gill J, Calleja J, Hickman PE, Beilby J, et al. A discussion of cases in the 2001 RCPA-AQAP Chemical Pathology Case Report Comments Program. Pathology 2003;35:145-50.
(7.) Vautier G, Murray M, Olynyk JK. Hereditary haemochromatosis: detection and management. Med J Aust 2001;175:418-21.
(8.) Bulaj ZJ, Ajioka RS, Phillips JD, LaSalle BA, Jorde LB, Griffen LM, et al. Disease-related conditions in relatives of patients with hemochromatosis. N Engl J Med 2002;343:1529-35.
(9.) Standing Advisory Committee in Chemical Pathology. Guidelines for the provision of interpretative comments on biochemical reports. Bull R Coll Pathol 1998;104:25.
(10.) Prinsloo PJ, Gray TA. A survey of laboratory practice in the clinical authorization and reporting of results. Ann Clin Biochem 2003; 40:149-55.
(11.) Smith BJ, McNeely MD. The influence of an expert system for test ordering and interpretation on laboratory investigations. Clin Chem 1999;45:1168-75.
EE MUN LIM,  KEN A. SIKARIS, [2,3] JANICE GILL,  JOHN CALLEJA,  PETER E. HICKMAN,  JOHN BEILBY, [1,5] and SAMUEL D. VASIKARAN5 [5,6] *
 Clinical Biochemistry, Pathcentre, Perth WA6909, Australia.
 Melbourne Pathology, Collingwood VIC3066, Australia.
 RCPA-AACB Chemical Pathology QAP, RCPA Quality Assurance Programs Pty Ltd., Flinders Medical Centre, Adelaide SA5042, Australia.
Chemical Pathology, The Canberra Hospital, Woden ACT2606, Australia.
 School of Surgery and Pathology, University of Western Australia, Perth WA6909, Australia.
 Core Clinical Pathology & Biochemistry, Royal Perth Hospital, Perth WA6001, Australia.
 Nonstandard abbreviations: NEQAS, national external quality assessment scheme; QA, quality assurance; RCPA, Royal College of Pathologists of Australasia; AACB, Australasian Association of Clinical Biochemists; and QAP, Quality Assurance Programs.
* Address correspondence to this author at: Department of Core Clinical Pathology and Biochemistry, Royal Perth Hospital, CPO Box X2213, Perth WA6847, Australia. Fax 61-8-9224-1789; e-mail firstname.lastname@example.org. gov.au.
Received July 22, 2003; accepted October 21, 2003.
Previously published online at DOI: 10.1373/clinchem.2003.024877
Table 1. Two of the 10 cases distributed in the Patient Report Comment Program in 2002. Case 1 Patient details 44-year-old woman Patient location Emergency department Plasma electrolytes Sodium 112 mmol/L (134-146 mmol/L) (a) Potassium 3.0 mmol/L (3.4-5.0 mmol/L) Bicarbonate 20 mmol/L (22-32 mmol/L) Urea 2.2 mmol/L (3.0-8.0 mmol/L) Creatinine 40 [micro]mol/L (50-95 [micro]mol/L) Clinical details Seizure Additional information Long history of schizophrenia Case 3 Patient details 16-year-old boy Patient location General practitioner Plasma iron studies Iron 30 [micro]mol/L (10-30 [micro]mol/L) Transferrin 23 [micro]mol/L (23-43 [micro]mol/L) Transferrin saturation 65% (14-53%) Ferritin 110 [micro]g/L (30-500 [micro]g/L) Alanine aminotransferase 17 U/L (<40 U/L) [gamma]-Glutamyltransferase 17 U/L (<40 U/L) Clinical details ?Hemochromatosis Additional information Newly diagnosed type I diabetes mellitus (a) Values in parentheses are the reference intervals. Table 2. Key phrase classification for case reports 1 and 3. Preferred Case 1 Suggest serum/urine osmolality ?Secondary to medication Suggest urine electrolytes Severe hyponatremia ?Hyponatremia causing fit Suggest repeat sample Suggest plasma glucose ?Psychogenic polydipsia Case 3 Increased transferrin saturation Suggest repeat iron studies Normal ferritin ?Iron therapy Patient's young age noted Diabetes not Ilry to HC ?Fasting morning sample Less relevant Case 1 ?SIADH (a) Hyponatremia ?Water overload ?Drip arm Check patient hydration status ?Phenothiazines Low electrolytes/urea/creatinine ?Renal or gastrointestinal loss ?Hyperglycemia ?Sodium intake ?Tricyclic antidepressant ?Sample integrity ?Drug overdose Cannot exclude SIADH Suggest TFTs, LFTs ?Lithium induced Suggest serum cortisol Exclude adrenal, pituitary disease Exclude cardiovascular/renal/liver disease ?Diuretic therapy Hypokalemia Consult clinician ?Anticonvulsant therapy Cerebral edema Exclude pseudohyponatremia ?Addison disease Phone laboratory with queries Restrict fluid intake Hemodilution ?Vasopressin disorder ?Diarrhea Possible metabolic acidosis Normal renal function Results checked Suggest anion gap Case 3 Suggest HFE genotype ?Hemochromatosis (HC) ?Family history ?Early HC Normal LFTs Screen family ?Abnormal iron studies, Ilry to diabetes mellitus Suggests no iron overload Consider family studies No comment No Liver tissue damage Suggest FBE, reticulocyte count ?Early iron overload Suggest iron studies on family Measure serum and urine albumin Suggests no organ damage No diagnostic findings High ferritin for age Suggest close follow-up Too young for HFE genotyping Consider venesection ?Diurnal variation If HC; monitor iron studies yearly Suggest fasting glucose, Hb [A.sub.1c] Suggest diabetes mellitus-related antibodies ?Juvenile HC ?Misentry of iron studies results Suggest unsaturated iron binding capacity Unacceptable Case 1 Suggest serum lithium Suggest blood gases, calcium, phosphate Suggest antidiuretic hormone, creatinine, chloride ?Nephrogenic diabetes insipidus Suggest full biochemical profile Suggest water deprivation test Consider hypertonic saline ?Acute tubular necrosis Suggest ACTH Exclude hypertonic hypernatremia ?Kidney problems ?Brain injury Case 3 Unlikely HC Iron overload Diabetes Ilry to HC Suggest HLA-H test Assess liver function Annual follow-up ?Diabetes secondary to HC Suggest liver biopsy ?Iron loading anemia Suggest LID and bilirubin Suggest repeat LFTs Check for ketones Suggest cardiac work-up Check lipase, amylase Idiopathic HC Consider viral hepatitis ?Liver disease (a) SAIDH, syndrome of inappropriate antidiuretic hormone; ACTH, adrenocorticotropin; TFT, thyroid function tests; LFT, liver function tests; HC, hemochromatosis; LID, lactate dehydrogenase; FBE, full blood examination; Hb, hemoglobin; Ilry, secondary. Table 3. Suggested comments and rationales for 2 of the 10 sample cases distributed. Case 1 Suggested Results confirmed by repeat analysis. Suggest comment repeat plasma electrolytes and measure plasma osmolality and spot urine osmolality and sodium. Severe hyponatremia may be the cause of seizure. Causes include psychogenic polydipsia and drugs. Rationale These results are abnormal enough to trigger an urgent phone call to the ward. The hyponatremia is severe enough to cause cerebral edema and seizures, especially if it develops over a short time. It is prudent to reanalyze the sample before reporting it (this may not be stated in a written comment) and to ask for a repeat sample to confirm the results. Serum osmolality would confirm hypoosmolar state (note that serum glucose was not done). In a psychiatric patient, one needs to consider psychogenic polydipsia as the cause of water intoxication as well as antipsychotic drugs, because some of them increase antidiuretic hormone secretion. It could be important to check urine sodium and osmolality to determine the mechanisms for the development of hyponatremia, but not to jump to a diagnosis of syndrome of inappropriate antidiuretic hormone because the latter is a diagnosis of exclusion. Refer to clinical chemistry text. 3 Suggested Increased transferrin saturation may be comment attributable to iron supplements. Although increased transferrin saturation is the first biochemical indication of genetic hemochromatosis, it is unusual to present in such a young person. Suggest repeat iron studies on a fasting sample, take patient off iron supplements, and if transferrin saturation is still increased, then consider genetic hemochromatosis. Rationale Percentage transferrin saturation is the first biochemical index to become abnormal in genetic hemochromatosis. However, it would be unusual to present in a 16-year-old. Most guidelines suggest that a repeat iron study in the fasting state should be performed because transferrin saturation may be increased in the nonfasting state as well as if the patient was on iron therapy. This patient returned to a normal (%) transferrin saturation on repeat testing. Clinically significant organ damage does not generally occur before 30 years in genetic hemochromatosis (Med J Aust 2001;175:418-21). Table 4. Number of participants who responded for each case report and number of key-phrase classifications used in the summary report. Participants, n (%) no. Official Unofficial 1 48 (50) 49 (51) 2 51 (52) 48 (48) 3 39 (47) 44 (53) 4 56 (53) 49 (47) 5 43 (58) 31 (42) 6 57 (59) 39 (41) 7 30 (41) 43 (59) 8 43 (54) 36 (46) 9 42 (52) 38 (48) 10 39 (53) 34 (47) Key phrases, n no. Preferred Less relevant Unacceptable 1 8 35 12 2 14 34 9 3 7 28 17 4 8 12 32 5 5 23 10 6 2 9 18 7 5 33 21 8 13 9 17 9 7 42 7 10 10 46 15 Table 5. Total usage of key phrases for the three categories of classification for each case, broken down by official and unofficial participants. Key phrases, n Preferred Less relevant Case Official Unofficial Official Unofficial 1 101 99 92 93 2 130 107 52 61 3 51 48 84 88 4 133 104 36 31 5 27 26 95 46 6 57 39 30 23 7 28 41 51 77 8 42 32 22 31 9 86 61 62 74 10 94 66 82 91 Total 749 623 606 615 Key phrases, n Unacceptable Case Official Unofficial 1 10 9 2 6 3 3 12 27 4 41 23 5 20 13 6 61 34 7 15 32 8 70 48 9 6 7 10 10 19 Total 251 215
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Laboratory Management|
|Author:||Lim, Ee Mun; Sikaris, Ken A.; Gill, Janice; Calleja, John; Hickman, Peter E.; Beilby, John; Vasikara|
|Date:||Mar 1, 2004|
|Previous Article:||Potential clinical utility of a new IRMA for parathyroid hormone in postmenopausal patients with primary hyperparathyroidism.|
|Next Article:||Tandem mass spectrometry for the direct assay of enzymes in dried blood spots: application to newborn screening for Krabbe disease.|