Performance Characteristics of Urinary Tract Cytology: Observations From the College of American Pathologists Interlaboratory Comparison Program in Nongynecologic Cytopathology.
There is a wide range in the reported sensitivity and specificity for cytologic diagnoses of low-grade and high-grade urothelial carcinoma (HGUC). This wide range of accuracy is primarily due to the bland nuclear features observed in low-grade lesions, making them difficult to distinguish from instrumentation artifact or urolithiasis. (4-10) The sensitivity of urine cytology for detecting low-grade lesions ranges between 26% and 45%, with a specificity of up to 98%. (5, 11) In contrast, cytologic samples from HGUC or carcinoma in situ show a sensitivity of 70% to 90%, with a specificity that approaches 100%.3 As eloquently explained by Koss, (12) urine cytology is highly effective in the detection and diagnosis of in situ and invasive urothelial carcinoma (UC) and other high-grade malignancies. (12) However, it is unrealistic to expect urine cytology to differentiate low-grade papillary neoplasms from reactive/reparative changes or other benign lesions of the urinary tract. Despite these diagnostic challenges, urinary cytology remains a primary tool in the screening and diagnosis of carcinoma of the urinary tract. (13)
Given the importance of urine cytology in the management of UC, the aim of this study was to evaluate the performance characteristics of urine cytology slide challenges from the College of American Pathologists Inter-laboratory Comparison Program in Nongynecologic Cytopathology.
MATERIALS AND METHODS
The members of the College of American Pathologists Cytopathology Committee submit cases used for the College of American Pathologists Interlaboratory Comparison Program in Nongynecologic Cytopathology. Cases are selected for the nongynecologic cytology program by consensus at screening sessions of the Cytopathology Committee. Each case is reviewed by at least 2 members of the Cytopathology Committee, who agree on general and specific interpretations. The members also agree that the slide is a good representation of the entity and that it is technically adequate for evaluation and assessment. Each slide distributed in the program has a specific reference diagnosis assigned to a body site as well as a general category of unsatisfactory, negative/benign, positive/malignant, or suspicious.
The College of American Pathologists Interlaboratory Comparison Program in Nongynecologic Cytopathology consists of 4 educational challenges per year, with each challenge composed of 5 slides mailed to participants. The slides are circulated to participants who voluntarily subscribe to the continuing education program. After review of the slides, participants are able to choose a general category (benign/negative, suspicious, positive, or unsatisfactory) and a specific reference diagnosis, in which the participants are asked to give a more-specific diagnosis, akin to a "real-life" case. After circulation to many laboratories, all slides have a unique performance profile that can be compared against others in the same reference category.
The quality and performance of the slides are continuously monitored. Every 6 months, each slide associated with a slide set is evaluated for stain quality, as assessed by a College of American Pathologists cytotechnologist. Any slide with a discordant rate of 70% or higher after circulation is reviewed by a committee member to determine whether it will continue circulation in the program. Any slide that has been identified by a participant on the result form as technically "unacceptable" is reviewed by at least 2 committee members. Slides are retired most often for technical reasons, such as breakage or fading stain. This quality control system ensures that the slide challenges remain equivalent over time, such that newer slides entering into the program perform similarly to older slides in circulation.
We reviewed the cumulative histories of all participant responses for urine cytology challenges with all the reference diagnoses, including HGUC, squamous cell carcinoma (SCC), or adenocarcinoma (ADC) obtained through the College of American Pathologists SCORES computer system, which circulated in the program from 2000 through 2010. Urine samples included voided/catheterized specimens, bladder washings, and ileal conduit specimens. The preparation type was noted for each challenge. Classic preparations in the program included both conventional tests and cytospin slide preparations. Liquid-based preparation slides consisted of ThinPrep (Hologic, Inc, Bedford, Massachusetts) and SurePath (BD Diagnostics, Franklin Lakes, New Jersey) preparations. Millipore filter preparations (EMD Millipore, Billerica, Massachusetts) have not been circulated in the program.
The analysis examines concordance to both the general and reference diagnoses. Concordance to the general diagnosis is classified as a participant response of positive or suspicious for a positive/suspicious case or a participant response of negative for a negative case. Concordance to the reference diagnosis indicates the participant reference interpretation matched the reference diagnosis. Preparation type was defined, so ThinPrep and SurePath were categorized as the liquid-based group. Non-small cell carcinoma, infection, and Cytomegalovirus challenges were excluded from the study because of few responses.
Two nonlinear mixed models were used to analyze the concordance rates. The 4 main factors included in the models were (1) reference or general diagnosis, (2) preparation type, (3) participant type, and (4) specimen type. The interaction terms between these factors were also included. Both models included a repeated-measures component to model the slide-factor correlation structure. A significance level of .05 was used for these analyses. All statistical analyses were performed using SAS v9.1 software (SAS Inc, Cary, North Carolina).
There were 96 093 responses from 1823 slide challenges representing voided, catheterized, and urinary bladder washings. There were responses from 46 637 pathologists (48.5%), 29 976 cytotechnologists (CTs) (31.2%), and 19 480 laboratories (20.3%).
For the general diagnostic category of positive and negative for malignancy, 88 806 of 96 093 responses (92.4%) were classified correctly. Of the 74 821 responses for the general category positive for malignancy 69 814 (93.3%) were concordant. In the negative for malignancy category, 18 698 of the 21 272 responses (87.9%) were concordant.
The concordance rates for CTs and pathologists were 94.5% (23 553 of 24 924) and 92.3% (36 210 of 39 230), respectively, for the reference diagnosis positive for malignancy. The concordance rates of the reference diagnosis negativefor malignancy were 85.5% (5489 of 6423) and 88.8% (9259 of 10 427) for the CTs and pathologists, respectively. The concordance rate for the reference diagnoses positive for malignancy and negative for malignancy was not statistically significant when stratified by preparation type. The concordance rates for participant, preparation type, and specimen type are presented in Table 1. The interaction between preparation type and general diagnosis was significant. Liquid-based preparation results were significantly better than those of cytospin and conventional preparations in the positive for malignancy general diagnosis; however, no significant differences were observed in the negative for malignancy group. For the specimen type, bladder washing had the best concordance for the positive for malignancy category however it had the lowest performance for the negative for malignancy. The highest concordance in the negative for malignancy category was observed in the voided urine specimens, as seen in Table 1.
The concordance rate for negative for malignancy was 73.9% (2108 of 2852), and concordance for recognizing ileal conduit specimens was 55.8% (2950 of 5291). The other specific reference diagnoses and concordance rates are outlined in Table 2.
HGUC and Polyomavirus
The concordance of pathologists and CTs with the reference diagnosis were 84.1% (29 256 of 34 794) and 82% (18 447 of 22 487), respectively, for HGUC, and 74.2% (1229 of 1656) and 64.6% (725 of 1122), respectively, for Polyomavirus, as presented in Table 3. The participant diagnoses for the reference diagnosis in the Polyomavirus challenges included 71.7% Polyomavirus (2534 of 3535), 6.4% reactive changes (226 of 3535), 6.2% HGUC (220 of 3535), and 3.1% Cytomegalovirus/herpes (109 of 3535), as presented in Table 4.
The participant diagnoses for the UC reference diagnosis included 83.5% HGUC (59 985 of 71 581), 4.1% atypical urothelial cells (2942 of 71 581), 3.1% ADC (2201 of 71 581), 1.1% SCC (776 of 71 581) and 0.9% Polyomavirus (609 of 71 581), among others, as presented in Table 5. The overall concordance rates for HGUC, ADC, and SCC were 83.3% (59 985 of 71 581), 43.9% (732 of 1667), and 49.1% (370 of 756), respectively. The participant responses for malignant reference diagnoses are outlined in the Figure.
Urinary tract cytopathology is historically one of the most difficult fields for the practicing cytopathologist and cytotechnologist. Urinary cytology has been criticized for its low sensitivity. The criticism is levied particularly in detecting low-grade urothelial lesions; lesions that are typically diagnosed by cystoscopic examination and have a less than 10% risk of progression, compared with the close to 60% risk of progression in HGUC. (14, 15) Thus, the primary aim of urinary cytology is to detect HGUC. However, other benign and malignant conditions can be readily identified in urinary tract specimens. Therefore, we sought to identify the performance characteristics of urinary cytology in an interlaboratory comparison program setting.
In this study, the participants performed well for the general diagnostic category of positive or negative for malignancy with a concordance rate of 92.4% (66 140 of 71 581). When stratified by participant type, the concordance rate on the positive diagnosis was slightly higher for CTs than it was for pathologists, whereas the concordance rate of negative cases was slightly higher in pathologists compared with the CTs. This could be attributed to the routine primary screening practice habits of the CT.
The concordance of conventional, cytospin, and liquid-based preparations were 90.5% [30 092 of 33 252], 91.6% [34 018 of 37 127], and 94.9% [24 408 of 25 714], respectively (P < .001). In general, responses for liquid-based preparation challenges were better than those for conventional and cytospin preparation challenges. This was also reported by Laucirica et al, (16) who evaluated the performance characteristics of liquid-based versus conventional preparations. In their study, the concordance rates were better in all 3 preparation types in the positive reference diagnoses compared with the negative reference diagnoses. In addition, results for voided urine and catheterized urine specimens in this study were better than those of bladder washing specimens for the negative reference diagnoses (P < .001). This may be due to misinterpretations of benign groups of urothelial cells as positive, or it may also be due to a test bias because bladder washing is a forced exfoliative specimen performed with an invasive technique--the participants may have a lower threshold for the positive interpretation.
Participants performed better on the specific diagnosis of HGUC with liquid-based preparations (17 529 of 20 495 responses; 85.5%), compared with cytospin (24 877 of 29 544 responses; 84.2%), and conventional preparations (17 579 of 21 513 responses; 81.7%) (P < .002). There was no significant difference among the specimen types (voided urine, bladder barbotage, and catheterized urine; P = .68). Diagnostic difficulties were mostly seen in the sub-classification of nonurothelial malignancies. False-positive diagnoses, specifically as HGUC, were made in 7.3% [509 of 6973], 6.2% [330 of 5323], and 6.2% [220 of 3548] of negative, ileal loop specimens, and Polyomavirus challenges, respectively. This finding may point out that Polyomavirus cases are not as prone to misclassification as previously thought, and, in fact, have a similar discordance rate as the other negative categories, at least in a test setting.
One drawback of our study is that it was different than routine daily practice. Although not a proficiency-testing environment, participants are biased. In this kind of environment, typically, the participants psychologically tend to favor a positive diagnosis over a negative Participant interpretation of malignant reference diagnosis. Abbreviations: ADC, adenocarcinoma; HCUC, high-grade urothelial carcinoma, SCC, squamous cell carcinoma. diagnosis, unlike the unbiased setting of daily practice. This bias may explain the lower concordance rates for the positive for malignancy category. Similarly, over-interpretation of ileal conduit specimens as positive may be due to the test environment effect; however, it could also be due to lack of familiarity with this specimen type. Ileal loop conduit specimens are typically seen in oncology centers after specialized surgery for malignant urinary tract neoplasm.
Another important finding was the difficulties in sub-classifying malignant cases. Although most participants were able to correctly identify UC (83.8%; 59 985 of 71 581), less than one-half of the participants were able to accurately identify SCC (49.1%; 370 of 756) and ADC (43.9%; 732 of 1667). This is most likely attributed to the relative infrequency of those tumors; SCC comprises 2% to 15% and ADC comprises approximately 2% of bladder tumors. In the United States, the incidence of UC with either glandular or squamous differentiation is greater than that of pure SCC or ADC. This knowledge may have affected the participant responses in subclassifying malignant cases.
In summary, CTs performed better than pathologists in the interpretation of positive diagnoses in urinary tract cytology specimens, whereas the pathologists performed better in correctly identifying negative cases. Although all participants performed well in the interlaboratory comparison program when faced with urine cytology challenges, there were problems subclassifying malignant cases. Currently, there is an international initiative to standardize the classification and reporting of urinary tract cytology, namely, the Paris System for Reporting Urinary Cytology. This system will bring standardized categories and defined criteria to the diagnosis of these specimens. As this system gets implemented, becomes established, and is widely used, akin to the Bethesda System for Reporting Gynecologic Cytopathology and the Bethesda System for Reporting Thyroid Cytopathology, we expect that practitioners of urinary tract cytopathology will perform better in test settings but, more important, will be more reliable and accurate in daily practice.
Please Note: Illustration(s) are not available due to copyright restrictions.
(1.) Sanders W. Cancer of the bladder: fragments forming urethral plugs discharged in urine: concentric colloid bodies. Edinb Med J. 1864; 10:273-275.
(2.) Lambl W. Uber harnblasenkrebs: einbeitag zur mikroskopischen diagnostic am krakenbette. Prager Vierteljahreshift. 1865; 49:1-15.
(3.) Papanicolaou GN, Marshall VF. Urine sediment smears as a diagnostic procedure in cancers of the urinary tract. Science. 1945; 101(2629):519-520.
(4.) Kern WH. The cytology of transitional cell carcinoma of the urinary bladder. Acta cytol. 1975; 19(5):420-428.
(5.) Bastacky S, Ibrahim S, Wilczynski SP, Murphy WM. The accuracy of urinary cytology in daily practice. Cancer. 1999; 87(3):118-128.
(6.) Jackson J, Barkan Ga, Kapur U, Wojcik EM. Cytologic and cystoscopic predictors of recurrence and progression in patients with low-grade urothelial carcinoma. Cancer Cytopathol. 2013; 121(7):398-402.
(7.) Garbar C, Mascaux C, Wespes E. Is urinary tract cytology still useful for diagnosis of bladder carcinomas?: a large series of 592 bladder washings using a five-category classification of different cytological diagnoses. Cytopathology 2007;18(2):79-83.
(8.) Sarnacki CT, McCormack LJ, Kiser WS, Hazard JB, McLaughlin TC, Belovich DM. Urinary cytology and the clinical diagnosis of urinary tract malignancy: a clinicopathologic study of 1,400 patients. J Urol. 1971; 106(5): 761-764.
(9.) Renshaw AA, Nappi D, Weinberg DS. Cytology of grade 1 papillary transitional cell carcinoma: a comparison of cytologic, architectural and morphometric criteria in cystoscopically obtained urine. Acta cytol. 1996; 40(4):676-682.
(10.) Esposti PL, Moberger G, Zajicek J. The cytologic diagnosis of transitional cell tumors of the urinary bladder and its histologic basis: a study of 567 cases of urinary-tract disorder including 170 untreated and 182 irradiated bladder tumors. Acta cytol. 1970; 14(3):145-155.
(11.) Raab SS, Lenel JC, Cohen MB. Low grade transitional cell carcinoma of the bladder. Cytologic diagnosis by key features as identified by logistic regression analysis. Cancer. 1994; 74(5):1621-1626.
(12.) Koss LG, Melamed MR. Koss' Diagnostic Cytology and Its Histopathologic Bases. 5th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2006.
(13.) Brown FM. Urine cytology: it is still the gold standard for screening? Urol Clin N Am. 2000; 27(1):25-37.
(14.) Holmang S, Hedelin H, Anderstrom C, Holmberg E, Busch C, Johansson SL. Recurrence and progression in low grade papillary urothelial tumors. J Urol. 1999; 162(3, pt 1):702-707.
(15.) Kwak C, Ku JH, Park JY, Lee E, Lee SE, Lee C. Initial tumor stage and grade as a predictive factor for recurrence in patients with stage T1 grade 3 bladder cancer. J Urol. 2004; 171(1):149-152.
(16.) Laucirica R, Bentz JS, Souers RJ, et al. Do liquid-based preparations of urinary cytology perform differently than classically prepared cases?: observations from the College of American Pathologists Interlaboratory Comparison Program in Nongynecologic Cytology. Arch Pathol Lab Med. 2010; 134(1):19-22.
GCiliz A. Barkan, MD; Rodolfo Laucirica, MD; Manon Auger, MD; Walid Khalbuss, MD; Vijyalakshmi Padmanabhan, MD; Rhona Souers, MS; Ann T. Moriarty, MD
Accepted for publication October 28, 2014.
From the Department of Pathology, Loyola University Medical Center, Maywood, Illinois (Dr Barkan); the Department of Pathology and Laboratory Medicine, University of Texas Medical School, Houston (Dr Laucirica); the Department of Pathology & Immunology, Baylor College of Medicine, Houston (Dr Laucirica); the Department of Pathology, McGill University, Montreal, Quebec, Canada (Dr Auger); the Department of Pathology, University of Pittsburgh, Pittsburgh, Pennsylvania (Dr Khalbuss); the Department of Pathology, Geisel School of Medicine, Hanover, New Hampshire (Dr Padmanabhan); the Department of Pathology, Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire (Dr Padmanabhan); the Biostatistics Department, College of American Pathologists, Northfield, Illinois (Ms Souers);the Department of Anatomic and Clinical Pathology, AmeriPath, Indiana, Indianapolis (Dr Moriarty); and GE Clarient Diagnostic Services, Riyadh, Riyadh Province, Saudi Arabia (Dr Khalbuss).
The authors have no relevant financial interest in the products or companies described in this article.
Presented in part at the 59th Annual Meeting of the American Society of Cytology; November 6, 2011; Baltimore, Maryland.
Reprints: Guliz A. Barkan, MD, Department of Pathology, Loyola University Medical Center, 2160 S First Ave, Maywood, IL 601 53 (email: email@example.com).
Caption: Participant interpretation of malignant reference diagnosis. Abbreviations: ADC, adenocarcinoma; HGUC, high-grade urothelial carcinoma, SCC, squamous cell carcinoma.
Table 1. General Concordance With the Reference Diagnoses Diagnoses Responses, Concordance No. Rate, % Participant type Positive Cytotechnologist, 22 263 94.6 n = 23 553 Pathologist, 33 437 92.3 n = 36 210 Negative Cytotechnologist, 5499 85.6 n = 6423 Pathologist, 9262 88.8 n = 10 427 Preparation type Positive Liquid-based, 20 964 95.9 n = 21 864 Cytospin, 28 404 92.6 n = 30 665 Conventional, 20 439 91.7 n = 22 292 Negative Liquid-based, 3444 89.5 n = 3850 Cytospin, n = 6462 5609 86.8 Conventional, 9653 88.1 n = 10 960 Specimen type Positive Voided urine, 27 691 92.8 n = 29 848 Bladder washing, 17 862 94.6 n = 18 889 Catheterized urine, 24 254 93.0 n = 26 084 Negative Voided urine, 6222 93.6 n = 6648 Bladder washing, 6277 83.6 n = 7507 Catheterized urine, 6207 87.2 n = 7177 Diagnoses P Value Participant type Positive Cytotechnologist, <.001 n = 23 553 Pathologist, n = 36 210 Negative Cytotechnologist, <.001 n = 6423 Pathologist, n = 10 427 Preparation type Positive Liquid-based, Liquid-based:cytospin, <.001 n = 21 864 Cytospin, Liquid-based:conventional, <.001 n = 30 665 Conventional, Cytospin:conventional, .06 n = 22 292 Negative Liquid-based, Liquid-based:cytospin, .05 n = 3850 Cytospin, n = 6462 Liquid-based:conventional, .06 Conventional, Cytospin:conventional, .13 n = 10 960 Specimen type Positive Voided urine, Voided urine:bladder n = 29 848 washing, .002 Bladder washing, Voided urine:catheterized n = 18 889 urine, .82 Catheterized urine, Bladder washing:catheterized n = 26 084 urine, .02 Negative Voided urine, Voided urine:bladder n = 6648 washing, <.001 Bladder washing, Voided urine:catheterized n = 7507 urine, <.001 Catheterized urine, Bladder washing:catheterized n = 7177 urine, .01 Table 2. Specific Reference Diagnosis and Concordance Responses, Concordance Reference Diagnosis No. Rate, % High-grade urothelial carcinoma, n = 59 985 71 581 83.8 Adenocarcinoma, n = 732 1667 43.9 Squamous cell carcinoma, n = 370 756 49.1 Intestinal loop urine, n = 2952 5291 55.8 Polyomavirus, n = 2535 3535 71.7 Negative for malignancy, n = 2108 2852 73.9 Treatment-related changes, n = 469 1293 36.3 Table 3. Interpretation of High-Grade Urothelial Carcinoma Versus Polyomavirus Pathologist Responses, No. Concordance, % High-grade urothelial 29 256 of 34 794 84.1 carcinoma Polyomavirus 1229 of 1656 74.2 Cytotechnologist Responses, No. Concordance, % High-grade urothelial 18 447 of 22 487 82.0 carcinoma Polyomavirus 725 of 1122 64.6 Table 4. Participant Interpretations for the Reference Diagnosis Polyomavirus Response, No. (%), Reference Interpretation n = 3535 Polyomavirus 2534 (71.7) Reactive 226 (6.4) High-grade urothelial carcinoma 220 (6.2) Infection, viral (Cytomegalovirus and herpes) 109 (3.1) Infectious, inflammatory 106 (3.0) Negative, inflammation 177 (5.0) Other 163 (4.6) Table 5. Participant Interpretations for the Reference Diagnosis Urothelial Carcinoma Response, No. (%), Reference Interpretation n = 71 552 High-grade urothelial carcinoma 59 985 (83.3) Atypical urothelial cells 2942 (4.1) Reactive urothelial cells 2273 (3.2) Adenocarcinoma 2201 (3.1) Negative 949 (1.3) Squamous cell carcinoma 776 (1.1) Polyomavirus 609 (0.9) Inflammatory 523 (0.7) Other 1294 (1.8)
|Printer friendly Cite/link Email Feedback|
|Author:||Barkan, GCiliz A.; Laucirica, Rodolfo; Auger, Manon; Khalbuss, Walid; Padmanabhan, Vijyalakshmi; Sou|
|Publication:||Archives of Pathology & Laboratory Medicine|
|Date:||Aug 1, 2015|
|Previous Article:||Brief Updates in Lung Pathology.|
|Next Article:||Significance of MYD88 L265P Mutation Status in the Subclassification of Low-Grade B-Cell Lymphoma/Leukemia.|