Accuracy of physician billing claims for identifying acute respiratory infections in primary care.
Monitoring antibiotic prescribing in primary care is challenging because well-developed measures of antibiotic prescribing are scarce, often inaccurate, and may not reflect real prescribing practices. Studies of antibiotic prescribing in primary care have relied on physician self-reported prescribing (Mangione-Smith et al. 1999; Nash et al. 2002; Steinman, Landefeld, and Gonzales 2003), chart review or audit (Hueston, Jenkins, and Mainous 2000; Hutchinson et al. 2001; Mangione-Smith et al. 2002), or prescription claims (Mainous, Hueston, and Clark 1996; Majeed and Moser 1999; Wang et al. 1999; Steinke et al. 2000; Gill and Roalfe 2001; Kozyrskyj et al. 2004; Cadieux et al. 2007). Self-reported antibiotic prescribing was shown to underestimate actual antibiotic prescribing by about 30 percent (Mangione-Smith et al. 2002), and the cost of chart review is too high for wide-scale use. Prescription claims data avoid self-report bias, do not require additional data collection, and because they involve financial transactions, they are carefully audited by payers and have been found to be highly accurate (Tamblyn et al. 1995). Owing to these advantages, prescription claims are now used routinely to monitor antibiotic prescribing for respiratory infections in primary care (Mainous, Hueston, and Clark 1996; Majeed and Moser 1999; Wang et al. 1999; Steinke et al. 2000; Gill and Roalfe 2001; McCaig, Besser, and Hughes 2002; Kozyrskyj et al. 2004; Cadieux et al. 2007).
However, an important limitation of using prescription claims to monitor antibiotic prescribing is that treatment indication is not recorded on prescription claims. Treatment indication is required to determine the appropriateness of antibiotic prescribing; therefore, it must be inferred from other sources of information, such as physician billing claims for patient visits. If prescription claims are to be used to monitor antibiotic prescribing, then the accuracy of using diagnostic information in physician billing claims to infer the indication for antibiotic treatment needs to be assessed.
Two previous studies have assessed the accuracy of physician billing claims for identifying respiratory infection diagnoses, and both have shown promising results. The first was a study of administrative claims data from seven health insurance providers in Colorado, and it found that 79 percent of bronchitis diagnoses and 83 percent of pharyngitis diagnoses in administrative claims had a corresponding diagnosis in the written medical record (PPV; Maselli and Gonzales 2001). However, this study did not investigate what proportion of bronchitis and pharyngitis diagnoses documented in patient medical records was accurately documented in physician billing claims (sensitivity and specificity). The second study assessed the accuracy of Research Patient Data Repository (RPDR) claims from nine primary care clinics in the Brigham and Women's Primary Care Practice-Based Research Network in Boston and reported that 86 percent of respiratory infection diagnoses in RPDR claims had a corresponding diagnosis in the electronic health record (Linder et al. 2006). However, sensitivity and specificity estimates were not corrected for the verification bias introduced by over-sampling claims with a diagnosis of respiratory infection relative to claims without such a diagnosis (i.e., the study design inflated the prevalence of respiratory infection in the sample, relative to the true population prevalence; Begg and Greenes 1983; Irwig et al. 1994).
The objective of this study was to assess the accuracy of physician billing claims for identifying episodes of acute respiratory infection in primary care. In particular, we sought to estimate the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of physician billing claims.
Study Design and Population
A validation study was carried out to assess the sensitivity, specificity, PPV, and NPV of physician billing claims for identifying episodes of respiratory infection, as compared with the patient medical record. The study population comprised 34 Montreal region family physicians and 17,002 of their patients who were participating in the MOXXI electronic medication management trial (Tamblyn et al. 2006) in 2002-2005. All patients participating in the MOXXI trial had previously consented to share their medical records and provincial health insurance (RAMQ) data with researchers. These data were available for a period starting 1 year before patient enrollment date (2001 or later) until 2005, when the present study was conducted. From the available physician billing claims, we identified those with a diagnostic code (International Classification of Diseases, 9th Revision, or ICD-9) for laryngitis/tracheitis (464), common cold (460), influenza (480, 487), acute unspecified upper respiratory infection (465), pharyngitis/tonsillitis (462, 463, 034), otitis media (381, 382), sinusitis (461), acute bronchitis (466), or bacterial pneumonia (481-486); all decimal place suffixes of these ICD-9 codes were included. We purposefully selected 10 physicians who had been enrolled in the MOXXI trial for at least 2 years and had the most MOXXI-consenting patients (and therefore also had the most physician billing claims available for research purposes), and requested their consent.
Sample of Physician Billing Claims
Among the 10 physicians selected, we identified all MOXXI-consenting patients who had at least one physician billing claim with a diagnosis of acute respiratory infection during the study period, and randomly sampled 635 of those patients. We also identified all patients without any physician billing claim with a diagnosis of acute respiratory infection during the study period, and randomly sampled 94 of those patients. To improve the efficiency of data collection, we over-sampled patients with at least one diagnosis of acute respiratory infection, relative to those with no diagnosis of acute respiratory infection (Begg and Greenes 1983; Irwig et al. 1994). For each of the 729 patients sampled, we identified all physician billing claims generated during the study period (i.e., from 2001 or later, depending on the enrollment date, until 2005) and validated each one against the paper-based patient medical record. Information available in the physician billing claims included the patient's lifelong RAMQ personal identifier, physician license number, visit date, and ICD-9 diagnostic code.
Medical Record Abstraction
Once the physician billing claims had been sampled, a list of sampled patients' names and RAMQ personal identifiers was generated and sent to each consenting physician's office. The selected patients' paper-based medical records were retrieved by the office staff and reviewed by one of the authors (G. C.). For each sampled physician billing claim, the corresponding visit was identified in the medical record, the date of the visit was recorded, and the presence or absence of all acute respiratory infections under study was ascertained from the clinical notes. At the time of the medical record abstraction, the reviewer was blinded to the ICD-9 code in the corresponding physician billing claim. Information abstracted from patient medical records was entered directly in an electronic, structured chart abstraction form and stored in an MS Access database, which had been prepopulated with participating physicians' license numbers, sampled patients' RAMQ identifiers, and visit dates according to physician billing claims. Intra-rater reliability was measured on 25 randomly selected visits that were reviewed a second time, and the percent agreement between the first and second review was 100 percent.
Linkage of Physician Billing Claims and Data Abstracted from Medical Records
Data retrieved from patient medical records were linked directly to physician billing claims using the patient lifelong RAMQ personal identifier, physician license number, and visit date [+ or -] 1 day.
For each type of acute respiratory infection under study, a 2 x 2 table of diagnoses abstracted from patient medical records versus diagnoses obtained from the corresponding physician billing claims was generated using SAS statistical software (Version 9.3, SAS Institute Inc., Cary, NC). Information retrieved from the patient medical record was treated as a gold standard. The prevalence, PPV, and NPV of physician billing claims for identifying acute respiratory infections were estimated. Sensitivity and specificity estimates were corrected for the over-sampling of claims with a diagnosis of acute respiratory infection relative to claims without such a diagnosis (Begg and Greenes 1983) using MS Excel 2003 (Version 5.1). To investigate between-physician variation in physician billing claim diagnosis accuracy, these analyses were repeated for each physician individually, combining all nine types of acute respiratory infection under study (because each physician contributed too few of each type of acute respiratory infection to analyze each type individually).
Because we sampled several claims (and medical record visits) per patient, we conducted a sensitivity analysis to assess the effect of clustering of claims within patients on our estimates of sensitivity, specificity, PPV, and NPV. We did this by generating 100 random samples of one claim per patient (n = 729 claims) from our total sample of 3,526 claims, and averaging the estimates of sensitivity, specificity, PPV, and NPV over all 100 random samples, which is similar to bootstrapping methodology (Efron and Tibshirani 1994).
Ten physicians participating in the MOXXI trial were purposefully selected for this study, and nine agreed to participate. Among these nine physicians' patients enrolled in the MOXXI trial, we randomly selected 635 patients who had at least one claim with a diagnosis of acute respiratory infection, and 94 patients without any claims with a diagnosis of acute respiratory infection. These 729 patients made 3,526 visits to their respective MOXXI physicians during the study period (duration of 1-5 years, depending on the date of enrollment), for an average of 4.8 visits per patient. The medical records of all 729 sampled patients were abstracted, and written documentation for each of the 3,526 visits identified from physician billing claims was found in the corresponding patient's medical record. In all, 1,173 (33.3 percent) of sampled claims were positive for respiratory infection (Figure 1). Sixty-six percent of sampled patients were women, and the mean age of sampled patients was 47.6 (SD 21.0, range < 1-90 years). The characteristics of patients enrolled in the MOXXI trial, as compared with those of the general population, have been discussed previously (Bartlett et al. 2005).
The agreement between the diagnosis in the medical record and the ICD-9 code in the physician billing claim is shown in Table 1, where shaded areas indicate concordant diagnoses. For example, there were 63 physician billing claims with a diagnosis of laryngitis, and for all 63 claims, a diagnosis of laryngitis was also documented in the medical record at the corresponding date; however, an additional 16 diagnoses of laryngitis were documented in medical records that were not documented in physician billing claims. The overall percent agreement for the presence of any acute respiratory infection was 72.5 percent, which is the sum of all diagnoses of respiratory infection present in the physician billing claim and the corresponding medical record (969) divided by the sum of all diagnoses of respiratory infection documented in medical records (1,337).
The proportion of physician billing claims with a diagnosis of acute respiratory infection confirmed in the patient medical record (PPV) was 0.93, 95 percent CI (0.91, 0.94), for all acute respiratory infection combined, and 0.84, 95 percent CI (0.81, 0.88), for respiratory infections of likely viral etiology (Table 2). The PPV for acute respiratory infections of potentially bacterial etiology was 0.89, 95 percent CI (0.87, 0.92), and ranged from 0.72, 95 percent CI (0.67, 0.78), for acute bronchitis to 0.91, 95 percent CI (0.85, 0.97), for bacterial pneumonia. Sensitivity of physician billing claims for all acute respiratory infections combined was 0.49, 95 percent CI (0.45, 0.53). With the exception of influenza, sensitivity was markedly lower for viral respiratory infections than for bacterial ones. Specificity was 0.99 or higher for all types of acute respiratory infection studied.
[FIGURE 1 OMITTED]
The prevalence of acute respiratory infection diagnoses in physician billing claims varied between physicians from 19.5 to 111.4 per 1,000 claims (Table 3). Sensitivity and PPV varied between physicians from 1.00, 95 percent CI (1.00, 1.00), to 0.19, 95 percent CI (0.06, 0.47), and from 0.98, 95 percent CI (0.96, 1.00), to 0.70, 95 percent CI (0.53, 0.87), respectively. The accuracy of physician billing claims for identifying acute respiratory infections did not appear to be higher among physicians who diagnosed more acute respiratory infections.
Our sensitivity analysis using only one claim per patient yielded estimates for sensitivity (0.55, 95 percent CI 0.45, 0.64), specificity (0.99, 95 percent CI 0.99, 1.00), PPV (0.93, 95 percent CI 0.90, 0.96), and NPV (0.94, 95 percent CI 0.91, 0.97) that were similar to the estimates obtained when all visits were used. The confidence intervals from the sensitivity analysis are wider because the sample size is smaller: 729 claims (one per patient) were used instead of all 3,526. This shows that the effect of within-patient clustering of claims on sensitivity, specificity, PPV, and NPV estimates is small.
The PPV of physician billing claims was high for all types of acute respiratory infection studied. Our study was the first to estimate the prevalence, sensitivity, and specificity of physician billing claims for identifying chart-documented acute respiratory infections in primary care. For all but one type of acute respiratory infection investigated, our sensitivity estimates were below 0.50. Our study was also the first to look at between-physician variation in physician billing claim diagnosis accuracy. We found that prevalence of respiratory infections in physician billing claims varied widely between primary care physicians. We also observed large unexplained between-physician variation in sensitivity and PPV of physician billing claims for identifying acute respiratory infections.
If physician billing claims had many false-positive diagnoses of respiratory infection, they would not be a useful data source for monitoring antibiotic prescribing. Therefore, a high PPV, or a high likelihood that diagnoses of respiratory infection in physician billing claims are also present in the corresponding patient medical record, provides support for using health administrative data for monitoring antibiotic prescribing. For most of the acute respiratory infection diagnoses investigated, our estimates of PPV were similar to those previously reported in the literature (Maselli and Gonzales 2001; Linder et al. 2006). However, our PPV estimate for influenza (0.66, 95 percent CI 0.58, 0.74) was much higher than the 0.20 reported by Linder et al. (2006), but the latter was aberrantly low as compared with other PPV estimates in the same study.
Previous studies have emphasized the high PPV of health administrative data for identifying episodes of respiratory infection, but have overlooked the importance of sensitivity (Maselli and Gonzales 2001; Linder et al. 2006). A high sensitivity is desirable because it suggests that the data capture a majority of visits for respiratory infections. A low sensitivity is problematic because it suggests that several visits for respiratory infections are not documented in health administrative data. Nondocumentation of visits for respiratory infections may or may not be associated with antibiotic prescribing, which may result in bias when using health administrative data to monitor antibiotic prescribing.
Our study estimated the sensitivity of physician billing claims for identifying acute respiratory infections. Our sensitivity estimates were below 0.50 for all types of acute respiratory infection studied except acute bronchitis, which raises concerns about the potential for bias. Whereas one previous study estimated the sensitivity of claims for identifying respiratory infections (Linder et al. 2006), the authors did not correct their sensitivity estimate for the verification bias introduced by over-sampling claims with a diagnosis of acute respiratory infection relative to claims without such a diagnosis (Begg and Greenes 1983; Irwig et al. 1994); consequently, they greatly overestimated sensitivity. For example, if we had not corrected our estimates for verification bias, our estimate of the sensitivity of physician billing claims for identifying laryngitis would have been 0.80, as compared with the corrected sensitivity estimate of 0.20.
We were first to investigate between-physician variation in physician billing claim diagnosis accuracy for acute respiratory infections. We found almost a sixfold variation between physicians in the prevalence of acute respiratory infections. We observed similar between-physician variation in the sensitivity and PPV of physician billing claims for identifying acute respiratory infections. We expected that claims submitted by physicians who diagnosed more acute respiratory infections would be more accurate for identifying acute respiratory infections, but we found that neither frequency nor prevalence of acute respiratory infections seemed to be related to physician billing claim diagnosis accuracy. This finding suggests that other factors are likely responsible for the observed between-physician variation in physician billing claim diagnosis accuracy.
A limitation of our study is that medical records may not represent a true gold standard for identifying acute respiratory infections diagnosed in primary care. The use of a single rater was also a limitation of our study, and systematic misclassification of acute respiratory infection diagnoses may have occurred as a result. Another limitation of our study was its small convenience sample of primary care physicians. Whereas physicians participating in the MOXXI trial are generally similar to other eligible physicians in the Montreal region, they tend to be younger than MOXXI nonparticipants. If physician billing claim diagnosis accuracy is related to physician age or practice experience, then our study results may not be applicable to older or more experienced physicians. Also, the MOXXI trial involves physicians practicing in urban and suburban areas, and our results may not be generalizable to physicians practicing in rural areas. Furthermore, patients enrolled in the MOXXI trial tend to differ from nonparticipating patients in that they are generally older, with more complex health status, and have more visits to the MOXXI physician (Bartlett et al. 2005). Younger, healthier patients may be underrepresented in our study sample. Future research should involve a large random sample of primary care physician from both urban and rural areas, and a stratified random sample of patients from each physician's practice population.
Because physician billing claims and prescription claims are now routinely used to monitor antibiotic prescribing for acute respiratory infections in primary care (Mainous, Hueston, and Clark 1996; Majeed and Moser 1999; Wang et al. 1999; Steinke et al. 2000; Gill and Roalfe 2001; Kozyrskyj et al. 2004; Cadieux et al. 2007), it is important for future research to determine whether half of all acute respiratory infections diagnoses are missing from physician billing claims at random, or whether bias is present. If bias is present, future research should also focus on identifying determinants of physician billing claim diagnosis accuracy, so that appropriate corrections for the resulting bias can be developed and applied when physician billing claims are used to infer treatment indication for antibiotic prescribing. As suggested by the large between-physician variation observed in this study, physician characteristics may be associated with physician billing claim diagnosis accuracy. The effect of physician characteristics, as well as patient, encounter, practice, and billing characteristics, on physician billing claim diagnosis accuracy should be assessed.
Joint Acknowledgement/Disclosure Statement: Operational funding for this study was provided by the Canadian Institutes of Health Research. We would like to thank the MOXXI field coordinator, Rola El Halabieh, for her help in contacting physicians to participate in this study, and the MOXXIIT team (Jimmy Fragos and Marc-Andre Larochelle) for their help with database management.
Arroll, B. 2005. "Antibiotics for Upper Respiratory Tract Infections: An Overview of Cochrane Reviews." Respiratory Medicine 99 (3): 255-61.
Austin, D.J., K. G. Kristinsson, and R. M. Anderson. 1999. "The Relationship between the Volume of Antimicrobial Consumption in Human Communities and the Frequency of Resistance." Proceedings of the National Academy of Sciences of the United States of America 96 (3): 1152-6.
Bartlett, G., R. Tamblyn, Y. Kawasumi, L. Poissant, and L. Taylor. 2005. "Non-Participation Bias in Health Services Research Using Data from an Integrated Electronic Prescribing Project: The Role of Informed Consent." Acta Bioethica 11 (2): 145-59.
Begg, C. B., and R. A. Greenes. 1983. "Assessment of Diagnostic-Tests When Disease Verification Is Subject to Selection Bias." Biometrics 39 (1): 207-15.
Cadieux, G., R. Tamblyn, D. Dauphinee, and M. Libman. 2007. "Predictors of Inappropriate Antibiotic Prescribing among Primary Care Physicians." Canadian Medical Association Journal 177 (8): 877-83.
Classen, D. C., S. L. Pestotnik, R. S. Evans, and J. P. Burke. 1991. "Computerized Surveillance of Adverse Drug Events in Hospital Patients." Journal of the American Medical Association 266 (20): 2847-51.
Cronk, G. A., D. E. Naumann, K. Mcdermott, P. Menter, and M. B. Swift. 1954. "A Controlled Study of the Effect of Oral Penicillin-G in the Treatment of Non-Specific Upper Respiratory Infections." American Journal of Medicine 16 (6): 804-9.
Efron, B., and R.J. Tibshirani. 1994. "An Introduction to the Bootstrap." Monographs on Statistics and Applied Probability. Boca Raton, FL: Chapman & Hall.
Gadomski, A. M. 1993. "Potential Interventions for Preventing Pneumonia among Young-Children--Lack of Effect of Antibiotic-Treatment for Upper Respiratory-Infections." Pediatric Infectious Disease Journal 12 (2): 115-20.
Gill, P. S., and A. Roalfe. 2001. "Antibiotic Prescribing by Single Handed General Practitioners: Secondary Analysis of Data." Journal of Clinical Pharmacy and Therapeutics 26 (3): 195-9.
Heikkinen, T., O. Runskanen, T. Ziegler, M. Wafts, and H. Puhakka. 1995. "Short-Term Use of Amoxicillin-Clavulanate during Upper Respiratory-Tract Infection for Prevention of Acute Otitis-Media." Journal of Pediatrics 126 (2): 313-6.
Hoaglund, R.J., E. N. Dietz, P. W. Myers, and H. C. Cosand. 1950. "Aureomycin in the Treatment of the Common Cold." New England Journal of Medicine 243: 773-5.
Howie, J. G., and G. A. Clark. 1970. "Double-Blind Trial of Early Demethylchlortetracycline in Minor Respiratory Illness in General Practice." Lancet 2 (7683): 1099-1102.
Hueston, W.J., R. Jenkins, and A. G. III Mainous. 2000. "Does Drug Treatment of Patients with Acute Bronchitis Reduce Additional Care Seeking? Evidence from the Practice Partner Research Network." Archives of Family Medicine 9 (10): 997-1001.
Hutchinson, J. M., S. Jelinski, D. Hefferton, G. Desaulniers, and P. S. Parfrey. 2001. "Role of Diagnostic Labeling in Antibiotic Prescription." Canadian Family Physician 47: 1217-24.
Irwig, L., P. P. Glasziou, G. Berry, C. Chock, P. Mock, and J. M. Simpson. 1994. "Efficient Study Designs to Assess the Accuracy of Screening-Tests." American Journal of Epidemiology 140 (8): 759-69.
Kaiser, L., D. Lew, B. Hirschel, R. Auckenthaler, A. Morabia, A. Heald, P. Benedict, F. Terrier, W. Wunderli, L. Matter, D. Germann, J. Voegeli, and H. Stalder. 1996. "Effects of Antibiotic Treatment in the Subset of Common-Cold Patients Who Have Bacteria in Nasopharyngeal Secretions." Lancet 47 (9014): 1507-10.
Kozyrskyj, A. L., M. E. Dalai, D. G. Chateau, G. B. Mazowita, T. P. Klassen, and B.J. Law. 2004. "Evidence-Based Prescribing of Antibiotics for Children: Role of Socioeconomic Status and Physician Characteristics." Canadian Medical Association Journal 171 (2): 139-45.
Linder, J. A., D. W. Bates, D. H. Williams, M. A. Connolly, and B. Middleton. 2006. "Acute Infections in Primary Care: Accuracy of Electronic Diagnoses and Electronic Antibiotic Prescribing." Journal of the American Medical Informatics Association 13 (1): 61-6.
Little, P., C. Gould, I. Williamson, G. Warner, M. Gantley, and A. L. Kinmonth. 1997. "Reattendance and Complications in a Randomised Trial of Prescribing Strategies for Sore Throat: The Medicalising Effect of Prescribing Antibiotics." British Medical Journal 315 (7104): 350-2.
Mainous, A. G. III, W.J. Hueston, and J. R. Clark. 1996. "Antibiotics and Upper Respiratory Infection: Do Some Folks Think There Is a Cure for the Common Cold." Journal of Family Practice 42 (4): 357-61.
Majeed, A., and K. Moser. 1999. "Age- and Sex-Specific Antibiotic Prescribing Patterns in General Practice in England and Wales in 1996." British Journal of General Practice 49 (446): 735-6.
Mangione-Smith, R., M. N. Elliott, L. McDonald, and E. A. McGlynn. 2002. "An Observational Study of Antibiotic Prescribing Behavior and the Hawthorne Effect." Health Services Research 37 (6): 1603-23.
Mangione-Smith, R., E. A. McGlynn, M. N. Elliott, P. Krogstad, and R. H. Brook. 1999. "The Relationship between Perceived Parental Expectations and Pediatrician Antimicrobial Prescribing Behavior." Pediatrics 103 (4 Pt 1): 711-8.
Maselli, J. H., and R. Gonzales. 2001. "Measuring Antibiotic Prescribing Practices among Ambulatory Physicians: Accuracy of Administrative Claims Data." Journal of Clinical Epidemiology 54 (2): 196-201.
McCaig, L. F., R. E. Besser, and J. M. Hughes. 2002. "Trends in Antimicrobial Prescribing Rates for Children and Adolescents." Journal of the American Medical Association 287 (23): 3096-102.
McCaig, L. F., and J. M. Hughes. 1995. "Trends in Antimicrobial Drug Prescribing among Office-Based Physicians in the United-States." Journal of the American Medical Association 273 (3): 214-9.
Nash, D. R., J. Harman, E. R. Wald, and K. J. Kelleher. 2002. "Antibiotic Prescribing by Primary Care Physicians for Children with Upper Respiratory Tract Infections." Archives of Pediatrics and Adolescent Medicine 156 (11): 1114-9.
Pihlajamaki, M., P. Kotilainen, T. Kaurila, T. Klaukka, E. Palva, P. Huovinen, A. Kostiala-Thompson, R. Renkonen, A. Muotiala, M. Vaara, A. Sivonen, H. Jousimies-Somer, K. Laitinen, J. Korpela, S. S. Kaukoranta-Tolvanen, H. Hiekkaniemi, P. Hirvonen, A. Nissinen, P. Ruuska, E. Ahonen, H. Jagerroos, S. Raisanen, U. Larinkari, B. Forsblom, M. L. Katila, U. Farkkainen, H. Sarkkinen, P. Karpanoja, M. Kauppinen, S. Paltemaa, P. Karkkainen, S. Silvennoinen-Kassinen, K. Lantto, M. Koskela, I. Pietarinen, M. L. Klossner, S. Pajarre, S. Oinonen, V. Ratia, P. Gronroos, R. Vuento, O. Liimatainen, M. R. Siro, E. Eerola, R. Maninen, O. Meurman, S. Luukkonen, P. Stranden, and K. Lager. 2001. "Macrolide-Resistant Streptococcus pneumoniae and Use of Antimicrobial Agents." Clinical Infectious Diseases 33 (4): 483-8.
Seppala, H., T. Klaukka, J. Vuopio-Varkila, A. Muotiala, H. Helenius, K. Lager, and P. Huovinen. 1997. "The Effect of Changes in the Consumption of Macrolide Antibiotics on Erythromycin Resistance in Group A Streptococci in Finland. Finnish Study Group for Anfimicrobial Resistance." New England Journal of Medicine 337 (7): 441-6.
Steinke, D. T., D.J. Bain, T. M. MacDonald, and P. G. Davey. 2000. "Practice Factors That Influence Antibiotic Prescribing in General Practice in Tayside." Journal of Antimicrobial Chemotherapy 46 (3): 509-12.
Steinman, M. A., C. S. Landefeld, and R. Gonzales. 2003. "Predictors of Broad-Spectrum Antibiotic Prescribing for Acute Respiratory Tract Infections in Adult Primary Care." Journal of the American Medical Association 289 (6): 719-25.
Stott, N. C., and R. R. West. 1976. "Randomised Controlled Trial of Antibiotics in Patients with Cough and Punflent Sputum." British Medical Journal 2 (6035): 556-9.
Tamblyn, R., A. Huang, Y. Kawasumi, G. Bartlett, R. Grad, A. Jacques, M. Dawes, M. Abrahamowicz, R. Perreault, L. Taylor, N. Winslade, L. Poissant, A. Pinsonnikault, and A. Pinsonneault. 2006. "The Development and Evaluation of an Integrated Electronic Prescribing and Drug Management System for Primary Care." Journal of the American Medical Informatics Association 13 (2): 148-59.
Tamblyn, R., G. Lavoie, L. Petrella, and J. Monette. 1995. "The Use of Prescription Claims Databases in Pharmacoepidemiological Research: The Accuracy and Comprehensiveness of the Prescription Claims Database in Quebec." Journal of Clinical Epidemiology 48 (8): 999-1009.
Verheij, T.J., J. Hermans, and J. D. Mulder. 1994. "Effects of Doxycycline in Patients with Acute Cough and Purulent Sputum: A Double Blind Placebo Controlled Trial." British Journal of General Practice 44 (386): 400-4.
Wang, E. E., T. R. Einarson, J. D. Kellner, and J. M. Conly. 1999. "Antibiotic Prescribing for Canadian Preschool Children: Evidence of Overprescribing for Viral Respiratory Infections." Clinical Infectious Diseases 29 (1): 155-60.
The following supplementary material for this article is available online: Appendix SA1: Verification Bias Correction.
Appendix SA2: Author Matrix.
This material is available as part of the online article from http:// www.blackwell-synergy.com/doi/abs/10.1111/j.1475-6773.2008.00873.x (this link will take you to the article abstract).
Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.
Address correspondence to Genevieve Cadieux, M.Sc., M.D.-Ph.D. Candidate, Clinical and Health Informatics Research Group, McGill University, 1140 Pine Avenue West, Montreal QC H3A 1A3, Canada; e-mail: firstname.lastname@example.org. Robyn Tamblyn, Ph.D., Professor, is with the Department of Medicine and Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Canada.
Table 1: Concordance of Diagnoses in Physician Billing Claims and Patient Medical Records Number of Visits in Patient Medical Records Unspecified Number of Physician Common Acute Billing Claims Laryngitis Cold Influenza URI Laryngitis 63 0 0 0 Common cold 1 66 1 0 Influenza 2 1 85 7 Unspecified acute URI 0 1 1 154 pharyngitis 2 1 1 2 Otitis media 0 0 0 5 Sinusitis 1 1 1 0 Acute bronchitis 6 3 1 11 pneumonia 0 0 0 0 No respiratory 4 29 5 28 infection Total 79 102 95 207 Uncorrected 0.80 0.65 0.89 0.74 sensitivity * Number of Visits in Patient Medical Records Number of Physician Otitis Acute Billing Claims Pharyngitis Media Sinusitis Bronchitis Laryngitis 0 0 0 0 Common cold 2 0 0 0 Influenza 1 1 6 6 Unspecified acute URI 3 0 2 4 pharyngitis 93 1 1 0 Otitis media 1 116 1 0 Sinusitis 0 0 138 3 Acute bronchitis 3 3 15 173 pneumonia 0 0 2 2 No respiratory 12 28 19 28 infection Total 115 149 184 216 Uncorrected 0.81 0.78 0.75 0.80 sensitivity * Number of Visits in Patient Medical Records Number of Physician No Respiratory Billing Claims Pneumonia Infection Total Laryngitis 0 0 63 Common cold 0 4 74 Influenza 5 15 129 Unspecified acute URI 0 22 187 pharyngitis 0 6 1117 Otitis media 1 8 132 Sinusitis 0 9 153 Acute bronchitis 9 15 239 pneumonia 81 4 89 No respiratory 11 2,189 2,353 infection Total 107 2,272 3,526 Uncorrected 0.76 sensitivity * Note: Shaded numbers indicate concordance. * The uncorrected sensitivity estimate is inflated due to the purposeful over-sampling of physician billing claims with a diagnosis of acute respiratory infection relative to claims without such a diagnosis (Begg and Greenes 1983; Irwig et al. 1994). URI, upper respiratory infection. Table 2: Sensitivity, Specificity, and Positive and Negative Predictive Values of the RAMQ Physician Billing Claims for Identifying Episodes of Acute Respiratory Infection RAMQ Physician Billing Claims Prevalenceper Sensitivity * 1,000 (95% CI) (95% CI) All respiratory infections 67.3 (65.6, 69.0) 0.49 (0.45, 0.53) All likely viral respiratory 16.4 (15.6, 17.3) 0.30 (0.26, 0.34) infections Laryngitis/tracheitis 1.2 (0.9, 1.4) 0.20 (0.13, 0.30) Common cold 1.4 (1.2, 1.7) 0.11 (0.08, 0.15) Influenza 3.6 (3.2, 4.0) 0.45 (0.30, 0.60) Unspecified acute URI 10.2 (9.6, 10.9) 0.35 (0.29, 0.42) All potentially bacterial 50.9 (49.4, 52.3) 0.51 (0.47, 0.56) respiratory infections Pharyngitis/tonsillitis 5.3 (4.9, 5.8) 0.42 (0.32, 0.53) Otitis media 8.6 (7.9, 9.2) 0.44 (0.35, 0.53) Sinusitis 12.5 (11.7, 13.2) 0.46 (0.38, 0.53) Acute bronchitis 19.5 (18.6, 20.4) 0.52 (0.46, 0.59) Pneumonia 5.0 (4.5, 5.5) 0.38 (0.28, 0.47) RAMQ Physician Billing Claims Specificity * PP V ([dagger]) (95% CI) (95% CI) All respiratory infections 0.99 (0.99, 1.00) 0.93 (0.91, 0.94) All likely viral respiratory 1.00 (1.00, 1.00) 0.84 (0.81, 0.88) infections Laryngitis/tracheitis 1.00 (1.00, 1.00) 1.00 (1.00, 1.00) Common cold 1.00 (1.00, 1.00) 0.89 (0.82, 0.96) Influenza 1.00 (1.00, 1.00) 0.66 (0.58, 0.74) Unspecified acute URI 1.00 (1.00, 1.00) 0.82 (0.77, 0.88) All potentially bacterial 0.99 (0.99, 0.99) 0.89 (0.87, 0.92) respiratory infections Pharyngitis/tonsillitis 1.00 (1.00, 1.00) 0.87 (0.81, 0.93) Otitis media 1.00 (1.00, 1.00) 0.88 (0.82, 0.93) Sinusitis 1.00 (1.00, 1.00) 0.90 (0.85, 0.95) Acute bronchitis 0.99 (0.99, 0.99) 0.72 (0.67, 0.78) Pneumonia 1.00 (1.00, 1.00) 0.91 (0.85, 0.97) RAMQ Physician Billing Claims NPV ([double dagger]) (95% CI) All respiratory infections 0.93 (0.92, 0.94) All likely viral respiratory 0.97 (0.96, 0.97) infections Laryngitis/tracheitis 1.00 (0.99, 1.00) Common cold 0.99 (0.99, 0.99) Influenza 1.00 (1.00, 1.00) Unspecified acute URI 0.98 (0.98, 0.99) All potentially bacterial 0.95 (0.95, 0.96) respiratory infections Pharyngitis/tonsillitis 0.99 (0.99, 1.00) Otitis media 0.99 (0.99, 0.99) Sinusitis 0.99 (0.98, 0.99) Acute bronchitis 0.99 (0.98, 0.99) Pneumonia 0.99 (0.99, 1.00) * Adjusted for over-sampling of physician billing claims with a diagnosis of acute respiratory infection relative to claims without such a diagnosis (Begg and Greenes 1983; Irwig et al. 1994). ([dagger]) Positive predictive value. ([double dagger]) Negative predictive value. URI, upper respiratory infection. Table 3: Sensitivity, Specificity, and Positive and Negative Predictive Values of the RAMQ Physician Billing Claims Database for Identifying All Acute Respiratory Infections Combined, by Study Physician RAMQ Physician Billing Claims Number of Claims Total with a Diagnosis Number Prevalence per Study Physician of Acute Rl of Claims 1,000 (95% CI) 1 1,324 16,264 81.4 (77.1, 85.6) 2 2,041 18,319 111.4 (l06.9, 116.0) 3 522 4,815 108.4 (99.6, 117.2) 4 356 9,614 37.0 (33.3, 40.8) 5 163 5,553 29.4 (24.9, 33.8) 6 159 6,905 23.0 (19.5, 26.6) 7 752 6,754 111.3 (103.8, 118.8) 8 94 4,815 19.5 (15.6, 23.4) 9 365 18,319 19.9 (17.9, 21.9) RAMQ Physician Billing Claims Sensitivity * Specificity * Study Physician (95% CI) (95% (I) 1 0.85 (0.78, 0.91) 1.00 (0.99, 1.00) 2 0.39 (0.33, 0.44) 0.98 (0.98, 0.99) 3 0.42 (0.35, 0.49) 1.00 (0.99, 1.00) 4 0.34 (0.24, 0.47) 1.00 (1.00, 1.00) 5 0.37 (0.22, 0.56) 1.00 (0.99, 1.00) 6 0.57 (0.16, 0.90) 0.99 (0.99, 1.00) 7 0.72 (0.45, 0.89) 0.98 (0.96, 1.00) 8 1.00 (1.00, 1.00) 0.99 (0.99, 1.00) 9 0.19 (0.06, 0.47) 1.00 (0.99, 1.00) RAMQ Physician Billing Claims NPW PPV ([dagger]) ([double dagger]) Study Physician (95% CI) (95% CI) 1 0.96 (0.94, 0.98) 0.99 (0.98, 1.00) 2 0.89 (0.86, 9.92) 0.82 (0.78, 0.86) 3 0.97 (0.95, 1.00) 0.84 (0.80, 0.88) 4 0.98 (0.96, 1.00) 0.93 (0.90, 0.96) 5 0.89 (0.81, 0.97) 0.95 (0.92, 0.99) 6 0.77 (0.64, 0.90) 0.99 (0.98, 1.00) 7 0.82 (0.66, 0.98) 0.96 (0.92, 1.00) 8 0.70 (0.53, 0.87) 1.00 (1.00, 1.00) 9 0.97 (0.95, 1.00) 0.92 (0.84, 1.00) * Adjusted for over-sampling of respiratory infections relative to other diagnoses (Begg and Greenes 1983; Irwig et al. 1994) ([dagger]0 Positive predictive value. ([double dagger]) Negative predictive value. RI, respiratory infection.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Methods and Research Briefs|
|Author:||Cadieux, Genevieve; Tamblyn, Robyn|
|Publication:||Health Services Research|
|Date:||Dec 1, 2008|
|Previous Article:||Can additional patient experience items improve the reliability of and add new domains to the CAHPS[R] Hospital Survey?|
|Next Article:||Adverse hospital events for mentally Ill patients undergoing coronary artery bypass surgery.|