Printer Friendly

Identifying Preanalytic and Postanalytic Laboratory Quality Gaps Using a Data Warehouse and Structured Multidisciplinary Process.

The complete laboratory testing process includes preanalytic (eg, test selection, ordering, and specimen collection), analytic (eg, specimen analysis), and postanalytic (eg, result reporting) components. (1) Laboratory quality improvement (QI) efforts have emphasized the analytic phase, in part because analytic data are generated within the laboratory; thus, those data are readily available for QI activities. Laboratories also focus on analytic-phase QI because analytic issues often negatively impact productivity or budget. Addressing preanalytic and postanalytic quality gaps is challenging for several reasons, one of which is that data generated outside the laboratory (eg, patient outcomes) have been of limited availability within the laboratory. (2) Unfortunately, emphasis on analytic-phase QI initiatives has contributed to a focus on test performance characteristics or cost per test rather than issues such as test utilization. Test utilization addresses questions that include whether the most appropriate test is ordered and performed in a timely manner, as well as whether the test results achieve the clinical intent. Nonlaboratory professionals are rarely included in analytic-focused laboratory QI activities, and conversely, efforts that use laboratory tests to improve health care outcomes often lack the input of laboratory professionals. Then again, laboratory test utilization teams, composed of laboratory professionals, clinicians, and others, have been established in many health care settings and tasked to assess the introduction of new tests and to recommend changes to test use based on cost, professional practice, and regulatory guidance.

Because most laboratory errors occur in the preanalytic and postanalytic phases of the total testing process, (3) laboratory QI projects should expand beyond analytic sources of error. This expanded approach requires timely access to comprehensive health care data including clinical, utilization, and administrative information. Electronic linkage of laboratory data with other health care data can support QI interventions to reduce quality gaps and to inform evidence-based laboratory practices. Importantly, decisions across the testing process can be facilitated by linking near-real-time laboratory, systems, and patient data to better understand relationships between test utilization and outcomes.

Medical data warehouses (MDWs) are repositories of clinical, utilization, and administrative data. Medical data warehouses have been used for research, surveillance, and QI. (4-18) An MDW contains data generated during health care delivery and payment. Medical data warehouse data originate from diverse sources that include the electronic health record (EHR), laboratory information system (LIS), pharmacy, and administrative databases (eg, patient enrollment, claims reimbursement). Data in an MDW are available in an architecture conducive to rapid searching. (19) These data are regularly updated, stored in a structured and consistent manner, quality checked, and linked to facilitate cross-table querying and analysis. Because data in an MDW reside in a database maintained separately from the source production information system databases (eg, the EHR), MDW data can be accessed and used at the same time as production information systems (19) without slowing production system response time or otherwise compromising production systems' requirements for real-time data access.

Medical data warehouses can be single site or multisite. One of the largest multisite MDWs is the Sentinel Initiative Distributed Database. It includes data from 17 data partners and contains information on more than 223 million members. (20, 21) One of the most comprehensive multisite MDWs is the 19-site Health Care Systems Research Network Virtual Data Warehouse (VDW). (22) Kaiser Permanente of Colorado (KPCO), Denver, Colorado, is one Health Care Systems Research Network site.

Studies using laboratory results data in an MDW (6, 18, 23-25) have addressed static questions relevant to laboratory testing (eg, adherence to testing guidelines), but have not been used as a tool to monitor test utilization to inform an intervention. Because laboratory professionals have specialized training and expertise in clinical testing, making data from an MDW available to laboratory professionals for QI initiatives can aid in identifying and solving practice gaps that otherwise might be missed. Therefore, the Centers for Disease Control and Prevention Division of Laboratory Systems, Atlanta, Georgia, and KPCO collaborated on a project to evaluate MDW usefulness in addressing a laboratory medicine-based QI intervention that links to patient or system outcomes. In this paper, we present the methods and findings from the first activities of this project (1) to demonstrate the utility of an MDW in characterizing laboratory QI opportunities and (2) to establish a multidisciplinary process that enables successful selection of a laboratory medicine-based QI opportunity for intervention.


This project was conducted in the KPCO Denver-Boulder metropolitan area. Kaiser Permanente of Colorado is a not-for-profit, integrated health care delivery system that in 2017 had more than 700 000 members, approximately 600 000 of whom were in the Denver-Boulder area. Members of KPCO receive outpatient health care at KP-owned medical facilities; facilities additionally provide laboratory, radiology, and pharmacy services. All KPCO facilities have fully integrated ambulatory EHR in all patient care areas with access from any KPCO location.

We applied a systematic, multicomponent approach to identify preanalytic and postanalytic laboratory medicine QI opportunities in the KPCO ambulatory care environment. We convened a Gap Identification Core Team (the Team) composed of local and national laboratory leaders, clinical content experts with diverse knowledge of quality issues, clinical medicine and pharmacy content experts, health information technology professionals, medical education specialists, evaluators, researchers, MDW experts, and project managers. For example, KPCO laboratory leaders who participated in the Team included the director of chemistry and toxicology (organizational leader), the director of pathology (physician champion; topic suggestions), the manager of laboratory QI (expertise on current laboratory QI projects), and the LIS manager (LIS capabilities; EHR interface). Team member expertise was instrumental in identifying, prioritizing, and selecting quality gaps. Because many evidence-based findings are not successfully implemented into routine clinical care, by carefully planning the composition of the Team and engaging key stakeholders early, our plan was to maximize the potential for success of this work. We intended to identify high-priority gaps that had local and national public health significance through proactively seeking KPCO sponsors and collaborators, educating internal stakeholders, and focusing on sustainability from the time of project initiation.

The Team interviewed 11 KPCO organizational and opinion leaders about QI priorities. These interviews established the local importance of the candidate gaps the Team had identified, while also identifying new potential quality gaps. The interviews also helped determine whether the local health care system was conducive to engagement with laboratory professionals to reduce the identified gaps. The leaders interviewed were selected because they represented expertise in clinical and operational quality, value, resource stewardship, health and prevention, chronic care, nursing, primary care, risk management, patient safety, and chemistry/ toxicology. Interviews were conducted by project managers and/or the project lead and began with an explanation of project goals and objectives. After the interviewers answered any general questions the interviewees had about the project, the interviewers asked the leaders about KPCO QI priorities, complementary or competing ongoing initiatives, and challenges the leaders believed should be addressed. Leaders were asked to prioritize (ie, high, medium, or low) several candidate potential laboratory medicine quality gaps. The interviewers also asked questions about the leaders' knowledge of the KPCO VDW. The qualitative input gathered during these interviews was incorporated into Team deliberations around potential gaps for intervention.

The MDW at KPCO, the VDW, is a comprehensive data repository that contains the timely, electronically linked data needed to characterize preanalytic and postanalytic laboratory medicine quality gaps. Key VDW content areas include demographics, enrollment, encounters, diagnoses, procedures, death, cause of death, tumor, census, pharmacy, vital signs, social history, provider, and LIS data such as laboratory results. Data tables are typically updated monthly or quarterly, but can be updated as often as daily if needed and if sufficient resources are available. Virtual data warehouse data lags range from essentially no lag (eg, laboratory results and other clinical data elements are available the same day) to 1 to 3 months (eg, external claims) to more than a year (eg, state death data). Virtual Data Warehouse tables are linked by a common, unique patient identifier that is different from the patient's health record number. The crosswalk between the VDW patient identifier and the patient's health record number is maintained in a separate data table for additional privacy and confidentiality protection. Implementation and operations of the VDW are governed by an operations committee. Ongoing Health Care Systems Research Network and KPCO processes ensure VDW quality is assessed and improved through programming and crowdsourcing via the user base. (22) Virtual data warehouse data were used to determine if current performance for each potential laboratory quality gap could be quantified, and, if so, whether those data indicated a gap. In addition, for each potential gap, the Team evaluated the feasibility of measuring patient or system outcomes using VDW data.

To select potential quality gaps, the Team used a standard decision aid tool (see examples in Table 1 and in Supplemental Tables 1 and 2 [see supplemental digital content at www. in the April 2019 table of contents]). The decision aid included health care system, patient, and data considerations. Completion of the decision aid for each potential gap was assisted by data and information gleaned from published literature, Team members' knowledge, leader input, and VDW data. The national significance of each potential quality gap was also considered with this tool.

This work is a public health activity. The KPCO Institutional Review Board determined that this project did not meet the regulatory definition of research involving human subjects.


Preanalytic and postanalytic laboratory medicine quality gap topics preliminarily considered are shown in Table 2. Based on knowledge developed using the decision aid, the preliminary list was narrowed to 3 topics for further consideration: (1) test ordering; (2) diagnosis, detection, and documentation; and (3) high-risk medication monitoring. The completed decision aid for high-risk medication monitoring is shown in Table 1. Completed decision aids for test ordering and diagnosis, detection, and documentation are available in Supplemental Tables 1 and 2.

Each of these 3 gap topics had several potential subtopics. Given available resources, the only subtopics examined within each topic were those of national importance identified by leaders as of high or medium interest at KPCO. For test ordering, subtopics included (1) tests that should be ordered only in certain circumstances (ie, prostate-specific antigen [PSA] testing, (26) 1,25-dihydroxyvitamin D testing, (26) and genetic testing), and (2) obsolete test ordering (ie, creatine kinase-MB isoenzyme for acute myocardial infarction). (26) Within diagnosis, detection, and documentation, the subtopics were conditions where laboratory testing was integral to diagnosis, including (1) serum creatinine (SCr) and glomerular filtration rate for chronic kidney disease, (27, 28) (2) glycosylated hemoglobin and fasting glucose for diabetes and prediabetes, (29-31) and (3) hemoglobin for anemia. For high-risk medication monitoring, the subtopics considered were areas where laboratory testing was important to safe and effective medication use. These included (1) international normalized ratio monitoring among patients with atrial fibrillation receiving warfarin and (2) alanine aminotransferase (ALT), aspartate aminotransferase (AST), complete blood count (CBC), and SCr monitoring among patients with a rheumatologic condition receiving a diseasemodifying antirheumatic drug (DMARD). (32)

The Team examined and deliberated on data extracted from the VDW and other evidence related to these potential quality gaps during 4 to 5 months. At the completion of this process, high-risk medication monitoring was selected as the laboratory medicine quality gap topic for future intervention, with ALT, AST, CBC, and SCr laboratory monitoring for DMARD therapy as the subtopic. A major consideration in selecting this quality gap was the potential for clinically important adverse events associated with DMARD use. For example, approximately 9% of patients taking the DMARD leflunomide have ALT and/or AST elevations exceeding 2- to 3-fold the upper limit of normal, with 2 to 4.9 per 10 000 patients per year experiencing leflunomide-associated hepatotoxicity of sufficient severity to require hospitalization. (33, 34) Similarly, the DMARD methotrexate is associated with a 15% occurrence of ALT and/or AST abnormalities during the first year of therapy, with 5% of patients discontinuing methotrexate because of hepatotoxicity. (35) Risks of DMARD-associated adverse events such as hepatotoxicity are minimized if laboratory test results are monitored in a timely manner and DMARD dosages adjusted or the DMARD discontinued if laboratory abnormalities are present or emerging. (36)

The VDW data tables accessed and linked to characterize the high-risk medication monitoring QI opportunity included enrollment, diagnoses (all individuals had at least 2 diagnoses of one of the following: rheumatoid arthritis, ankylosing spondylitis, psoriatic arthritis, or plaque psoriasis), laboratory results (dates of completion, result values, and reference ranges), pharmacy (dispensing dates and DMARD generic names), procedures (for DMARDs administered by infusion or injection), and death (an exclusion). From these VDW data, a gap in DMARD laboratory monitoring was characterized in guideline-concordant completion of laboratory tests by patients (timeliness of testing) (Table 3). For example, in patients treated with leflunomide or methotrexate, ALT, AST, CBC, and SCr are recommended to be evaluated at least every 12 weeks (84 days). (32) In preliminarily characterizing the gap, we used VDW pharmacy and laboratory data to estimate the proportion of patients with guideline-concordant completion of ALT, AST, CBC, and SCr, allowing a grace period of 16 days (ie, testing within 100 days). As shown in Table 3, among patients treated with leflunomide or methotrexate, many patients did not make themselves available for a blood sample to be drawn. More than 100 days elapsed between ALT test completions in 328 (65.5%) and 922 patients (60.7%) taking leflunomide or methotrexate, respectively. Similarly, more than 100 days elapsed between completion of CBC testing in 288 (57.5%) and 847 patients (55.7%), respectively.

Additional considerations that contributed to selecting this quality gap were organizational leadership, rheumatology, pharmacy, and laboratory departmental interest and the presence of DMARD regulatory labeling and national and local guidelines for laboratory testing and frequencies of testing. A final consideration in selecting high-risk medication monitoring as the gap for intervention was an intent to demonstrate that laboratory professionals are positioned to have leadership roles not only in identifying and characterizing quality gaps, but also in developing preanalytic or postanalytic interventions that link to health outcomes and process improvement when data through an MDW are available for analysis.

A test-ordering subtopic was not selected as the gap for intervention largely because the organization was already addressing several testing subtopics, including low-risk PSA testing, 1,25-dihydroxyvitamin D testing, and creatine kinase-MB isoenzyme ordering. For example, data from the VDW illustrated that an ongoing initiative to decrease low-risk PSA testing was effective: in 2012 (preintervention), 36 168 PSA results were documented in the VDW laboratory results table, whereas in 2015 (postintervention), 19 498 PSA results were documented. Further information is in Supplemental Table 1.

A diagnosis, detection, and documentation subtopic was not selected for intervention because the impact of a laboratory-led QI intervention would have been difficult to separate from the impact of ongoing initiatives led by other departments. For example, data extracted using VDW enrollment, death, diagnosis, and laboratory result tables indicated that an existing initiative was effective at ensuring diabetes was documented: among 6154 patients with initial and follow-up glycosylated hemoglobin result values of 6.5% or higher, 5817 (94.5%) had a diabetes diagnosis within 1 year. Further, because of multiple ongoing efforts within the organization aimed at improving diagnosis, detection, and documentation subtopics, organizational leaders encouraged the Team to pursue a different gap topic for this project. Further details are in Supplemental Table 2.


These results demonstrate the utility of an MDW for providing data to document the presence or absence of quality gaps in preanalytic and postanalytic phases of laboratory medicine. To develop the information needed to identify QI opportunities, we used linked MDW data content that included enrollment, demographics, diagnoses, pharmacy, laboratory results, and death. We then extracted relevant data and examined the results to confirm performance gaps.

Although production databases such as EHR and LIS usually have the capability to interrogate data within that database (eg, the LIS can be queried to provide data about laboratory testing), production databases typically cannot access and link data from other databases. This technical capability is a major strength of MDWs that enables their use for identifying preanalytic and postanalytic laboratory medicine QI opportunities. For example, because MDW data include dates of medication dispensing and laboratory results, temporal relationships relevant to patient testing are readily derived. Further, the quality of the data in MDWs such as the KPCO VDW and the VDWs at other Health Care Systems Research Network sites is maintained and improved through routine, ongoing characterization and quality checks of content areas. Also, all sites that participate in a multisite MDW use the same data structure, supporting conduct of the same or similar studies at other sites and enabling cross-site comparisons.

The innovative approach we applied to use MDW data to select laboratory medicine QI opportunities is generalizable and transferable, as health systems increasingly have MDWs as well as health information technology capabilities that can use MDW data. The potential scope of MDW utility in laboratory medicine QI is significant when placed within the context of the role that laboratory testing plays in medical care. Laboratory tests are ordered during one-third of primary care encounters (37) and laboratory results inform diagnosis, treatment selection, and monitoring of conditions and therapies. (2) Up to 54% of errors reported by primary care physicians and staff are related to laboratory medicine. (37)

Timeliness of guideline-concordant testing during DMARD therapy is a potential gap involving appropriate test utilization that the laboratory can identify and characterize using data available in an MDW. Characterizing this gap enables the laboratory to collaborate with clinicians to bring patients in for specimen collection (preanalytic) so that timely testing can be performed and reported (postanalytic) to inform medication adjustment, if needed. Contributing to appropriate clinical test utilization is within the laboratory's role, (38) but is part of the role that is difficult to fulfill without linking laboratory data to patient and outcomes data via an MDW. This linkage provides opportunities to advance the laboratory's role in improving health outcomes, considering that the intervention will include identifying patients who are not guideline concordant (a laboratory role relative to the DMARD gap).

Notably, our method included convening a multidisciplinary team to identify quality gaps and to select a gap for future intervention. The Team's work was supported by a decision aid tool where relevant information was compiled and reviewed. The expertise of Team members was instrumental in identifying performance gaps. Additionally, Team engagement was crucial to developing commitment to the project, agreement on gap topics to pursue, awareness around MDW utility in quality initiatives, and building the trust necessary for laboratory professionals to initiate and effectively engage with the local health care system. Furthermore, we believe that the project work now underway (developing and implementing the intervention to improve timeliness of laboratory monitoring of DMARD therapy) is progressing smoothly in large part because of the multidisciplinary relationships established during the gap selection activities.

Interviewing leaders to determine their knowledge and QI priorities served the dual purpose of also educating them about preanalytic and postanalytic laboratory QI opportunities. The interviews also aided in developing the organizational commitments necessary to identify a locally important laboratory medicine QI gap and to establish engagement for a future intervention targeting that gap.

One fact the Team learned from the leader interviews was that, in a large organization such as KPCO, no single individual is knowledgeable about all ongoing QI initiatives or all complementary or competing priorities. This learning underscores the importance of the multidisciplinary process when identifying and planning for a successful laboratory-centric QI intervention that impacts the preanalytic or postanalytic phase of the total testing process.

The multidisciplinary team approach and organizational leader engagement efforts are key strategies to, and activities for, local success and internal sustainability, as well as strategies to enhance external generalizability and dissemination. The multistakeholder involvement approach intersects with a recommendation regarding the utility of diagnostic management teams (and health care teams in general) made in the Institute of Medicine (39) (now National Academy of Medicine) report, Improving Diagnosis in Health Care. It also intersects with laboratory test utilization teams. It is potentially more accurate to consider our project's multidisciplinary team as a combination of a diagnostic management team and a laboratory test utilization team (ie, to improve test utilization in conjunction with an intervention/outreach). The Institute of Medicine report also emphasizes the importance of health information technologies such as MDWs to support patients and health care professionals.

A limitation to the transferability of our work is that although all MDWs contain administrative data, some MDWs do not include the clinical data needed to identify laboratory QI opportunities. (20, 40) However, even at organizations with this constraint, MDW tables such as diagnosis, pharmacy, and procedures (eg, whether a laboratory test was completed) can be linked to identify QI opportunities.

In conclusion, providing laboratory access to clinical and other data contained within an MDW provides novel and important opportunities to identify and characterize quality gaps that may otherwise not be addressed. The process described also highlights the need for the laboratory to engage with other departments because patient and system outcomes include broad components of a health care delivery system. Our example demonstrated that the combination of laboratory, clinical, and administrative health care data available in a comprehensive MDW, together with the expertise of a multidisciplinary team of professionals, supplemented by organizational stakeholder engagement, is effective at identifying, prioritizing, and characterizing quality gaps in the preanalytic and postanalytic phases of laboratory testing. The baseline data developed and the information learned to date will inform the development and implementation of a specific QI intervention.

We thank Glenn Goodrich, MS, for programming support. We also thank Matthew Rubinstein MS, MT (ASCP), for his valued consultation when the project was beginning. Finally, we thank Bharati Bhardwaja, PharmD, BCPS, LSSBB, for her guidance on laboratory monitoring among patients with a rheumatologic condition receiving a disease-modifying antirheumatic drug. Drs Raebel, Smith, and Sterrett, Mss Quintana, Shetterly, and Pieper, and Mr Epner received salary support for this work through a cooperative agreement from the Centers for Disease Control and Prevention (5NU47OE000102-02-00) to Kaiser Permanente Colorado.


(1.) Smith ML, Raab SS, Fernald DH, et al. Evaluating the connections between primary care practice and clinical laboratory testing: a review of the literature and call for laboratory involvement in the solutions. Arch Pathol Lab Med. 2013; 137(1):120-125.

(2.) Plebani M. Exploring the iceberg of errors in laboratory medicine. Clin Chim Acta. 2009;404(1):16-23.

(3.) Christenson RH, Snyder SR, Shaw CS, et al. Laboratory medicine best practices: systematic evidence review and evaluation methods for quality improvement. Clin Chem. 2011;57(6):816-825.

(4.) Raebel MA, Zeng C, Cheetham TC, et al. Risk of breast cancer with long-term use of calcium channel blockers or angiotensin-converting enzyme inhibitors among older women. Am J Epidemiol. 2017;185(4):264-273.

(5.) Andrade SE, Raebel MA, Brown J, et al. Use of antidepressant medications during pregnancy: a multi-site study. Am J Obstet Gynecol. 2008;198(2): 194.e191-e195.

(6.) Raebel MA, Ross C, Xu S, et al. Diabetes and drug-associated hyperkalemia: effect of potassium monitoring. J Gen Intern Med. 2010;25(4):326-333.

(7.) Raebel MA, Newcomer SR, Reifler LM, et al. Chronic use of opioid medication before and after bariatric surgery. J'AMA. 2013;310(13):1369-1376.

(8.) Raebel MA, Ellis JL, Schroeder EB, et al. Intensification of antihyperglycemic therapy among patients with incident diabetes: a Surveillance Prevention and Management of Diabetes Mellitus (SUPREME-DM) study. Pharmacoepidemiol Drug Saf. 2014;23(7):699-710.

(9.) Nichols GA, Schroeder EB, Karter AJ, et al. Trends in diabetes incidence, 2006-2011, in seven million insured US adults: the SUPREME-DM project. Am J Epidemiol. 2015;181(1):32-39.

(10.) Lafata EJ, Schmittdiel J, Karter AJ, et al. Medication adherence does not explain black-white differences in cardiometabolic risk factor control among insured patients with diabetes. J Gen Intern Med. 2016;31(2):188-195.

(11.) Lu CY, Zhang F, Lakoma MD, et al. Changes in antidepressant use by young people and suicidal behavior after FDA warnings and media coverage: quasiexperimental study. BMJ. 2014;348:g3596.

(12.) Neugebauer R, Fireman B, Roy J, O'Connor PJ, SelbyJV. Dynamic marginal structural modeling to evaluate the comparative effectiveness and safety of more or less aggressive treatment intensification strategies in type 2 diabetes. Pharmacoepidemiol Drug Saf. 2012;21(suppl 2):99-113.

(13.) Nichols GA, Desai J, Elston LJ, et al. Construction of a multisite DataLink using electronic health records for the identification, surveillance, prevention, and management of diabetes mellitus: the SUPREME-DM project. Prev Chronic Dis. 2012;9:E110.

(14.) Schroeder EB, Goodrich GK, Newton KM, Schmittdiel JA, Raebel MA. Implications of different laboratory-based incident diabetic kidney disease definitions on comparative effectiveness studies. J Comp Eff Res. 2014;3(4): 359-369.

(15.) Schmittdiel JA, Raebel MA, Dyer W, et al. Medicare STAR excludes patients with poor CVD risk factor control. Am J Manag Care. 2014;20(12):e581-e589.

(16.) Flory JH, Roy J, Gagne JJ, et al. Missing laboratory results data in electronic health databases: implications for monitoring diabetes risk. J Comp Eff Res. 2017; 6(1):25-32.

(17.) Raebel MA, Shetterly S, Lu CY, et al. Methods for using clinical laboratory test results as baseline confounders in multi-site observational database studies when missing data are expected. Pharmacoepidemiol Drug Saf. 2016;25(7):798-814.

(18.) Raebel MA, Penfold R, McMahon AW, et al. Adherence to guidelines for glucose assessment in starting second-generation antipsychotics. Pediatrics. 2014;134(5):e1308-e1314.

(19.) Krasowski MD, Schriever A, Mathur G, Blau JL, Stauffer SL, Ford BA. Use of a data warehouse at an academic medical center for clinical pathology quality improvement, education, and research. J Pathol Inform. 2015;6:45.

(20.) Sentinel Initiative. Distributed database and common data model. Available at Published 2017. Accessed October 26, 2017.

(21.) Curtis LH, Weiner MG, Boudreau DM, et al. Design considerations, architecture, and use of the Mini-Sentinel distributed data system. Pharmacoepidemiol Drug Saf. 2012;21(suppl 1):23-31. doi:10.1002/pds.2336

(22.) Ross TR, Ng D, Brown JS, et al. The HMO Research Network Virtual Data Warehouse: a public data model to support collaboration. EGEMS (Wash DC). 2014;2(1):1-8.

(23.) Feldstein AC, Smith DH, Perrin N, et al. Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med. 2006;166(17): 1848-1854.

(24.) Raebel MA, Chester EA, Newsom EE, et al. Randomized trial to improve laboratory safety monitoring of ongoing drug therapy in ambulatory patients. Pharmacotherapy. 2006;26(5):619-626.

(25.) Raebel MA, Carroll NM, Andrade SE, et al. Monitoring of drugs with a narrow therapeutic range in ambulatory care. Am J Manag Care. 2006;12(5):268274.

(26.) ABIM Foundation. Choosing wisely. Available at http://www. Published 2017. Accessed October 30, 2017.

(27.) Centers for Disease Control and Prevention Office of Disease Prevention and Health Promotion. Healthy People 2020: chronic kidney disease objectives. Available at chronic-kidney-disease/objectives. Published 2010. Accessed October 30, 2017.

(28.) KDIGO. CKD evaluation and management. Available at guidelines/. Accessed October 30, 2017.

(29.) American Diabetes Association. Classification and diagnosis of diabetes. Diabetes Care. 2015;38(suppl 1):S8-S16.

(30.) Centers for Disease Control and Prevention Office of Disease Prevention and Health Promotion. Healthy People 2010: diabetes. Available at http://www. Accessed November 22, 2017.

(31.) American Diabetes Association. Standards of medical care in diabetes--2016. Diabetes Care. 2016;39(suppl 1):S1-S112.

(32.) Singh JA, Saag KG, Bridges SL Jr, et al. 2015 American College of Rheumatology guideline for the treatment of rheumatoid arthritis. Arthritis Care Res (Hoboken). 2016;68(1):1-25.

(33.) Suissa S, Ernst P, Hudson M, et al. Newer disease-modifying antirheumatic drugs and the risk of serious hepatic adverse events in patients with rheumatoid arthritis. Am J Med. 2004;117(2):87-92.

(34.) Alcorn N, Saunders S, Madhok R. Benefit-risk assessment of leflunomide: an appraisal of leflunomide in rheumatoid arthritis 10 years after licensing. Drug Saf. 2009;32(12):1123-1134.

(35.) Lopez-Olivo MA, Siddhanamatha HR, Shea B, Tugwell P, Wells GA, Suarez-Almazor ME. Methotrexate for treating rheumatoid arthritis. Cochrane Database Syst Rev. 2014;(6):CD000957. doi:10.1002/14651858.CD000957. pub2

(36.) Saag KG, Teng GG, Patkar NM, et al. American College of Rheumatology 2008 recommendations for the use of nonbiologic and biologic disease modifying antirheumatic drugs in rheumatoid arthritis. Arthritis Rheum. 2008; 59(6):762-784.

(37.) Hickner J, Fernald D, Harris D, Poon EG, Elder N, Mold J. Issues and initiatives in the testing process in primary care physician offices. Jt Comm J Qual Patient Saf. 2005;31(2):81-89.

(38.) Sarkar MK, Botz CM, Laposata M. An assessment of overutilization and underutilization of laboratory tests by expert physicians in the evaluation of patients for bleeding and thrombotic disorders in clinical context and in real time. Diagnosis. 2017;4(1):21-26.

(39.) Institute of Medicine, National Academies of Sciences, Engineering, Medicine. Improving Diagnosis in Health Care. Washington, DC: National Academies Press; 2015.

(40.) Raebel MA, Haynes K, Woodworth TS, et al. Electronic clinical laboratory test results data tables: lessons from Mini-Sentinel. Pharmacoepidemiol Drug Saf. 2014;23(6):609-618.

Marsha A. Raebel, PharmD; LeeAnn M. Quintana, MSW; Emily B. Schroeder, MD, PhD; Susan M. Shetterly, MS; Lisa E. Pieper, MBA, MSHA; Paul L. Epner, MBA, MEd; Laura K. Bechtel, PhD; David H. Smith, PhD; Andrew T. Sterrett, PhD; Joseph A. Chorny, MD; Ira M. Lubin, PhD

Accepted for publication June 8, 2018.

Published online December 10, 2018.

Supplemental digital content is available for this article at in the April 2019 table of contents.

From the Institute for Health Research (Drs Raebel, Schroeder, and Sterrett and Mss Quintana, Shetterly, and Pieper), Kaiser Permanente Colorado, Denver; the Society to Improve Diagnosis in Medicine, Evanston, Illinois (Mr Epner); the Regional Laboratory, Kaiser Permanente Colorado, Aurora (Dr Bechtel); the Center for Health Research, Kaiser Permanente Northwest, Portland, Oregon (Dr Smith); the Regional Laboratory, Colorado Permanente Medical Group, Aurora (Dr Chorny); and the Quality and Safety Systems Branch, Division of Laboratory Systems, Centers for Surveillance, Epidemiology, and Laboratory Services, Centers for Disease Control and Prevention, Atlanta, Georgia (Dr Lubin).

The authors have no relevant financial interest in the products or companies described in this article.

The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention or the US Agency for Toxic Substances and Disease Registry.

Corresponding author: Marsha A. Raebel, PharmD, Institute for Health Research, Kaiser Permanente Colorado, PO Box 378066, Denver, CO 80237 (email:
Table 1. Completed Decision Aid for Identifying and Selecting a
Laboratory Medicine Quality Gap for Potential Intervention:
High-Risk Medication Monitoring (a)

Consideration                   Completed Assessment (in Brief) for
                                  High-Risk Medication Monitoring
                                           Potential Cap

Quality gap is preventable    Yes. Laboratory monitoring (CBC, ALT,
or has potential targets      AST, SCr) is recommended for many of
for improvement (list         these agents for safety and/or
specifics)                    appropriate dosage

Strategies/interventions      Practice level intervention using EHR or
could be used to reduce the   laboratory decision support reminder to
gap (list)                    order recommended laboratory monitoring;
                              patient IVR laboratory test reminder

Potential harms that might    Unintended consequences that change
be associated with the        existing workflow
interventions (1st)

Intermediate surrogate        Yes. Compliance with guidelines/
outcomes that could be        labeling recommendations: No. (%) with
measured (list)               laboratory test(s) completed System:
                              Time to dosage or medication change
                              after laboratory test result available
                              Clinical: (1) No. (%) with elevated ALT/
                              AST, (2) No. (%) with neutropenia, (3)
                              No. (%) with elevated SCr

Long-term outcomes that       Early detection of developing adverse
could directly affect         events
patients or the health care
system (list)                 Timely laboratory results enhance
                              ability to titrate dosage to better
                              manage condition Decreased provider
                              burden associated with laboratory
                              outreach Enhanced guideline concordance

Local importance (low/        Medium
medium/high; potential
effect size at least

National importance (low/     Medium. Impacts moderate No. of patients
medium/high; include          Drug toxicity contributes to patient
references)                   morbidity and mortality
                              HEDIS criteria include DMARD use

                              American College of Rheumatology
                              recommends optimal follow-up intervals
                              for complete blood count, liver
                              transaminase levels, and creatinine for
                              patients with rheumatoid arthritis
                              receiving DMARDs

                              Product labeling from FDA recommends
                              laboratory monitoring for safe DMARD use

Local health care system      Yes. Rheumatoid arthritis registry in
conducive to the              development that includes
intervention (yes/no;         patient-specific laboratory monitoring
explain)                      status. EHR Smart Set reminds providers
                              to order laboratory tests for rheumatoid
                              arthritis patients

                              Registries planned for gastrointestinal
                              and dermatology patients receiving these

                              Clinical pharmacy specialist resources
                              in rheumatology, gastroenterology, and
                              dermatology are enthusiastic about
                              opportunity to support appropriate
                              laboratory monitoring of these high-
                              risk drugs

Feasibility of establishing   High. Required data already in VDW
baseline status (low/
medium/high; explain)

Baseline data indicate a      Yes
gap in performance (yes/
no; provide summary)

Availability of published     Singh et al (32)
standards/criteria to
enable evaluation (provide

Feasibility of assessing      High. Data in VDW
outcomes using MDW (low/
medium/high; explain)         Experience using these laboratory and
                              drug data

Relevant data in MDW (list    Laboratory results (CBC, ALT/AST, SCr)
data needed)                  Drug dispensing and infusion data

Data needed from outside      No
MDW (yes/no; if yes, list)

Potential challenges (list)   Interventions to reduce quality gap
                              likely will require electronic patient
                              outreach and/or EHR alerts or messages.
                              These intervention types require
                              development assistance and approvals
                              from departments not on the study. Such
                              activities have been collaboratively
                              accomplished in other projects; time
                              frame to develop must be coordinated
                              with these other departments

                              Guidelines for monitoring are changing
                              over time

                              Differences in laboratory monitoring
                              frequencies exist between patients newly
                              started on DMARD and patients being
                              chronically maintained on DMARD.
                              Different programming and evaluation
                              needed for different patient groups

                              Differences in recommended laboratory
                              monitoring by individual agents
                              increases complexity

                              Not a high-intensity intervention

Abbreviations: ALT, alanine aminotransferase; AST, aspartate
aminotransferase; CBC, complete blood count; DMARD,
disease-modifying antirheumatic drug; EHR, electronic health record;
FDA, US Food and Drug Administration; H EDIS, Healthcare
Effectiveness Data and Information Set; IVR, interactive voice
response; MDW, medical data warehouse; SCr, serum creatinine; VDW,
virtual data warehouse.

(a) Completed decision aids for the laboratory medicine quality gaps
test ordering and diagnosis, detection, and documentation are
available in Supplemental Tables 1 and 2.

Table 2. Laboratory Medicine Quality Gap Topics Considered

Gap                     Gap                       Testing Phase(s)

Topic                   Subtopic(s)

Decrease                CK-MB                   Preanalytic
inappropriate test      PSA
ordering                1,25-dihydroxyvitamin
                        D Genetic tests

Improve diagnosis,      CKD at risk-no          Preanalytic and
detection, and          diagnosis; laboratory   postanalytic
documentation           test evidence of CKD

                        Diabetes and
                        prediabetes Anemia

Improve high-risk       Anticoagulants          Preanalytic and
medication monitoring   DMARDs                  postanalytic

Improve CKD             CKD diagnosis; no       Preanalytic
monitoring,             microalbuminuria/
automated ordering      creatinine ratio, SCr

                        Test grouping with
                        SCr, lipids)

Improve workup of       JAK2 testing for        Preanalytic
polycythemia            unexplained

Address posttesting     Follow-up testing for   Postanalytic
gaps in colorectal      positive fecal
cancer screening        immunochemical test

                        Time to diagnosis

Address failure to      Test pending at         Postanalytic
retrieve laboratory     discharge but after
test results            discharge, primary
                        care provider unaware

                        Results not looked at
                        until the next office

Reduce                  Prediabetes diagnosis   Postanalytic
diabetes-related gaps   not in problem list

                        Microalbumin annually

Improve secondary       Elevated parathyroid    Postanalytic
hyperparathyroidism     hormone without a
recognition             diagnosis of

Address laboratory      By test type            Preanalytic
testing noncompletion
                        By patient

                        By clinic

Gap                      How Gap Identified     Consideration Status


Decrease                Gap Identification      Finalist
inappropriate test      Core Team

Improve diagnosis,      Gap Identification      Finalist
detection, and          Core Team

Improve high-risk       Organizational          Finalist and selected
medication monitoring   leaders                 for future

Improve CKD             Gap Identification      Preliminary only
monitoring,             Core Team
automated ordering

Improve workup of       Gap Identification      Preliminary only
polycythemia            Core Team

Address posttesting     Gap Identification      Preliminary only
gaps in colorectal      Core Team
cancer screening

Address failure to      Gap Identification      Preliminary only
retrieve laboratory     Core Team
test results

Reduce                  Gap Identification      Preliminary only
diabetes-related gaps   Core Team

Improve secondary       Organizational          Preliminary only
hyperparathyroidism     leaders

Address laboratory      Organizational          Preliminary only
testing noncompletion   leaders

Abbreviations: CKD, chronic kidney disease; CK-MB, creatine kinase-MB
isoenzyme; DMARD, disease-modifying antirheumatic drug; JAK2, Janus
kinase 2 gene; PSA, prostate-specific antigen; SCr, serum creatinine.

Table 3. Data From Virtual Data Warehouse
Indicating Gap in Timeliness of Disease-Modifying
Antirheumatic Drug (DMARD) Monitoring Among
2203 Patients With a Rheumatologic Diagnosis (a)

Laboratory Testing              Patients With a Gap Between
for the DMARDs                      Laboratory Testings
Leflunomide and Methotrexate        >100 d, No. (%) (b)

Leflunomide (501 patients)
  Alanine aminotransferase              328 (65.5)
  Aspartate aminotransferase            305 (60.9)
  Complete blood count                  288 (57.5)
  Serum creatinine                      293 (58.5)
Methotrexate (1520 patients)
  Alanine aminotransferase              922 (60.7)
  Aspartate aminotransferase            886 (58.3)
  Complete blood count                  847 (55.7)
  Serum creatinine                      857 (56.4)

(a) Date range: January 1, 2014-September 30, 2016. Patients were
counted only once within a DMARD and within a specific laboratory
test. Patients could appear more than once across laboratory tests (eg,
have a gap in both alanine aminotransferase testing and complete
blood count testing) and could have gaps in both DMARDs if they
were taking both medications.

(b) Patients were counted as having a gap of >100 days only if they
remained enrolled in the health plan, remained alive, had at least 100
days elapse after starting DMARD therapy, and did not have the
DMARD discontinued during those >100 days (based on both
DMARD dispensing/administration data and no DMARD
discontinuation documented). National (32) and Kaiser Permanente
Colorado guidelines recommend monitoring every 12 weeks (ie, 84
COPYRIGHT 2019 College of American Pathologists
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2019 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Raebel, Marsha A.; Quintana, LeeAnn M.; Schroeder, Emily B.; Shetterly, Susan M.; Pieper, Lisa E.; E
Publication:Archives of Pathology & Laboratory Medicine
Article Type:Report
Date:Apr 1, 2019
Previous Article:Teaching Genomic Pathology: Translating Team-Based Learning to a Virtual Environment Using Computer-Based Simulation.
Next Article:Gastrointestinal Presentations of Common Variable Immunodeficiency: Hiding in Plain Sight.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |