Finding gaps in quality and value (Part 2).

In Part I of this series, we described the problem of inappropriate care in the United States and how solutions to cost and quality in health care can be effectively dealt with at the organizational level.

We began a consideration of the sequential phases of evidence-based quality improvement work, starting out with readying an organization for evidence-based quality improvement. Now, let's look at details of using the five A's (1) of evidence-based medicine to identify and close quality and cost gaps in health care organizations.

Once an organization creates the structural components, such as committees and work groups, and establishes processes for those groups, and once it ensures that staff have the needed knowledge, skills and tools to carry out the five A's of evidence-based clinical improvement (Asking, Acquiring, Appraising, Applying, A's Again), each group can begin to identify "fixable" or "closable" quality, cost, satisfaction and uncertainty gaps in clinical care.

The sequential steps in this evidence-based process are outlined in Table 1.

The Institute of Medicine outlined six quality domains (Table 2) which are useful in determining areas within an organization where there may be quality gaps and where quality may be improved. (2)

The size of the gap should justify the effort it will take to close it. To determine the size of the performance gap in a potential work area, groups need to compare internal organizational data (usually obtained from organizational databases) with the best available evidence (obtained from content resources, such as sources for guidelines, clinical recommendations or the medical literature itself.)

All the data must be appraised for validity unless they come from a trusted source such as Cochrane, Clinical Evidence or the Database of Reviews of Effects (DARE)--and must be updated and assessed for usefulness.

In Part I of this series, we indicated that frequently physicians, as well as quality improvement professionals and other decision-making health care professionals, lack the skills to effectively and efficiently search for, critically appraise and synthesize scientific evidence using processes that yield valid, useful and usable content likely to improve desired outcomes.

Individuals doing quality improvement work may benefit from training that can successfully provide the skills and tools for evaluating the medical literature. Training should improve competencies in finding and utilizing studies with appropriate designs, valid methods and useful results.

An approach we have found useful is to teach these skills using the five "A"s of evidence-based medicine:

* Ask -- How to construct effective clinical questions

* Acquire -- Tips and strategies for systematically capturing potentially useful content through awareness of the best sources for information, application of successful search techniques and filtering strategies

* Appraise -- Concepts and methods for evaluating content for validity, usefulness and usability, along with organizational considerations (e.g., cost, legal, marketing, public relations and other value considerations)

* Apply -- Using valid and useful content, how to synthesize the body of evidence, creating information, decision and action aids for use by clinicians, patients and others

* "A"s Again -- When and how to repeat the process to ensure information is current

These sequential steps are summarized in Table 3 and can be made easier by utilizing various tools.

After completing the evidence synthesis, we strongly recommend making evidence-based estimates regarding local quality and cost outcomes followed by the development of information, decision and action tools, implementation plans and measurement plans. We will describe the details of these steps in Part III of this series. (Watch for it in the May/June 2005 issue of The Physician Executive.)

TABLE 1 Identifying Quality, Cost, Satisfaction and Uncertainty Gaps

Ask & Acquire: Identify gaps and uncertainties; select projects

Clinical improvement projects find and close gaps in key areas such as
health care status, satisfaction of clinical staff and patients,
utilization and cost
Look for gaps between current and optimal practice based on internal
organizational data and the best available evidence, or areas of
clinical uncertainty
At times gaps may be created by regulators:
Generally you will benefit by preparing a written statement containing
your preliminary best guess of the following:
* Rationale (introduction, background, summary of need/gap based on
  evidence)
* Summary of what the project will accomplish
* Required resources
* Projections of outcomes (quality, cost and other considerations)
* Implementation plan
* Measurement plan

TABLE 2 Quality Domains from the Institute of Medicine (IOM)

Domains                      Requirements for Improving Quality

Safety -- avoiding injuries  Requires attention to reducing errors.
to patients from the care
that is intended to help
them
Effectiveness -- providing   Requires the identification of those
services based on            interventions that should be put into
scientific knowledge to all  practice through application of usable
who could benefit and        medical evidence which is valid and useful.
refraining from providing    Requires the identification of medical
services to those not        practices for which there is insufficient
likely to benefit (avoiding  evidence or evidence of no benefit or harm
underuse and overuse).       through analysis of usable medical
                             evidence.
Patient-centeredness --      Requires attention to providing information
providing care that is       to patients on issues of import to them
respectful of and            such as benefits, harms, risks, costs,
responsive to individual     uncertainties and alternatives. Successful
patient preferences, needs,  engagements between clinicians and patients
and values, and ensuring     include providing information along with
that patient values guide    warmth, empathy, respect, and frequently
all clinical decisions.      facilitating patients' choices while
                             attending to individual patient preferences
                             for decision styles. Successful
                             communications with patients effectively
                             supply knowledge, facilitate decision-
                             making and/or describe potential actions to
                             be taken.
                             Requires sensitivity to patient care,
                             comfort and emotional needs from the
                             patient's point of view.
Timeliness -- reducing       Requires attention to access, coordination
waits and sometimes harmful  of care and patient pathways through the
delays for both those who    health system along with potential
receive those who give       mechanisms for how care is made available
care.                        to patients (i.e., in person visits, group
                             visits, Web site care centers, self-
                             management protocols, etc.).
Efficiency -- avoiding       Requires attention to all processes used in
waste, in particular waste   health care to reduce complexity and
of equipment, supplies,      redundancy.
ideas, and energy.
Equity -- providing care     Requires a respectful approach to the
that does not vary in        individual and the individual's needs. It
quality because of personal  also requires respect for various
characteristics, ethnicity,  populations.
geographic location, and
socioeconomic status.

TABLE 3 The 5 "As" of EBM--Detailed Steps

Clinical Content  How to decide on the clinical content for your
                  improvement project. What does the evidence say?

Ask & Acquire:    Apply systematic strategies to obtain evidence,
                  filtering for strength of study design and for
                  relevance
Search for the best available  Formulate questions and search strategy
evidence. This may be in the   and, pose those questions to the medical
form of guidelines, clinical   literature and to other sources for
recommendations, studies of    content to systematically obtain all
studies or original articles.  potentially useful content
                               Prioritize sources. It may be efficient
                               to first consider guidelines, then
                               systematic reviews and then primary
                               sources.
Filter for appropriateness of  Review title and/or abstract to identify
study design and relevance.    potentially useful content, screening for
                               most appropriate method and applicability
                               to patients and circumstances for care.
                               Acquire selected information:
                               If obtaining secondary sources such as
                               guidelines, clinical recommendations or
                               systematic reviews, you also need to
                               search for important studies that may
                               have been published in the primary
                               literature after the date of the
                               published article's literature search.
                               For committees and work groups, abstract
                               key elements of study into an evidence
                               table or evidence summary to help
                               facilitate appraisal.
Assess work needed.            Are you comfortable that you have
                               potentially sufficient evidence that can
                               be adapted from another source, or will
                               you need to develop your own content from
                               the primary and secondary sources in the
                               medical literature, and do you have the
                               necessary resources available for this
                               work?

Appraise:         Critically appraise studies and content for validity.
Assess validity (closeness to  Become mindful of the pitfalls of the
truth) and usability of        medical literature, research, clinical
information.                   practice guidelines, QI improvements and
                               other sources of content.
                               Understand strengths of different
                               methodologies for research, guideline
                               development and clinical improvement
                               projects.
                               Assess selected content for relevance to
                               population and your circumstances for
                               care.
                               Evaluate content for validity. This
                               requires doing a formal analysis of study
                               type and study methodology which is best
                               done using a quality critical appraisal
                               tool.

Appraise:         Examine results of valid studies and content
Examine the results of valid   "Usable Evidence" = Valid + Useful +
studies                        Usable
                               For valid studies and content, examine
                               results by considering the following
                               categories.
                               Is the evidence:
                               Useful * Effective
                               Clinically significant -- directly
                               benefits patients in areas of
                               morbidity, mortality, symptom relief,
                               functioning and quality of life
                               Size of study results -- measures of
                               outcomes (aka "estimates of effect")
                               Efficacy versus effectiveness (e.g.,
                               outcomes resulting from idealized
                               conditions in research = efficacy/
                               versus what is likely to happen in the
                               real world = effectiveness)
                               Appropriate * Relevant to patients
                               Patient perspective -- benefits, harms,
                               risks, costs, uncertainties, alternatives
                               Applicability -- to which patients and
                               under what circumstances
                               Usable * Acceptable
                               Will clinicians and staff accept this
                               and apply it appropriately?
                               Will patients accept and adhere to
                               treatment?
                               Actionable (e.g., is there FDA approval,
                               is the change affordable, is it doable in
                               your organization? Will you actually be
                               able to apply the information and
                               successfully implement the change?
                               Summarize validity and other results of
                               your assessment, making a summary
                               judgment.

Apply:            Summarize and synthesize the evidence using systematic
                  methods for combining information obtained from
                  different sources

References:

1. Modified by Delfini Group, LLC (www.delfini.org) from Leung GM. "Evidence-based practice revisited." Asia Pac J Public Health. 2001; 13(2):116-21.

2. Institute of Medicine, Crossing the Quality Chasm, Washington, D.C: National Academy Press, 2001.

By Sheri Strite and Michael E. Stuart, MD

Sheri Strite is associate director of program development, family and preventive medicine, at the School of Medicine at University of California, San Diego, Calif. She is also managing partner, Delfini Group, LLC, in San Diego. She can be reached at [e-mail] or 619-683-3819.

Michael E. Stuart, MD, is clinical assistant professor of family medicine at the School of Medicine at University of Washington in Seattle, Wash. He is also president of Delfini Group, LLC. He can be reached at [e-mail] or 206-522-4279.