Printer Friendly

Data Analytics for Improved Decision Making at a Veterans Affairs Medical Center.

INTRODUCTION

There are over 5,000 registered hospitals in the US with approximately 900,000 beds [1]. Hospital care expenditure in the US is over $970B, up from about $9B in 1960 [2]. Healthcare is going through a major transformation, and will see significant efforts to lower costs, improve patient outcomes, and in general go towards increased efficiencies. Using data analytics for improved decision making is no longer a luxury but a necessity, and yet most major hospitals do not use the full potential of the data that they have to make decisions. The federal VA program is approximately 4% of the total healthcare industry, and provides care to the nation's veterans through 152 centrally administered hospitals. These hospitals do the best they can under trying conditions, severe budget issues and limited direction from the center other than to increase efficiency and provide improved care/outcomes. All of these problems stem from not having standardized tools for better decision making, so most decisions are made ad-hoc, typically responding to immediate concerns (e.g. reduce the 30-day mortality rate at the VA in Marion, IL). Managing this large enterprise is challenging, and using data analytics to improve patient outcomes and reduce healthcare costs has become a huge priority [3,4]. Strategic Analytics for Improvement and Learning Value Model or SAIL [5], is a system for summarizing hospital system performance within the VAs. The VA developed the SAIL model to measure, evaluate and benchmark quality and efficiency at all its medical centers [6]. The SAIL report is an excellent tool, and provides benchmarked data for all 152 VA centers. It must be noted here that SAIL reports do not provide division level data, and a case is made in this paper for divisions, such as Cardiology, to request similar data from their IT departments on a weekly basis, if they are to have any chance of making lasting changes in their divisions.

Each quarter when the SAIL reports come out, there is usually a meeting called by the VA Chief of Staff, and is attended by numerous division heads, clinicians and staff. The SAIL report, which is a single Excel worksheet is usually put up on the screen. For the Marion VA, the immediate focus has been the 30-Day mortality rate (SMR30), as they are shown to be in the lowest 10% so there is intense discussion on how to improve it. The hospital has some of the finest clinicians in the country, so the executives and the staff struggle to find a viable roadmap to make changes as they do not know if this is a resource issue or a system inefficiency. There are very few visual tools to help the staff make sense of the data, and there are certainly no tools to see past trends, model the system, play what-if scenarios, etc. Hence, typically, committees are put together to consider the matter, make suggestions, and usually there is little appreciable difference year after year. That is certainly not due to the staff not making a concerted effort, but it is more to do with not having the tools to make informed data-driven decisions.

There is a critical need for new ways to use the SAIL data, identify inefficiencies, and execute/monitor the changes made. This paper provides a methodology to do exactly that, so that individual divisions can compare themselves to other more efficient counterpart divisions around the country, and make deliberate changes, and monitor the progress weekly, rather than quarterly.

Data Analytics: Visualization leading to Prioritization

The SAIL reports are an excellent system to concisely provide information on 32 metrics to all 152 VAs along with benchmark data, but the VAs do not have an easy-to-understand visualization tool, and most discussions are still based on just pointing to the numbers on the SAIL report (see Figure 1 for a typical SAIL report). Our approach is to take the numbers from the SAIL report and provide a visual method for honing in to what are the most critical areas to talk about. The SAIL report, as mentioned before, has 32 metrics. Each metric row is followed by 5 items in the 5 columns:

Col 1: how the metric was scored

Col 2: preferred direction for metric (should be low or high)

Col 3: metric score for that VA

Col 4: benchmark for all 152 VAs

Col 5: 10th-50th-90th percentiles

Trying to make decisions by studying the SAIL report is not easy, and often leads to more confusion, especially since for some metrics, the preferred direction is going up, while for others it may be going down.

This paper extracts the data from the SAIL report and provides an easy-to-understand and the intuitive first step is to provide a visualization tool that gives a "Green" color for those in the top 50%, "Yellow" for those between 50-10%, and "Red" for those in the lowest 10% in the country, and therefore calling for immediate action (see Figure 2). Just a simple glance at the plot shows that there are five metrics that need immediate attention as they are in the lowest 10th percentile, and it turns out that one of the most important metrics, SMR30, is one of these (the second bar in Figure 2).

Now that we have identified that SMR30 is the outcome metric that we need to bring down, we consider five variables given below, as the primary variables influencing SMR30. All of the data is available in the SAIL reports, but at the hospital level, but each division within the hospital can request division level data from their IT department.

1. In Hospital Complications [right arrow] IHC

2. Health care Associated Infections (HAI)

a. Catheter Associated Urinary Tract Infection: [right arrow] CAUTI

b. Central Line Associated Bloodstream Infection: [right arrow] CLABI

c. Ventilator Associated Pneumonia: [right arrow] VAP (Not used due to lack of data)

d. Methicillin-Resistant Staphylococcus Aureus Infection: [right arrow] MRSA

3. Patient Safety Indicator [right arrow] PSI

Data Analytics: Comparison with other VAs

Given in Figure 3 is a color-coded chart that compares SMR30 and the five of the six metrics, that have data available, for various other VAs around the country (Erie, PA; Hampton, VA; St. Louis, MO; Wichita, KS). This chart helps in visualizing how important the metrics are in relation to the SMR30. For example, better IHC and CAUTI numbers lead to a better SMR30 for Erie, PA. For Marion, IL, to improve their SMR30, they must improve their IHC and CAUTI numbers.

The question then is which one to target first: IHC or CAUTI? We have at our disposal data from 17 quarters for all VAs, and we would like to use that to see if we can find these correlations. For the purposes of this study, we will look at the 17 quarters worth of SAIL data for Marion VA, Illinois. It must be noted here, that the objective of this study is to develop a methodology to use data analytics to find insight from the data, and once we can reach some actionable conclusions, we can easily scale the study to any VA in the country. It must also be noted that some VAs do not have data for some metrics in one or more quarters, so any approach must consider missing data (as we did in this section by not considering VAP).

Data Analytics: Identifying a roadmap for change

This paper reports on using regression analysis to find the correlations between SMR30 and the other metrics, with the correlation coefficient being the number that is to be calculated. The correlation coefficient is a measure that determines the degree to which two variables are associated. The range of values for the correlation coefficient is -1.0 to 1.0 and it can't be greater than 1.0 or less than -1.0. A correlation of -1.0 indicates a perfect negative correlation, while a correlation of 1.0 indicates a perfect positive correlation. The most common calculation is known as the Pearson product-moment correlation. It is determined using the below equation,

[mathematical expression not reproducible] (eq.1)

[SIGMA] is sigma, the symbol of 'sum up'

([x.sub.i] - [bar.x]) is the difference between each x value and the mean of x

([y.sub.i] - [bar.y]) is the difference between each x value and the mean of y

The correlation values between SMR30 and the selected metrics are shown in Table 1 As can be seen above, SMR30 is the most correlated with IHC (correlation coefficient of 0.4725). This is a very reliable outcome as we had data for all 17 quarters.

Data Analytics: Modeling to study the impact of changes

This, by far is the most significant contribution of this paper, i.e. the ability to model the impact of changing any metric on other metrics. A stepwise regression analysis was performed on the data to come up with the following linear models with a confidence level of 93%. In the regression analysis, response variable was taken as SMR30 and the continuous predictors were IHC, CAUTI, MRSA and PSI. The regression equation ended up with just IHC and excluded other predictors demonstrating that SMR30 is highly related to IHC. This statistical model that relates SMR30 to IHC is used to target a change to be made in IHC based on a desired change in SMR30:

SMR30 = 1.028 + 0.1934 IHC

The models which predict the consequences of making the above change in IHC on other critical metrics such as MRSA, CAUTI and PSI are obtained by simple linear regression and are given by:

MRSA = 0.1075 + 0.0600 IHC

CAUTI = 1.889 - 0.488 IHC

PSI = 0.761 - 0.102 IHC

Figure 4 is the plot of the four models. The x-axis is values for IHC ranging from 0.0 to 2.0. The solid black line is the plot for SMR30, and desired direction for it is shown (high to low). This is the driver for all changes to be made.

Data Analytics: Case Study

Given below are the results from the model (also shown visually in Figure 4). Let us try and see what happens when we want to reduce the SMR30 value by 5%.

Current SMR30 value: 1.2070

Corresponding IHC value: 1.1030 (vertical line on plot labeled current)

New desired SMR30 value: 1.2070*0.95 = 1.1466

Targeted IHC value from model: 0.6135 (decrease of 44% from 1.1030)

Consequence on MRSA: 0.1443 (decrease of 16.9% from 0.1737)

Consequence on CAUTI: 1.5896 (increase of 17.7% from 1.3507)

Consequence on PSI: 0.6984 (increase of 7.7% from 0.6485)

As can be seen above, a desired 5% change in SMR30 would require a targeted 44% decrease in IHC. If this change was accomplished, then this would come with good consequences on MRSA, but potentially unintended consequences on CAUTI and PSI. It is this knowledge that allows healthcare executives to make informed decisions, and monitor the progress to make sure that the unintended consequences are minimized.

CONCLUSION

This reports on a study using data analytics for decision making at the Veterans Affairs (VA) Medical Center in Marion, IL, to improve patient outcomes, specifically the SMR30 (30-day Standardized Mortality Ratio). At the overall VA level, the SAIL data is used for visualizing the data so that critical problem areas can be quickly identified and then compared to other VAs around the country. A regression analysis is then conducted to see which metric to target so as to have the maximum impact on SMR30, and finally a statistical model is developed to have some idea on intended and unintended consequences of making any changes. A case study using more than four years of data is used to demonstrate the power of the methodology. It must be noted here, that the output of the model is based on data analytics, and may not necessarily be clinically accurate, but it does provide a framework for making changes, and monitoring the consequences. This paper has shown a data-driven methodology for using the SAIL data to make decisions at the overall VA level, but more importantly this approach can be used at the division level, where the actual changes need to be made. The division level data is not part of the SAIL report, but may be requested by the division heads internally, and then used exactly via the methodology presented in this paper to improve their division level patient outcomes.

REFERENCES

[1] AHA Hospital Statistics, "Fast facts on US Hospitals," Health Forum, an American Hospital Association Affiliate, 2017 ed.

[2] The Statista Portal, https://www.statista.com/statistics/184772/u s-hospital-care-expenditures-since-1960/. May 2017.

[3] U.S. Department of Veterans Affairs, https://www.va.gov/, April 20, 2017.

[4] Mark Byers, "Using big data to benefit veterans," https://fcw.com/articles/2015/01/12/commen t-big-data-vha.aspx, Jan 12, 2015.

[5] US Department of Veterans Affairs, https://www.va.gov/OUALITYOFCARE/m easure up/Strategic Analytics for Improvement a nd Learning SAIL.asp, Feb 10, 2017.

[6] Factsheet, http://www.blogs.va. gov/V Antag e/wpcontent/uploads/2014/11/SAILFactShee t.pdf, Nov 2014.

Ajay Mahajan (1), Padmini Selvaganesan (1), Parag Madhani (2) and Sanjeevi Chitikeshi (3)

(1) University of Akron, Akron, OH, (2) VA Medical Center, Marion, IL and (3) Old Dominion University, Norfolk, VA

Caption: Figure 8. New visualization tool

Caption: Figure 4. Modelling Results
Figure 1. SAIL Report

Strategic Analytics for Improvement and Learning (SAIL)

NOTE EFFICIENCY FOR FY2013-2014 IS BASED ON FY2013 DATA; MPATIENT
SHEPAND FOVH SURVEY FOR FY2014Q4-FY2015Q1 IS BASED ON FY2014Q4 DATA.

SAIL IS REFRESHED ON A QUARTERLY BASIS. MEASURE VALUES MAY CHANGE IN
ACCORDANCE WITH CHANGES IN THE SOURCE DATA.

These documents or records or information contained herein, which
resulted from the Operational Analytics and Reporting, VA Office of
Informatics and Analytics are confidential and privileged under the
provisions of 38 USC 5705 and its implementing regulations. This
material will not be disclosed to anyone w ithout authorization as
provided for by that law or its regulations. The statute provides for
fines up to $20,000 for unauthorized disclosures.

Marion IL Scorecard for FY2015Q1

Measure                               Measure Unit         Preferred

Acute care mortality
  1. Acute care Standardized               O/E           [down arrow]
     Mortality Ratio (SMR)
  2. Acute care 30-day                     O/E           [down arrow]
     Standardized Mortality
     Ratio (SMR30)
Avoidable adverse events
  1. In-hospital complications             O/E           [down arrow]
  2. Health care associated
     infections (HAI)
    a. Catheter associated         inf/1k device days    [down arrow]
       urinary tract infection
    b. Central line associated     inf/1k device days    [down arrow]
       bloodstream infection
    c. Ventilator associated       inf/1k device days    [down arrow]
       Pneumonia
    d. Methicillin-resistant         inf/1k bed days     [down arrow]
       Staphylococcus aureus
       (MRSA) infection
  3. Patient safety indicator              O/E           [down arrow]
       (PSI)
CMS 30-day Risk Standardized
Mortality Rate (RSMR)
  1. AMI RSMR                               %            [down arrow]
  2. CHF RSMR                               %            [down arrow]
  3. Pneumonia RSMR                         %            [down arrow]
CMS 30-day Risk Standardized
Readmission Rate (RSRR)
  1. AMI RSRR                               %            [down arrow]
  2. CHF RSRR                               %            [down arrow]
  3. Pneumonia RSRR                         %            [down arrow]
Adjusted length of stay                   days           [down arrow]
Performance measures
  1. Inpatient performance                  %             [up arrow]
     measures (ORYX)
  2. Outpatient performance               wct%            [up arrow]
     measures (HEDIS like)
Customer satisfaction
  1. Patient satisfaction             score (0-300)       [up arrow]
  2. Best places to work              score (1-100)       [up arrow]
    a. Overall job satisfaction        score (1-5)        [up arrow]
    b. Satisfaction with               score (1-5)        [up arrow]
       organization
    c. Recommend my organization       score (1-5)        [up arrow]
       as a good place to work
  3. Registered nurse turnover              %            [down arrow]
     rate
Ambulatory Care Sensitive             hosp/1000 pts      [down arrow]
Condition hospitalizations
Access
  1. Primary care w ait time
    a. New primary care                     %             [up arrow]
       appointments completed
       within 30 days from
       preferred date
    b. PCMH Access composite       casemix adjusted %     [up arrow]
    i. Get an urgent care          casemix adjusted %     [up arrow]
       appointment as soon as
       needed
    ii. Get a routine care         casemix adjusted %     [up arrow]
       appointment as soon as
       needed
2. Specialty care wait time
  a. New specialty care                     %             [up arrow]
     appointments completed
     within 30 days from
     preferred date
3. Mental health wait time
  a. New mental health                      %             [up arrow]
     appointments completed
     within 30 days from
     preferred date
4. Call responsiveness
  a. Call center speed in                seconds         [down arrow]
     responding to calls in
     seconds
  b. Call center abandonment                %            [down arrow]
     rate
Mental Health                      Standardized score     [up arrow]
  1. Population coverage           Standardized score     [up arrow]
  2. Continuity of care            Standardized score     [up arrow]
  3. Experience of care            Standardized score     [up arrow]
Efficiency (1/SFA)                    score (0-100)       [up arrow]

Measure                            Marion IL    Benchmark

Acute care mortality
  1. Acute care Standardized           0.951        0.483
     Mortality Ratio (SMR)
  2. Acute care 30-day                 1.455        0.731
     Standardized Mortality
     Ratio (SMR30)
Avoidable adverse events
  1. In-hospital complications         1.868        0.339
  2. Health care associated
     infections (HAI)
    a. Catheter associated             0.731        0.000
       urinary tract infection
    b. Central line associated         0.000        0.000
       bloodstream infection
    c. Ventilator associated           0.000        0.000
       Pneumonia
    d. Methicillin-resistant           0.137        0.000
       Staphylococcus aureus
       (MRSA) infection
  3. Patient safety indicator          0.000        0.000
       (PSI)
CMS 30-day Risk Standardized
Mortality Rate (RSMR)
  1. AMI RSMR
  2. CHF RSMR                          7.248        6.469
  3. Pneumonia RSMR                    9.069        7.474
CMS 30-day Risk Standardized
Readmission Rate (RSRR)
  1. AMI RSRR                                      16.336
  2. CHF RSRR                         18.304       17.892
  3. Pneumonia RSRR                   14.583       13.507
Adjusted length of stay                4.540        3.674
Performance measures
  1. Inpatient performance            92.584       99.492
     measures (ORYX)
  2. Outpatient performance           91.190       91.473
     measures (HEDIS like)
Customer satisfaction
  1. Patient satisfaction            265.670      267.586
  2. Best places to work              58.254       64.734
    a. Overall job satisfaction        3.631        3.737
    b. Satisfaction with               3.461        3.605
       organization
    c. Recommend my organization       3.751        3.872
       as a good place to work
  3. Registered nurse turnover         3.814        3.491
     rate
Ambulatory Care Sensitive             33.579       20.257
Condition hospitalizations
Access
  1. Primary care w ait time
    a. New primary care               98.857       99.740
       appointments completed
       within 30 days from
       preferred date
    b. PCMH Access composite          40.996       53.800
    i. Get an urgent care             43.785       59.919
       appointment as soon as
       needed
    ii. Get a routine care            60.330       66.284
       appointment as soon as
       needed
2. Specialty care wait time
  a. New specialty care               93.936       98.551
     appointments completed
     within 30 days from
     preferred date
3. Mental health wait time
  a. New mental health                99.508       99.885
     appointments completed
     within 30 days from
     preferred date
4. Call responsiveness
  a. Call center speed in            114.648       19.052
     responding to calls in
     seconds
  b. Call center abandonment          24.943        3.359
     rate
Mental Health                         -0.204        1.130
  1. Population coverage              -0.407        1.147
  2. Continuity of care               -0.579        0.994
  3. Experience of care                0.586        1.023
Efficiency (1/SFA)                    91.466       96.093

Measure                                10th-50th-90th ptile

Acute care mortality
  1. Acute care Standardized             0.483 - 0.877 - 1.178
     Mortality Ratio (SMR)
  2. Acute care 30-day                   0.731 - 0.955 - 1.192
     Standardized Mortality
     Ratio (SMR30)
Avoidable adverse events
  1. In-hospital complications           0.339 - 1.052 - 1.523
  2. Health care associated
     infections (HAI)
    a. Catheter associated               0.000 - 1.013 - 3.013
       urinary tract infection
    b. Central line associated           0.000 - 0.478 - 1.471
       bloodstream infection
    c. Ventilator associated             0.000 - 0.000 - 3.442
       Pneumonia
    d. Methicillin-resistant             0.000 - 0.082 - 0.284
       Staphylococcus aureus
       (MRSA) infection
  3. Patient safety indicator            0.000 - 0.759 - 1.184
       (PSI)
CMS 30-day Risk Standardized
Mortality Rate (RSMR)
  1. AMI RSMR
  2. CHF RSMR                            6.469 - 7.508 - 8.850
  3. Pneumonia RSMR                     7.474 - 9.152 - 11.290
CMS 30-day Risk Standardized
Readmission Rate (RSRR)
  1. AMI RSRR                         16.336 - 16.386 - 16.451
  2. CHF RSRR                         17.892 - 19.422 - 21.678
  3. Pneumonia RSRR                   13.507 - 14.956 - 16.540
Adjusted length of stay                  3.674 - 4.500 - 5.395
Performance measures
  1. Inpatient performance            95.407 - 97.734 - 99.492
     measures (ORYX)
  2. Outpatient performance           87.271 - 89.457 - 91.473
     measures (HEDIS like)
Customer satisfaction
  1. Patient satisfaction          238.804 - 256.250 - 267.586
  2. Best places to work              49.650 - 58.185 - 64.734
    a. Overall job satisfaction          3.448 - 3.609 - 3.737
    b. Satisfaction with                 3.150 - 3.410 - 3.605
       organization
    c. Recommend my organization         3.430 - 3.659 - 3.872
       as a good place to work
  3. Registered nurse turnover          3.491 - 6.242 - 11.213
     rate
Ambulatory Care Sensitive             20.257 - 26.169 - 32.975
Condition hospitalizations
Access
  1. Primary care w ait time
    a. New primary care               83.536 - 97.367 - 99.740
       appointments completed
       within 30 days from
       preferred date
    b. PCMH Access composite          32.078 - 42.025 - 53.800
    i. Get an urgent care             30.553 - 45.549 - 59.919
       appointment as soon as
       needed
    ii. Get a routine care            41.417 - 54.364 - 66.284
       appointment as soon as
       needed
2. Specialty care wait time
  a. New specialty care               89.604 - 95.595 - 98.551
     appointments completed
     within 30 days from
     preferred date
3. Mental health wait time
  a. New mental health                96.667 - 99.196 - 99.885
     appointments completed
     within 30 days from
     preferred date
4. Call responsiveness
  a. Call center speed in            19.052 - 58.461 - 195.731
     responding to calls in
     seconds
  b. Call center abandonment            3.359 - 8.918 - 22.063
     rate
Mental Health                           -1.351 - 0.094 - 1.130
  1. Population coverage                -1.194 - 0.009 - 1.147
  2. Continuity of care                 -1.167 - 0.083 - 0.994
  3. Experience of care                 -1.153 - 0.046 - 1.023
Efficiency (1/SFA)                    91.008 - 94.386 - 96.093

Figure 9. Color-coded visualization tool

            Erie     Hampton     Marion    St. Louis  Wichita

SMR30     63.028%    90.683%       5%        35%      46.719%
IHC         100%     73.511%    20.952%    21.275%    28.528%
CAUTI       90%        90%       46.49%     49.58%      90%
CLABI     No Data     17.74%      90%         5%        90%
MRSA        90%      34.182%     37.34%    30.727%      10%
PSI         10%         5%        90%      65.827%    49.097%

Table 1. Correlation Values

            IHC     HAI--CAUTI     HAI--CLABI    HAI--MRSA      PSI

Marion    0.4725      -0.1424     Missing Data     0.1682     -0.0573
COPYRIGHT 2018 Mississippi Academy of Sciences
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Mahajan, Ajay; Selvaganesan, Padmini; Madhani, Parag; Chitikeshi, Sanjeevi
Publication:Journal of the Mississippi Academy of Sciences
Article Type:Report
Date:Apr 1, 2018
Words:3711
Previous Article:PULSED LASER DEPOSITION OF BIOACTIVE COATING FROM WHITE PORTLAND CEMENT.
Next Article:THE EFFECTIVENESS OF DRY NEEDLING ON THE REDUCTION OF PROXIMAL UPPER QUARANT PAIN USING COHEN'S d: A SYSTEMATIC REVIEW.
Topics:

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |