Data Analytics for Improved Decision Making at a Veterans Affairs Medical Center.
There are over 5,000 registered hospitals in the US with approximately 900,000 beds . Hospital care expenditure in the US is over $970B, up from about $9B in 1960 . Healthcare is going through a major transformation, and will see significant efforts to lower costs, improve patient outcomes, and in general go towards increased efficiencies. Using data analytics for improved decision making is no longer a luxury but a necessity, and yet most major hospitals do not use the full potential of the data that they have to make decisions. The federal VA program is approximately 4% of the total healthcare industry, and provides care to the nation's veterans through 152 centrally administered hospitals. These hospitals do the best they can under trying conditions, severe budget issues and limited direction from the center other than to increase efficiency and provide improved care/outcomes. All of these problems stem from not having standardized tools for better decision making, so most decisions are made ad-hoc, typically responding to immediate concerns (e.g. reduce the 30-day mortality rate at the VA in Marion, IL). Managing this large enterprise is challenging, and using data analytics to improve patient outcomes and reduce healthcare costs has become a huge priority [3,4]. Strategic Analytics for Improvement and Learning Value Model or SAIL , is a system for summarizing hospital system performance within the VAs. The VA developed the SAIL model to measure, evaluate and benchmark quality and efficiency at all its medical centers . The SAIL report is an excellent tool, and provides benchmarked data for all 152 VA centers. It must be noted here that SAIL reports do not provide division level data, and a case is made in this paper for divisions, such as Cardiology, to request similar data from their IT departments on a weekly basis, if they are to have any chance of making lasting changes in their divisions.
Each quarter when the SAIL reports come out, there is usually a meeting called by the VA Chief of Staff, and is attended by numerous division heads, clinicians and staff. The SAIL report, which is a single Excel worksheet is usually put up on the screen. For the Marion VA, the immediate focus has been the 30-Day mortality rate (SMR30), as they are shown to be in the lowest 10% so there is intense discussion on how to improve it. The hospital has some of the finest clinicians in the country, so the executives and the staff struggle to find a viable roadmap to make changes as they do not know if this is a resource issue or a system inefficiency. There are very few visual tools to help the staff make sense of the data, and there are certainly no tools to see past trends, model the system, play what-if scenarios, etc. Hence, typically, committees are put together to consider the matter, make suggestions, and usually there is little appreciable difference year after year. That is certainly not due to the staff not making a concerted effort, but it is more to do with not having the tools to make informed data-driven decisions.
There is a critical need for new ways to use the SAIL data, identify inefficiencies, and execute/monitor the changes made. This paper provides a methodology to do exactly that, so that individual divisions can compare themselves to other more efficient counterpart divisions around the country, and make deliberate changes, and monitor the progress weekly, rather than quarterly.
Data Analytics: Visualization leading to Prioritization
The SAIL reports are an excellent system to concisely provide information on 32 metrics to all 152 VAs along with benchmark data, but the VAs do not have an easy-to-understand visualization tool, and most discussions are still based on just pointing to the numbers on the SAIL report (see Figure 1 for a typical SAIL report). Our approach is to take the numbers from the SAIL report and provide a visual method for honing in to what are the most critical areas to talk about. The SAIL report, as mentioned before, has 32 metrics. Each metric row is followed by 5 items in the 5 columns:
Col 1: how the metric was scored
Col 2: preferred direction for metric (should be low or high)
Col 3: metric score for that VA
Col 4: benchmark for all 152 VAs
Col 5: 10th-50th-90th percentiles
Trying to make decisions by studying the SAIL report is not easy, and often leads to more confusion, especially since for some metrics, the preferred direction is going up, while for others it may be going down.
This paper extracts the data from the SAIL report and provides an easy-to-understand and the intuitive first step is to provide a visualization tool that gives a "Green" color for those in the top 50%, "Yellow" for those between 50-10%, and "Red" for those in the lowest 10% in the country, and therefore calling for immediate action (see Figure 2). Just a simple glance at the plot shows that there are five metrics that need immediate attention as they are in the lowest 10th percentile, and it turns out that one of the most important metrics, SMR30, is one of these (the second bar in Figure 2).
Now that we have identified that SMR30 is the outcome metric that we need to bring down, we consider five variables given below, as the primary variables influencing SMR30. All of the data is available in the SAIL reports, but at the hospital level, but each division within the hospital can request division level data from their IT department.
1. In Hospital Complications [right arrow] IHC
2. Health care Associated Infections (HAI)
a. Catheter Associated Urinary Tract Infection: [right arrow] CAUTI
b. Central Line Associated Bloodstream Infection: [right arrow] CLABI
c. Ventilator Associated Pneumonia: [right arrow] VAP (Not used due to lack of data)
d. Methicillin-Resistant Staphylococcus Aureus Infection: [right arrow] MRSA
3. Patient Safety Indicator [right arrow] PSI
Data Analytics: Comparison with other VAs
Given in Figure 3 is a color-coded chart that compares SMR30 and the five of the six metrics, that have data available, for various other VAs around the country (Erie, PA; Hampton, VA; St. Louis, MO; Wichita, KS). This chart helps in visualizing how important the metrics are in relation to the SMR30. For example, better IHC and CAUTI numbers lead to a better SMR30 for Erie, PA. For Marion, IL, to improve their SMR30, they must improve their IHC and CAUTI numbers.
The question then is which one to target first: IHC or CAUTI? We have at our disposal data from 17 quarters for all VAs, and we would like to use that to see if we can find these correlations. For the purposes of this study, we will look at the 17 quarters worth of SAIL data for Marion VA, Illinois. It must be noted here, that the objective of this study is to develop a methodology to use data analytics to find insight from the data, and once we can reach some actionable conclusions, we can easily scale the study to any VA in the country. It must also be noted that some VAs do not have data for some metrics in one or more quarters, so any approach must consider missing data (as we did in this section by not considering VAP).
Data Analytics: Identifying a roadmap for change
This paper reports on using regression analysis to find the correlations between SMR30 and the other metrics, with the correlation coefficient being the number that is to be calculated. The correlation coefficient is a measure that determines the degree to which two variables are associated. The range of values for the correlation coefficient is -1.0 to 1.0 and it can't be greater than 1.0 or less than -1.0. A correlation of -1.0 indicates a perfect negative correlation, while a correlation of 1.0 indicates a perfect positive correlation. The most common calculation is known as the Pearson product-moment correlation. It is determined using the below equation,
[mathematical expression not reproducible] (eq.1)
[SIGMA] is sigma, the symbol of 'sum up'
([x.sub.i] - [bar.x]) is the difference between each x value and the mean of x
([y.sub.i] - [bar.y]) is the difference between each x value and the mean of y
The correlation values between SMR30 and the selected metrics are shown in Table 1 As can be seen above, SMR30 is the most correlated with IHC (correlation coefficient of 0.4725). This is a very reliable outcome as we had data for all 17 quarters.
Data Analytics: Modeling to study the impact of changes
This, by far is the most significant contribution of this paper, i.e. the ability to model the impact of changing any metric on other metrics. A stepwise regression analysis was performed on the data to come up with the following linear models with a confidence level of 93%. In the regression analysis, response variable was taken as SMR30 and the continuous predictors were IHC, CAUTI, MRSA and PSI. The regression equation ended up with just IHC and excluded other predictors demonstrating that SMR30 is highly related to IHC. This statistical model that relates SMR30 to IHC is used to target a change to be made in IHC based on a desired change in SMR30:
SMR30 = 1.028 + 0.1934 IHC
The models which predict the consequences of making the above change in IHC on other critical metrics such as MRSA, CAUTI and PSI are obtained by simple linear regression and are given by:
MRSA = 0.1075 + 0.0600 IHC
CAUTI = 1.889 - 0.488 IHC
PSI = 0.761 - 0.102 IHC
Figure 4 is the plot of the four models. The x-axis is values for IHC ranging from 0.0 to 2.0. The solid black line is the plot for SMR30, and desired direction for it is shown (high to low). This is the driver for all changes to be made.
Data Analytics: Case Study
Given below are the results from the model (also shown visually in Figure 4). Let us try and see what happens when we want to reduce the SMR30 value by 5%.
Current SMR30 value: 1.2070
Corresponding IHC value: 1.1030 (vertical line on plot labeled current)
New desired SMR30 value: 1.2070*0.95 = 1.1466
Targeted IHC value from model: 0.6135 (decrease of 44% from 1.1030)
Consequence on MRSA: 0.1443 (decrease of 16.9% from 0.1737)
Consequence on CAUTI: 1.5896 (increase of 17.7% from 1.3507)
Consequence on PSI: 0.6984 (increase of 7.7% from 0.6485)
As can be seen above, a desired 5% change in SMR30 would require a targeted 44% decrease in IHC. If this change was accomplished, then this would come with good consequences on MRSA, but potentially unintended consequences on CAUTI and PSI. It is this knowledge that allows healthcare executives to make informed decisions, and monitor the progress to make sure that the unintended consequences are minimized.
This reports on a study using data analytics for decision making at the Veterans Affairs (VA) Medical Center in Marion, IL, to improve patient outcomes, specifically the SMR30 (30-day Standardized Mortality Ratio). At the overall VA level, the SAIL data is used for visualizing the data so that critical problem areas can be quickly identified and then compared to other VAs around the country. A regression analysis is then conducted to see which metric to target so as to have the maximum impact on SMR30, and finally a statistical model is developed to have some idea on intended and unintended consequences of making any changes. A case study using more than four years of data is used to demonstrate the power of the methodology. It must be noted here, that the output of the model is based on data analytics, and may not necessarily be clinically accurate, but it does provide a framework for making changes, and monitoring the consequences. This paper has shown a data-driven methodology for using the SAIL data to make decisions at the overall VA level, but more importantly this approach can be used at the division level, where the actual changes need to be made. The division level data is not part of the SAIL report, but may be requested by the division heads internally, and then used exactly via the methodology presented in this paper to improve their division level patient outcomes.
 AHA Hospital Statistics, "Fast facts on US Hospitals," Health Forum, an American Hospital Association Affiliate, 2017 ed.
 The Statista Portal, https://www.statista.com/statistics/184772/u s-hospital-care-expenditures-since-1960/. May 2017.
 U.S. Department of Veterans Affairs, https://www.va.gov/, April 20, 2017.
 Mark Byers, "Using big data to benefit veterans," https://fcw.com/articles/2015/01/12/commen t-big-data-vha.aspx, Jan 12, 2015.
 US Department of Veterans Affairs, https://www.va.gov/OUALITYOFCARE/m easure up/Strategic Analytics for Improvement a nd Learning SAIL.asp, Feb 10, 2017.
 Factsheet, http://www.blogs.va. gov/V Antag e/wpcontent/uploads/2014/11/SAILFactShee t.pdf, Nov 2014.
Ajay Mahajan (1), Padmini Selvaganesan (1), Parag Madhani (2) and Sanjeevi Chitikeshi (3)
(1) University of Akron, Akron, OH, (2) VA Medical Center, Marion, IL and (3) Old Dominion University, Norfolk, VA
Caption: Figure 8. New visualization tool
Caption: Figure 4. Modelling Results
Figure 1. SAIL Report Strategic Analytics for Improvement and Learning (SAIL) NOTE EFFICIENCY FOR FY2013-2014 IS BASED ON FY2013 DATA; MPATIENT SHEPAND FOVH SURVEY FOR FY2014Q4-FY2015Q1 IS BASED ON FY2014Q4 DATA. SAIL IS REFRESHED ON A QUARTERLY BASIS. MEASURE VALUES MAY CHANGE IN ACCORDANCE WITH CHANGES IN THE SOURCE DATA. These documents or records or information contained herein, which resulted from the Operational Analytics and Reporting, VA Office of Informatics and Analytics are confidential and privileged under the provisions of 38 USC 5705 and its implementing regulations. This material will not be disclosed to anyone w ithout authorization as provided for by that law or its regulations. The statute provides for fines up to $20,000 for unauthorized disclosures. Marion IL Scorecard for FY2015Q1 Measure Measure Unit Preferred Acute care mortality 1. Acute care Standardized O/E [down arrow] Mortality Ratio (SMR) 2. Acute care 30-day O/E [down arrow] Standardized Mortality Ratio (SMR30) Avoidable adverse events 1. In-hospital complications O/E [down arrow] 2. Health care associated infections (HAI) a. Catheter associated inf/1k device days [down arrow] urinary tract infection b. Central line associated inf/1k device days [down arrow] bloodstream infection c. Ventilator associated inf/1k device days [down arrow] Pneumonia d. Methicillin-resistant inf/1k bed days [down arrow] Staphylococcus aureus (MRSA) infection 3. Patient safety indicator O/E [down arrow] (PSI) CMS 30-day Risk Standardized Mortality Rate (RSMR) 1. AMI RSMR % [down arrow] 2. CHF RSMR % [down arrow] 3. Pneumonia RSMR % [down arrow] CMS 30-day Risk Standardized Readmission Rate (RSRR) 1. AMI RSRR % [down arrow] 2. CHF RSRR % [down arrow] 3. Pneumonia RSRR % [down arrow] Adjusted length of stay days [down arrow] Performance measures 1. Inpatient performance % [up arrow] measures (ORYX) 2. Outpatient performance wct% [up arrow] measures (HEDIS like) Customer satisfaction 1. Patient satisfaction score (0-300) [up arrow] 2. Best places to work score (1-100) [up arrow] a. Overall job satisfaction score (1-5) [up arrow] b. Satisfaction with score (1-5) [up arrow] organization c. Recommend my organization score (1-5) [up arrow] as a good place to work 3. Registered nurse turnover % [down arrow] rate Ambulatory Care Sensitive hosp/1000 pts [down arrow] Condition hospitalizations Access 1. Primary care w ait time a. New primary care % [up arrow] appointments completed within 30 days from preferred date b. PCMH Access composite casemix adjusted % [up arrow] i. Get an urgent care casemix adjusted % [up arrow] appointment as soon as needed ii. Get a routine care casemix adjusted % [up arrow] appointment as soon as needed 2. Specialty care wait time a. New specialty care % [up arrow] appointments completed within 30 days from preferred date 3. Mental health wait time a. New mental health % [up arrow] appointments completed within 30 days from preferred date 4. Call responsiveness a. Call center speed in seconds [down arrow] responding to calls in seconds b. Call center abandonment % [down arrow] rate Mental Health Standardized score [up arrow] 1. Population coverage Standardized score [up arrow] 2. Continuity of care Standardized score [up arrow] 3. Experience of care Standardized score [up arrow] Efficiency (1/SFA) score (0-100) [up arrow] Measure Marion IL Benchmark Acute care mortality 1. Acute care Standardized 0.951 0.483 Mortality Ratio (SMR) 2. Acute care 30-day 1.455 0.731 Standardized Mortality Ratio (SMR30) Avoidable adverse events 1. In-hospital complications 1.868 0.339 2. Health care associated infections (HAI) a. Catheter associated 0.731 0.000 urinary tract infection b. Central line associated 0.000 0.000 bloodstream infection c. Ventilator associated 0.000 0.000 Pneumonia d. Methicillin-resistant 0.137 0.000 Staphylococcus aureus (MRSA) infection 3. Patient safety indicator 0.000 0.000 (PSI) CMS 30-day Risk Standardized Mortality Rate (RSMR) 1. AMI RSMR 2. CHF RSMR 7.248 6.469 3. Pneumonia RSMR 9.069 7.474 CMS 30-day Risk Standardized Readmission Rate (RSRR) 1. AMI RSRR 16.336 2. CHF RSRR 18.304 17.892 3. Pneumonia RSRR 14.583 13.507 Adjusted length of stay 4.540 3.674 Performance measures 1. Inpatient performance 92.584 99.492 measures (ORYX) 2. Outpatient performance 91.190 91.473 measures (HEDIS like) Customer satisfaction 1. Patient satisfaction 265.670 267.586 2. Best places to work 58.254 64.734 a. Overall job satisfaction 3.631 3.737 b. Satisfaction with 3.461 3.605 organization c. Recommend my organization 3.751 3.872 as a good place to work 3. Registered nurse turnover 3.814 3.491 rate Ambulatory Care Sensitive 33.579 20.257 Condition hospitalizations Access 1. Primary care w ait time a. New primary care 98.857 99.740 appointments completed within 30 days from preferred date b. PCMH Access composite 40.996 53.800 i. Get an urgent care 43.785 59.919 appointment as soon as needed ii. Get a routine care 60.330 66.284 appointment as soon as needed 2. Specialty care wait time a. New specialty care 93.936 98.551 appointments completed within 30 days from preferred date 3. Mental health wait time a. New mental health 99.508 99.885 appointments completed within 30 days from preferred date 4. Call responsiveness a. Call center speed in 114.648 19.052 responding to calls in seconds b. Call center abandonment 24.943 3.359 rate Mental Health -0.204 1.130 1. Population coverage -0.407 1.147 2. Continuity of care -0.579 0.994 3. Experience of care 0.586 1.023 Efficiency (1/SFA) 91.466 96.093 Measure 10th-50th-90th ptile Acute care mortality 1. Acute care Standardized 0.483 - 0.877 - 1.178 Mortality Ratio (SMR) 2. Acute care 30-day 0.731 - 0.955 - 1.192 Standardized Mortality Ratio (SMR30) Avoidable adverse events 1. In-hospital complications 0.339 - 1.052 - 1.523 2. Health care associated infections (HAI) a. Catheter associated 0.000 - 1.013 - 3.013 urinary tract infection b. Central line associated 0.000 - 0.478 - 1.471 bloodstream infection c. Ventilator associated 0.000 - 0.000 - 3.442 Pneumonia d. Methicillin-resistant 0.000 - 0.082 - 0.284 Staphylococcus aureus (MRSA) infection 3. Patient safety indicator 0.000 - 0.759 - 1.184 (PSI) CMS 30-day Risk Standardized Mortality Rate (RSMR) 1. AMI RSMR 2. CHF RSMR 6.469 - 7.508 - 8.850 3. Pneumonia RSMR 7.474 - 9.152 - 11.290 CMS 30-day Risk Standardized Readmission Rate (RSRR) 1. AMI RSRR 16.336 - 16.386 - 16.451 2. CHF RSRR 17.892 - 19.422 - 21.678 3. Pneumonia RSRR 13.507 - 14.956 - 16.540 Adjusted length of stay 3.674 - 4.500 - 5.395 Performance measures 1. Inpatient performance 95.407 - 97.734 - 99.492 measures (ORYX) 2. Outpatient performance 87.271 - 89.457 - 91.473 measures (HEDIS like) Customer satisfaction 1. Patient satisfaction 238.804 - 256.250 - 267.586 2. Best places to work 49.650 - 58.185 - 64.734 a. Overall job satisfaction 3.448 - 3.609 - 3.737 b. Satisfaction with 3.150 - 3.410 - 3.605 organization c. Recommend my organization 3.430 - 3.659 - 3.872 as a good place to work 3. Registered nurse turnover 3.491 - 6.242 - 11.213 rate Ambulatory Care Sensitive 20.257 - 26.169 - 32.975 Condition hospitalizations Access 1. Primary care w ait time a. New primary care 83.536 - 97.367 - 99.740 appointments completed within 30 days from preferred date b. PCMH Access composite 32.078 - 42.025 - 53.800 i. Get an urgent care 30.553 - 45.549 - 59.919 appointment as soon as needed ii. Get a routine care 41.417 - 54.364 - 66.284 appointment as soon as needed 2. Specialty care wait time a. New specialty care 89.604 - 95.595 - 98.551 appointments completed within 30 days from preferred date 3. Mental health wait time a. New mental health 96.667 - 99.196 - 99.885 appointments completed within 30 days from preferred date 4. Call responsiveness a. Call center speed in 19.052 - 58.461 - 195.731 responding to calls in seconds b. Call center abandonment 3.359 - 8.918 - 22.063 rate Mental Health -1.351 - 0.094 - 1.130 1. Population coverage -1.194 - 0.009 - 1.147 2. Continuity of care -1.167 - 0.083 - 0.994 3. Experience of care -1.153 - 0.046 - 1.023 Efficiency (1/SFA) 91.008 - 94.386 - 96.093 Figure 9. Color-coded visualization tool Erie Hampton Marion St. Louis Wichita SMR30 63.028% 90.683% 5% 35% 46.719% IHC 100% 73.511% 20.952% 21.275% 28.528% CAUTI 90% 90% 46.49% 49.58% 90% CLABI No Data 17.74% 90% 5% 90% MRSA 90% 34.182% 37.34% 30.727% 10% PSI 10% 5% 90% 65.827% 49.097% Table 1. Correlation Values IHC HAI--CAUTI HAI--CLABI HAI--MRSA PSI Marion 0.4725 -0.1424 Missing Data 0.1682 -0.0573
|Printer friendly Cite/link Email Feedback|
|Author:||Mahajan, Ajay; Selvaganesan, Padmini; Madhani, Parag; Chitikeshi, Sanjeevi|
|Publication:||Journal of the Mississippi Academy of Sciences|
|Date:||Apr 1, 2018|
|Previous Article:||PULSED LASER DEPOSITION OF BIOACTIVE COATING FROM WHITE PORTLAND CEMENT.|
|Next Article:||THE EFFECTIVENESS OF DRY NEEDLING ON THE REDUCTION OF PROXIMAL UPPER QUARANT PAIN USING COHEN'S d: A SYSTEMATIC REVIEW.|