Designing standardized patient assessments to measure SBIRT skills for residents: a literature review and case study.
Objectives: Resident physicians report insufficient experience caring for patients with substance use disorders (SUDs). Resident training in Screening, Brief Intervention, and Referral to Treatment (SBIRT) has been recommended. We describe the development of a standardized patient (SP) assessment to measure SBIRT skills, resident perceptions of the exercise, and confidence in SBIRT skills.
Methods: Fifteen Internal Medicine residents participated in the pre-curriculum SP assessment and 12 participated in the post-curriculum assessment.
Results: Residents reported that SP encounters were similar to patients seen, and resident satisfaction was high. Residents felt confident screening for alcohol abuse, but less confident developing SUD treatment plans.
Conclusion: An SP assessment can evaluate SBIRT skills and is well received. Residents may need additional practice to improve confidence in making SUD treatment plans.
Key words: SBIRT, graduate medical education, standardized patients, substance use disorders
While substance use disorders (SUDs) and other forms of problematic substance use (e.g., illicit drug use, misuse of prescription drugs, and hazardous drinking) pose critical public health problems, many patients do not seek treatment and remain unidentified even while receiving medical care (Madras et al., 2009). Awareness of this gap led to the development of techniques to promote clinical attention in general medical settings to substance use. In recent years, empirical research and policy leaders in the SUD treatment field have encouraged the use of Screening, Brief Intervention, and Referral to Treatment (SBIRT) as a public health approach to improving patient care. SBIRT is a "comprehensive and integrated approach to the delivery of early intervention and treatment services through universal screening for persons with substance use disorders and those at risk" (Babor et al., 2007, p. 8). The U.S. Preventive Services Task Force has established the efficacy of screening and counseling for alcohol use disorders and recommends universal, annual screening for all adults (United States Preventive Services Task Force, 2004). SBIRT yields short-term improvements in rates of drug use and hazardous drinking, although researchers have not yet demonstrated long-term effects in controlled trials (Babor et al., 2007).
In spite of the evidence for SBIRT efficacy, physicians-in-training (resident physicians) may be reluctant to adopt SBIRT due to insufficient training in screening and treatment procedures (Stimmel, Cohen, Colliver, & Swarz, 2002). To address these barriers, core SUD competencies have been proposed for all primary care resident physicians (Jackson, Alford, Dube, & Saitz, 2010; Office of National Drug Control Policy, 2004), and a number of institutions have increased substance use training for medical students and/or residents (Chossis et al., 2007; Parish, Ramaswamy, Stein, Kachur, & Arnsten, 2006; Kokotailo, Langhough, Neary, Matson, & Flemming, 1995; Wilk & Jensen, 2002). Instruction in SBIRT has been suggested as a standard feature of residency training programs (O'Connor, Nyquist, & McLellan, 2011). However, challenges remain regarding how to best evaluate the efficacy of SBIRT curricula and the competency of trainees in SBIRT skills.
There are several potential options to consider when assessing residents' competency in conducting SBIRT (Jackson et al., 2010; O'Connor et al., 2011). One possibility is direct observation, using tools such as the "mini-clinical evaluation exercise" (mini-CEX) in which an attending physician observes a resident for a period of 10-20 minutes and provides feedback on history taking, physical examination and patient interaction skills (Norcini, Blank, Duffy, & Fortna, 2003). However, direct observation is patient-dependent and for any given clinical encounter, the opportunity to evaluate SBIRT skills may be limited. Additionally, inter-rater variability and content specificity are problems inherent in the mini-CEX (Holmboe & Hawkins, 1998). While chart review could assess rates of screening, brief intervention, and referral to treatment, medical records are often incomplete, may not adequately document visit content or capture the complexities of patient-physician communication required for SBIRT. Resident self-report of changes in substance use treatment competencies and self-efficacy are also commonly used to assess curricula, but may not provide valid measures of changes in clinical skills (Parish et al., 2006).
Standardized patient (SP) assessments have been proposed as a way to assess resident competencies in substance abuse treatment (Jackson et al., 2010; O'Connor et al., 2011). SP assessments evaluate learner performance in a controlled clinical setting designed to mimic an actual clinical encounter. Studies show that under certain circumstances, SP assessments are judged to be as reliable to assess learner performance as ratings of directly observed encounters with real patients (Wass, Jones, & Van der Vleuten, 2001). At least 80% of medical schools use SPs in the training and education of medical students, (Brownell, Stillman, & Wang, 1994) and residency programs are increasingly integrating this method into their evaluation of residents to demonstrate clinical competencies mandated by accrediting bodies (Accreditation Council for Graduate Medical Education).
SPs have been used to assess the effect of substance abuse training programs among resident physicians (Kokotailo et al., 1995; Levin, Owen, Stinchfield, Rabinowitz, & Pace, 1999; MacLeod, Hungerford, Dunn, & Hartzler, 2008; Ockene, Wheeler, Adams, Hurley, & Hebert, 1997; Wilk & Jensen, 2002), to assess and teach competencies in addiction medicine to internal medicine residents (Parish et al., 2006), and to assess resident physician behaviors towards patients with substance use (Wilson, Kahan, Liu, Brewster, Sobell, & Sobell, 2002). However, to date, there has been only one published description of SPs used specifically to evaluate SBIRT skills and that study examined SBIRT only for alcohol use and did not include resident learners (Fleming et al., 2009).
This report addresses this gap in the literature. We describe the development and implementation of an SP assessment in an internal medicine residency program to assess the impact of a SBIRT curriculum, as well as the results of a survey of the residents who participated in SP-based training. The survey focused on resident satisfaction, confidence in SBIRT skills, and perceived similarity of SP cases to real-world clinic interactions. Using our development process as a case study, we discuss common issues and challenges that arise in developing and implementing an SBIRT SP assessment program and offer recommendations for enhancing resident training.
The goal for our SP assessment was to evaluate the impact of a SBIRT curriculum for primary care internal medicine residents, focusing specifically on SBIRT knowledge and skills. A secondary goal was to provide an opportunity for residents to receive direct feedback from faculty about their skills. Our SBIRT SP team included faculty with expertise in addiction medicine and addiction psychiatry, a faculty member with expertise in developing SP cases, an educational specialist with assessment expertise, an internal medicine residency program director, an internal medicine resident and a fourth-year medical student with an interest in medical education. Our team worked collaboratively with two SP trainers with extensive experience in developing and running Objective Structured Clinical Examinations (OSCEs) at our institution. Details of the cases have been published elsewhere (Wamsley et al., 2011).
Developing the SP Assessment Blueprint
A "blueprint" for the content and objectives for each SBIRT SP case is presented in Table 1. Ideally, for assessment purposes, an SP assessment should include a minimum of 10 cases over 3 to 4 hours to achieve a score with a reliability of 0.85 to 0.90 (Reznick et al., 1993). This length overcomes content specificity, which is considered to be the major source of measurement error in SP assessments (Van der Vleuten & Swanson, 1990). However, because we focused on a circumscribed set of skills, we limited our SBIRT SP assessment to 3 cases.
We wanted residents to apply SBIRT skills to a variety of different situations, ranging from at-risk substance use to addiction, and for differing stages of readiness to change substance use behavior (e.g., pre-contemplative to preparation) (Prochaska, DiClemente, & Norcross, 1992; Samet & O'Connor, 1998). Cases included a range of substances, medical and mental health comorbidities, and patient ages. By developing moderately complex cases in which substance use was one salient aspect of each patient's overall clinical presentation, we sought to mimic the outpatient clinic setting in which residents would apply SBIRT skills on a day-today basis. Due to the complexity of the cases and our desire that residents have sufficient time to apply SBIRT skills successfully, we chose 20 minutes as an optimal length. Because our primary focus was on resident ability to screen for substance use, perform brief interventions and refer to treatment for substance use disorders, we did not include physical examinations.
SP Case Development
The SBIRT SP assessment team used our collective experience and expertise in addiction medicine to create cases that were realistic and contained scripted details for each SP's presenting problem, medical history, substance use patterns, social history and presentation, and general emotional tone. For one of our cases ("Katie Boyle") we modified a pre-existing case developed to teach internal medicine and family medicine residents about substance abuse (Parish et al., 2006). In developing all cases, we utilized the outline suggested for standardized patient cases from the Association of Standardized Patient Educators (ASPE) (Association of Standardized Patient Educators, 2009).
Measures of Performance
SP Assessment Checklist: We developed assessment checklist items to measure resident SBIRT performance, consistent with the objectives for each case. Checklists for individual cases in our SBIRT SP assessment ranged from 28 to 33 items, including 24 to 27 items that were focused on key SBIRT skills. We focused on key SBIRT skills since too many checklist items may negatively impact inter-rater reliability and the validity of the score from the station (Wilkinson, Frampton, Thompson-Fawcett, & Egan, 2003). SPs completed checklists because previous studies suggest that adequate inter-rater agreement can be achieved through the use of SPs or physician raters, provided that SPs receive adequate instruction on completing checklists (Van der Vleuten & Swanson, 1990). Although global ratings have been found to be as reliable or more reliable than checklists when used by trained examiners and SPs (Regehr, Freeman, Robb, Missiha, & Heisey, 1999), because we were interested in resident use of specific SBIRT skills, the majority of items in each case were dichotomous checklist items ("Yes" or "No"). All cases did, however, include one global rating item to assess the standardized patient's satisfaction with the encounter and eight items utilizing a 6-point rating scale (Outstanding to Unacceptable) to better capture the complex interpersonal interactions between the resident provider and the SE
Post Case Exercises: To further assess resident SBIRT knowledge and skills we developed post-case exercises for residents to complete immediately after each SP encounter. The content for the exercises is detailed in Table 2. We created a scoring key to address the points that should have been included in the residents' written responses.
Resident Satisfaction: The residents completed an anonymous electronic survey to evaluate their satisfaction with the assessment. The survey contained 7 items regarding resident impressions of the SP assessment, including 2 global assessment items regarding the experience. All items were rated on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). Additionally, surveys included 2 open-ended questions, one asking about strengths and the other about weaknesses of using SPs as a learning tool for SBIRT.
Resident Self-Efficacy: Resident confidence was measured using an anonymous 4-item self-report survey. All items were rated on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree; higher score representing higher confidence). The items included confidence regarding alcohol and drug screening, assessing patient readiness to change, and creating treatment plans for patients with SUDs.
Statistical Analysis: Mean scores for each item on the resident satisfaction and self-efficacy surveys were calculated for participants in the pre-curriculum SP assessment and the post-curriculum SP assessment. Because the surveys were anonymous we could not match residents to conduct paired sample t-tests.
Standardized patients: Standardized patients received a total of 7 hours of training conducted by professional SP trainers and the SBIRT SP assessment team faculty. During these sessions, SPs role-played the case with faculty and practiced using the checklist form to ensure reliability. Faculty and SP trainers provided feedback to SPs regarding the accuracy of their portrayal of the case details and consistency in their use of the checklist. Minor modifications were made in the cases and checklists based on feedback from SPs and SP trainers.
Resident physicians: Fifteen Primary Care Internal Medicine residents participated in the pre-curriculum SBIRT SP assessment and 12 residents participated in the post-curriculum SBIRT SP assessment. For the pre-assessment, 66.7% of participants were women and 53.7% were in their third-year of residency. For the post assessment, 83% were women and 33.3% were in their third-year of residency.
SP Assessment Procedure
Residents participated in the SBIRT SP assessment prior to and after completing a 10-hour SBIRT curriculum dealing with hazardous drinking and substance use [Satterfield et al., 2012]. The pre and post SP assessments were separated by 10 months. The procedures described below were identical for the pre/post exercise. For each SP assessment, residents completed a 1.5hour session in which they performed three SP cases. Each SP case was 20 minutes in length and was followed by a 10-minute post-case exercise. Residents completed the satisfaction survey after the last SP post exercise and prior to the debriefing session. SBIRT SP assessment team faculty attended the SP assessment and monitored resident and SP performance remotely using video monitors. After all cases were completed, SBIRT faculty led a 50-minute debriefing session. During this session, faculty provided general feedback about resident performance in the SBIRT SP assessment, elicited resident feedback regarding the experience of participating in the exercise, and feedback about each case.
As described in Satterfield et al. (2012), residents demonstrated large improvements in SBIRT Knowledge, History Taking, Screening for Substance Use, and SUD diagnosis from the pre to the post-curriculum assessment. There was a moderate effect for Brief Intervention skill acquisition. Results of the satisfaction survey are detailed in Table 3. Residents reported that SP encounters were similar to patients typically seen in their clinic and that they had sufficient time for SP encounters and post-case exercises. Prior to the SBIRT curriculum, evaluations indicated that residents had mixed feelings about the use of SP assessments to teach clinical skills, the value of the SP assessment learning experience and whether they would recommend the exercise to fellow learners. The overall trend was toward more positive attitudes towards the SP experience after the post-curriculum assessment.
Residents commented that the strengths of the pre-curriculum assessment focused on the ability to practice skills in a safe environment with realistic cases. They also reported that by working with SPs, it highlighted SBIRT skills and how they felt less comfortable with them, but that it motivated them to practice these skills. Several residents expressed that they did not like SP assessments in general because they felt artificial. Comments regarding strengths of the post-curriculum SP assessment focused on the opportunity to practice skills learned with patients at various stages of change and to see improvement in their knowledge and skills from the pre-curriculum assessment to the post-curriculum assessment. Residents commented that they would have preferred different cases for the post-curriculum assessment to make the interactions seem more realistic.
Resident self-efficacy in SBIRT skills is detailed in Table 4. Residents expressed most confidence in screening for alcohol use disorders and expressed less confidence in developing treatment plans for patients with substance abuse. This was true even after participation in the 10-hour targeted curriculum and SBIRT SP assessment.
We successfully developed and implemented an SP assessment to evaluate resident knowledge of SBIRT and competence in SBIRT skills before and after a curricular intervention. Residents reported that the cases were realistic and representative of patients seen in practice and that a twenty-minute encounter time was adequate. The SBIRT SP assessment was acceptable to residents who reported that they saw value in working with SPs, particularly at the post-curriculum assessment when they were applying skills previously learned. Acceptability to learners and faculty is a key factor to consider when selecting an assessment method (Van der Vleuten, 1996). Our experience suggests that an SBIRT SP assessment can meet this criterion.
There are a number of additional benefits to using an SP assessment to assess resident competency in SBIRT skills and/ or the impact of an SBIRT curriculum. An SP assessment can be easily tailored to meet educational goals, allows for consistent case presentation and has been shown to be a reliable method of assessment (Epstein, 2007). Additionally, it targets the "shows how" level of Miller's pyramid: [the lowest level is knowledge (knows), followed by competence (knows how), performance (shows how) and action (does)]. It is optimal to select an assessment tool that targets the higher levels of Miller's pyramid when assessing competence (Miller, 1990). As noted in the introduction, other methods have shortcomings that make SPs attractive.
An SBIRT SP assessment provides residents an opportunity to practice SBIRT skills with realistic cases in a safe environment and to receive feedback. Residents expressed that practicing core SBIRT skills was a major strength of the exercise. Previous research suggests that resident performance in a substance abuse OSCE improves from the first to final OSCE case (Parish et al., 2006). This suggests that SBIRT SP assessments, whereby learners are focusing solely on screening, brief intervention and referral to treatment for substance use disorders, can be used for both formative (guiding future learning, providing feedback to learners and shaping values) and summative (making an overall judgment about competence) purposes, and that an SBIRT SP assessment may have a direct impact on future learning and practice.
However, despite participation in a 10-hour targeted curriculum and practice during the SP assessment, residents expressed a lack of confidence in making treatment plans for patients with substance use disorders. Anecdotally, residents pointed to the complexities of the local referral systems and the limited access to specialty SUD care even for well-insured patients. Residents may require additional practice and/or reinforcement of SBIRT skills to enhance their confidence in making treatment plans and accessing local SUD specialty care programs. Alternatively, self-efficacy may not be well correlated with actual skills. Parish et. al. (2006) found that resident-perceived competence in treating substance abuse did not correlate with performance in a substance abuse OSCE.
Although there are benefits to using an SP assessment to evaluate SBIRT competencies in resident learners, there are potential limitations as well. Cost remains a barrier to implementation of OSCE assessments (Cusimano, Cohen, Tucker, Kodama, & Reznick, 1994). Our SBIRT SP assessment required over 500 person-hours to develop and implement, including significant faculty and administrative time (Table 5). However, a large proportion of these costs related to time spent to develop and refine the SP cases. If programs used cases that had already been developed for this purpose, the costs of the SP assessment would be substantially less. Additionally, an SP assessment is most easily undertaken in a specialized facility with capacity for remote observation by faculty and recording of encounters; not all training programs have access to such facilities.
There are several limitations to the SBIRT SP assessment we developed. Our assessment only included 3 cases and was 1.5 hours in length. Previous studies suggest that an OSCE should be a minimum of 3-4 hours to overcome case specificity (Van der Vleuten & Swanson, 1990). However, because our assessment focused on a specific set of skills that were the same across cases, case specificity is less of a concern. While we sought to create a diverse representation of patients, having a small number of cases necessarily limits the range of patient demographic characteristics, and the alcohol and drug use profiles are included. Another limitation is that we were unable to conduct reliability analyses for checklist items due to the limited number of residents who participated in the exercise. For future studies with larger sample sizes, this would be an important area of focus.
An SBIRT SP assessment is one method to assess both the impact of an SBIRT curriculum and competency in SBIRT skills. Based on our experience in a primary care internal medicine residency-training program, SP sessions are a useful complement to classroom-based instruction. SP assessment is acceptable to learners and provides an opportunity for learners to practice skills in a safe environment and can be used for both formative and evaluative purposes.
CONFLICT OF INTEREST
None of the authors have any conflicts of interest to report.
Corresponding author at: Maria A. Wamsley, M.D., Division of General Internal Medicine, University of California, 1545 Divisadero St., Box 0320, San Francisco, CA94143, USA. Phone 415-514-8660. Fax 415-353-2640. Email: email@example.com
This research project was supported by a SAMHSA/CSAT Grant U79T1020295.
Association of Standardized Patient Educators. (2009). Recommended standardized patient case outline. Retrieved March 4, 2013 from http://aspeducators.org/img/mededportal%20submit_criteria(1).pdf.
Babor, T.F., McRee, B.G., Kassebaum, P.A., Grimaldi, P.L., Ahmed, K., & Bray, J. (2007). Screening, brief intervention, and referral to treatment (SBIRT): Toward a public health approach to the management of substance abuse. Substance Abuse, 28(3), 7-30.
Brownell, M., Stillman, P., & Wang, Y. (1994). Growing use of standardized patients in teaching and evaluation in medical education. Teaching and Learning in Medicine, 6, 15-22:
Chossis, I., Lane, C., Gache, P., Michaud, P.A., P6coud, A. Rollnick, S., & Daeppen, J.B. (2007). Effect of training on primary care residents' performance in brief alcohol intervention: a randomized controlled trial. Journal of General Internal Medicine, 22(8), 1144-1149.
Cusimano, M.D., Cohen, R., Tucker, W., Kodama, R., & Reznick, R. (1994). A comparative analysis of the costs of administration of an OSCE (objective structured clinical examination). Academic Medicine, 69, 571-576.
Epstein, R.M. (2007). Assessment in medical education. New England Journal of Medicine, 356, 387-396.
Fleming, M., Oisen, D., Stathes, H., Boteler, L., Grossberg, R, Pfeifer, J., Skochelak, S. (2009). Virtual reality skills training for health professionals in alcohol screening and brief intervention. Journal of the American Board of Family Medicine, 22(4), 387-398.
Holmboe, E.S., Hawkins, R.E. (1998). Methods for evaluating the clinical competence of residents in internal medicine: a review. Annals of Internal Medicine, 129, 42-48.
Jackson, A.H., Alford, D.R, Dube, C.E., & Saitz, R. (2010). Internal medicine residency training for unhealthy alcohol and other drug use: Recommendations for curriculum design. BMC Medical Education, 10, 22. doi:10.1186/1472-6920-10-22
Kokotailo, P.K., Langhough, R., Neary, E.J., Matson, S.C., & Flemming, M.F. (1995). Improving pediatric residents' alcohol and other drug use clinical skills: Use of an experiential curriculum. Pediatrics, 96(1), 99-104.
Levin, F.R., Owen, R, Stinchfield, R., Rabinowitz, E., & Pace, N. (1999). Use of standardized patients to evaluate the physicians in residence program: A substance abuse training approach. Journal of Addictive Disease, 18, 39-50.
MacLeod, J.B., Hungerford, D.W., Dunn, C., & Hartzler, B. (2008). Evaluation of training of surgery interns to perform brief alcohol interventions for trauma patients. Journal of American College Surgeons, 207, 639-645. doi: 10.1016/j.jamcollsurg.2008.06.327
Madras, B.K., Compton, W.M., Ayula, D., Stegbauer, T., Stein, J.B., & Clark, H.W. (2009). Screening, brief interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare sites: Comparison at intake and 6 months later. Drug and Alcohol Dependence, 99(1-3), 280-295.
Miller, G.E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9 Suppl), $63-$67.
Norcini, J.J., Blank, L.L., Duffy, F.D., & Fortna, G.S. (2003). The mini-CEX: A method for assessing clinical skills. Annals of Internal Medicine, 138, 476-481.
Ockene, J.K., Wheeler, E.V., Adams, A., Hurley, T., & Hebert, J. (1997). Provider training for patient-centered alcohol counseling in a primary care setting. Archives of Internal Medicine, 157, 2334-2341.
O'Connor, P.G., Nyquist, J.G., & McLellan, T. (2011). Integrating addiction medicine into graduate medical education in primary care: The time has come. Annals of Internal Medicine, 154, 56-59.
Office of National Drug Control Policy. (2004). Leadership conference on medical education in substance abuse. Washington, DC. Retrieved 4 March 2013 from http://www.ncjrs.gov/ondcppubs/publications/pdf/medical_educ_2004.pdf.
Parish, S.J., Ramaswamy, M., Stein, M.R., Kachur, E.K., & Arnsten, J.H. (2006). Teaching about substance abuse with objective structured clinical exams. Journal of General Internal Medicine, 21, 453-459.
Prochaska, J.O., DiClemente, C.C., & Norcross, J.C. (1992). In search of how people change: Applications to addictive behaviours. American Psychologist, 47, 1102-1114.
Regehr, G., Freeman, R., Robb, A., Missiha, N., & Heisey, R. (1999). OSCE performance evaluations made by standardized patients: Comparing checklist and global rating scores. Academic Medicine, 74, S135-S137.
Reznick, R.K., Blackmore, D., Cohen, R., Baumber, B., Rothman, A., Smee, S., Walsh, P. (1993). An objective structured clinical examination for the licentiate of the Medical Council of Canada: From research to reality. Academic Medicine, 68(10 Suppl), $4-$6.
Samet J.H., & O'Connor, P. (1998). Alcohol abusers in primary care: Readiness to change behavior. American Journal of Medicine,105, 302-306.
Satterfield, J.M., O'Sullivan, P., Satre, D., McCance-Katz, E., Batki, S., Tsoh, J., Wamsley, M. (2012). Using standardized-patients to evaluate screening, brief intervention, and referral to treatment (SBIRT) knowledge and skill acquisition for internal medicine residents. Substance Abuse, 33 (3), 303-307.
Stimmel, B., Cohen, D., Colliver, J., & Swarz, M. (2002). An assessment of house staff's knowledge of alcohol and substance abuse utilizing standardized patients. Substance Abuse, 21, 1-7.
United States Preventive Services Task Force: Screening and behavioral counseling interventions in primary care to reduce alcohol misuse recommendation statement. (2004). Annals of Internal Medicine, 140, 555-557.
Van der Vleuten, C.EM. (1996). The assessment of professional competence: Developments, research and practical implications. Advances in Health Science Education, 1, 41-67.
Van der Vleuten, C.P.M., & Swanson, D.B. (1990). Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine, 2, 58-76.
Wamsley, M., Treit, K., Satterfield, J., Levitt, D., McCance-Katz, E., Moreno-John, G., O'Sullivan, P. (2011). Three standardized patient cases to measure screening, brief intervention and referral to treatment (SBIRT) skills in primary care residents. Retrieved 4 March 2013 from MedEdPortal www.mededportal.org/publication/9005.
Wass, V., Jones, R., & Van der Vleuten, C. (2001) Standardized or real patients to test clinical competence? The long case revisited. Medical Education, 35, 21-325.
Wilk, A.I., & Jensen, N.M. (2002). Investigation of a brief teaching encounter using standardized patients. Journal of General Internal Medicine, 17, 356-360.
Wilkinson, T.J., Frampton, C.M., Thompson-Fawcett, M., & Egan, T. (2003). Objectivity in objective structured clinical examinations: Checklists are no substitute for examiner commitment. Academic Medicine, 78, 219-223.
Wilson, L., Kahan, M., Liu, E., Brewster, J.M., Sobell, M.B., & Sobell, L.C. (2002). Physician behavior towards male and female problem drinkers: A controlled study using simulated patients. Journal of Addictive Diseases, 21, 87-99.
Maria A. Wamsley, MD, Katherine A. Julian, MD, Patricia O'Sullivan, EdD & Jason M. Satterfield, PhD (1)
Division of General Internal Medicine
University of California, San Francisco
Derek D. Satre, PhD, Elinore McCance-Katz, MD, PhD & Steven L. Batki, MD
Department of Psychiatry
University of California, San Francisco
TABLE 1 SBIRT Standardized Patient (SP) Assessment Blueprint Stage of Patient Profile Change Case objectives for trainee Case 1: Michael 1) Screen for alcohol and Smith substance use Age 35 2) Screen for coexisting Binge-drinking Contemplative depression and anxiety Paroxysmal atrial 3) Obtain substance use fibrillation history 4) Assess binge-drinking 5) Deliver brief intervention to address binge drinking 6) Use effective communication skills around substance use Case 2: Katie Boyle 1) Screen for alcohol and Age 63 substance use Alcohol use disorder 2) Screen for coexisting Depression Preparation depression and anxiety Upper 3) Obtain substance use gastrointestinal history bleed 4) Assess for alcohol-use disorder 5) Deliver brief intervention to address alcohol use disorder 6) Referral to treatment for an alcohol use disorder 7) Use effective communication skills around substance use Case 3: Molly Bunk 1) Screen for alcohol and Age 39 substance use Prescription drug 2) Screen for coexisting misuse Pre- depression and anxiety Chronic low back contemplative 3) Obtain substance use pain history 4) Assess for prescription drug misuse 5) Deliver brief intervention to address prescription drug misuse 6) Use effective communication skills around substance use TABLE 2 Post-case Exercises for Medical Residents Completing Standardized Patient (SP) Assessment Post-case Exercise Objectives Case 1 - Michael Smith Appropriate documentation of substance use in a patient encounter Case 2 - Katie Boyle Knowledge of brief intervention strategies Knowledge of how ethnicity/race/age affects assessment/treatment of substance use Knowledge of treatment referral options Case 3 - Molly Bunk Appropriate documentation of substance use in a patient encounter Knowledge of specific tools to use when prescription drug use is suspected (pain medication contracts, random drug screening) TABLE 3 Resident Satisfaction with Standardized Patient (SP) Assessment Pre-and Post-SBIRT Training Pre Post (n=15) (n=12) Std. Question Mean * Std. Dev. Mean Dev. The goals of this learning 3.47 0.92 4.25 1.14 sessions were made clear to me The scenarios were realistic 3.67 0.98 4.42 0.79 and representative of patients I might see in clinic I felt I had an appropriate 4.20 0.94 4.17 0.58 amount of time for the standardized patient encounters I felt I had an appropriate 4.40 0.63 4.33 0.65 amount of time to complete the interstation (post-case) exercises Standardized patients should 3.67 1.18 4.08 0.90 be used to teach residents clinical skills I felt this was a valuable 3.67 1.23 4.08 1.00 learning experience I would recommend this 3.60 1.24 4.08 1.00 experience to my fellow residents Notes: * We used a five-point Likert scale in which 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree. TABLE 4 Participants' Evaluation of their Self-Efficacy Pre-and Post-Standardized Patient (SP) Examination Pre (n=15) Post (n=12) Question Mean Std. Dev. Mean Std. Dev. I feel confident screening 3.73 .961 4.17 .58 patients for alcohol abuse I feel confident screening 3.60 .986 3.83 .72 patients for drug use 1 feel confident assessing 3.60 .910 3.92 .90 a patient's stage of change 1 feel confident making a 3.13 .990 3.17 .84 treatment plan for patients with substance use disorders Notes: * We used a five-point Likert scale in which 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree 5 = strongly agree. TABLE 5 SBIRT Standardized Patient (SP) Assessment Resources: Person Hours * Task Person Time (Hours) Developing 3 SP cases Faculty 120 SP Trainers 30 Training 3 SP cases * Faculty 21 SP Trainers 42 SPs 126 SP Assessment * Faculty 28 Administrative support 40 SPs 108 Total Hours 515 * Note: Hours are assuming 3 SP cases per SBIRT SP assessment, 4 sessions of the SP assessment for 9 residents each session (total of 18 residents pre-curriculum, 18 residents post-curriculum)
|Printer friendly Cite/link Email Feedback|
|Author:||Wamsley, Maria A.; Julian, Katherine A.; O'Sullivan, Patricia; Satterfield, Jason M.; Satre, Derek D|
|Publication:||Journal of Alcohol & Drug Education|
|Date:||Apr 1, 2013|
|Previous Article:||The impact of reality television on the alcohol-related beliefs and behaviors of Hispanic college students.|
|Next Article:||Friends don't let friends drive drunk, but do they let friends drive high?|