Implementation Findings From an Effectiveness-Implementation Trial of Tablet-Based Parent Training in Pediatric Primary Care.
The ezParent program is a six-module, tablet-based PT program adapted from the group-based Chicago Parent Program (Breitenstein, Fogg, Ocampo, Acosta, & Gross, 2016). ezParent was developed to be culturally and contextually relevant for low-income, ethnic minority families of children ages 2-5 years. ezParent helps parents develop positive and effective parenting skills and decrease physical punishment through use of behavioral strategies (e.g., routines, labeled praise), videos of parents using the strategies, activities, quizzes, and assignments. In a previous randomized controlled trial (RCT; n = 79 parents), 85% completed all six program modules, 88% reported that ezParent was very helpful, and 82% would highly recommend the program (Breitenstein et al., 2016). Modest improvements in parenting and child outcomes (Cohen's d = .14-.31) are consistent with universal primary prevention program effects (Tanner-Smith, Durlak, & Marx, 2018).
PT in PPC
PPC is an ideal setting for providing PT because it offers a consistent and supportive context (Perrin, Leslie, & Boat, 2016). PPC is accessible to parents and has an existing infrastructure for disseminating information, and parents view PPC providers as trusted sources of information (McLearn et al., 2004). PPC providers are often the first professionals that families approach regarding parenting concerns or child behavior problems (Berkout & Gross, 2013). Recent American Academy of Pediatrics guidelines for decreasing corporal punishment highlight the need for accessibility of resources and programs like ezParent in PPC (Sege, Siegel, & Council on Child Abuse and Neglect, 2018).
Expanding PT in PPC can maximize the public health impact of these interventions by improving access and reach. However, interventions need to be easily implemented to avoid adding to an already burdened system (Leslie et al., 2016; Perrin et al., 2016). There is a need identify implementation processes to create organizational change (Brown, Raglin Bignall, & Ammerman, 2018), and studying PT implementation in PPC provides critical information to identify and address barriers and facilitators and promote sustainability efforts.
Hybrid Type I Effectiveness-Implementation Design
The three types of hybrid effectiveness-implementation designs vary on the emphasis placed on effectiveness testing and implementation evaluation (Curran, Bauer, Mittman, Pyne, & Stetler, 2012). This Type I study design employs a rigorous test of intervention effectiveness while collecting implementation data for feasibility and acceptability of implementation (Curran et al., 2012). In this article, we report on the implementation processes in PPC using
RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance; Glasgow & Estabrooks, 2018). We report encountered barriers and facilitators to implementation. This implementation evaluation will inform future efforts to add PT to standard practice in primary care.
Study Design and Context
A descriptive design using RE-AIM guided the evaluation of implementation in PPC. We evaluated provider practice of introducing the study to parents of children ages 2-5 years during a well-child visit (WCV). This introduction served as proxy for recommending the intervention. The implementation preceded individual-level randomization to ezParent or the control program. We report on reach, adoption, implementation, and maintenance components that were assessed using quantitative measures.
Implementation occurred between April 2016 and April 2018 in four urban PPC sites that served predominantly low-income Chicago communities. All sites report a diverse and a large Medicaid/underserved population base. PPC1 is a medicine-pediatrics practice, PPC3 a family medicine practice, and PPC2 and PPC4 are general pediatric clinics.
Implementation Approach and Procedures
The implementation approach for introduction to the study was tailored to each of the PPC sites. As part of implementation agreements with each site, we identified key stakeholders (providers, staff, and administration). We worked closely with the stakeholders to understand individual practice environments and workflows and to develop tailored implementation plans. Across all sites, PPC providers introduced the study to parents as part of a WCV, allowing integration into the workflow and approximate the process that would occur in practice. The Consolidate Framework for Implementation Research (Keith, Crosson, O'Malley, Cromp, & Taylor, 2017) and a review of strategies for implementation in health care (Powell et al., 2015) guided our implementation approach.
Once the implementation approach was finalized (see Table 1), we developed scripts and training materials. All sites provided parents with written information describing the study and an interest form with contact information for parents to complete. Two sites (PPC1 and PPC4) included the study material in the packets given to parents at their WCV. In the two other sites (PPC2 and PPC3), the staff who brought the family to the exam room provided the study information sheet. Therefore, all parents should have the study information when the provider entered the room. The providers briefly (< 2 min) presented the opportunity to receive parent support via tablet-based apps and to participate in the study. After the visit, the providers completed tracking logs (paper or electronic health record [EHR]) to report compliance with the implementation procedures. Implementation was planned for 10 months at each site, and start times were staggered by 4 months. Study procedures and protocols were approved by institutional review boards at the two primary institutions of the four PPC sites.
Measures and Data Sources
We collected data at three points: preimple-mentation, during implementation, and post-implementation.
Reach. To evaluate that we were reaching our intended population (i.e., parents of children ages 2-5 years), we measured the characteristics of parents from the practices as compared to those who enrolled in the study. All parents who presented with their 2- to 5-year-old child for a WCV were eligible. All data collected from the EHR were stripped of identifiers and aggregated.
Adoption. To evaluate adoption of the implementation procedures, we examined the characteristics of the PPC sites. We tracked the number of educational sessions to orient providers and staff to the project and implementation procedures. We assessed providers' current practices for responding to parent and child behavior concerns. Prior to the start of implementation, providers responded to the Primary Provider Survey (Metzler, Sanders, Rivara, Christakis, & Rusby, 2014), demographic questions, and 32 items assessing their current practice and attitudes in helping parents with managing child behavior problems. Item responses were on a 5-point scale assessing level of difficulty in addressing children's behavior problems, parenting concerns, confidence in managing these problems, and rating of the practice's capacity for dealing with parenting difficulties or child behavior problems.
Implementation. To track provider delivery, three sites used paper tracking logs and one site used the EHR to record whether the provider discussed and gave the study information to parents. If not discussed, providers recorded the reason. Variations in tracking procedures (e.g., paper tracking logs or EHR) occurred due to preference and administrative support to amend the EHR. We also tracked and monitored any adaptations made during delivery. Given variations in resident rotations, we were unable to identify the proportion of residents who implemented or maintained procedures; therefore, proportions are presented for staff providers.
Maintenance. To evaluate the maintenance of procedures, we developed a postimplementation survey to evaluate providers' knowledge and implementation of the procedures and assess barriers and facilitators to implementation. Providers responded to seven questions assessing their role and ease or difficulty of implementation of the procedures and rated their perception of implementing the study introduction into their day-to-day activities on a 5-point scale: not at all, only slightly, somewhat, quite, or very. A final open-ended question queried respondents if they had other thoughts or suggestions. We invited providers to complete the survey after the end of implementation.
For all analyses, descriptive statistics were calculated by clinic and overall. Given inherent differences between clinics, we did not assess between-clinic differences. However, we provide descriptive data by clinic to illustrate variability across sites. To evaluate reach, chi-square tests were used to compare the demographic characteristics of the study participants to the patient population.
For maintenance, we assessed only those providers who had five or more WCVs in at least 1 month during the implementation period. This approach was taken to ensure adequate exposure to the study and opportunities for implementation of the study protocol. All providers with fewer than five WCVs were either part-time staff or medical residents.
Across sites, the monthly number of WCVs for children ages 2-5 years ranged from 20 to 269. On average, providers introduced 14% (18/124) of eligible parents to the study each month. Of the parents introduced to the study, 78% (14/18) expressed interest in participating (see Table 2). In total, 759 parents signed forms indicating their interest in the study; however, the research staff never reached 29% (217/759) to screen for eligibility. Of those screened for eligibility, 3% (14/542) were ineligible, 44% (241/542) did not show up for their baseline appointment, and 53% (287/542) enrolled.
Although race/ethnicity of enrolled parents did not differ between sites, in two sites, the race distribution of those who enrolled differed from the race distribution of the practice. Specifically, in PPC2, enrolled parents were less likely to be White (12% vs. 23% in the practice) and more likely to be Black/African American (57% vs. 47%) or other (31% vs. 24%; p = .008). In PPC4, participants were more likely to be White (17% vs. 8%) and Black/African American (61% vs. 48%) and less likely to be other (23% vs. 37%; p = .002). There were no significant child gender differences between sites, however, PPC3 had fewer enrolled females (39% vs. 52%; p = .05).
All sites adopted their individualized implementation plan (see Table 3). We report responses from the staff MDs and nurse practioners (NPs) as they were primarily responsible for implementation. Across the four sites, 24 (67%) providers responded to the Primary Provider Survey. They reported an average of 14.94 (SD = 10.07) years of practice, 83% reported that it was slightly (58%) to very (25%) difficult to address parents' concerns about their children's behavior problems during WCVs, 79% reported that it was very difficult to make an appropriate referral for children's behavior problems, and 92% reported that they felt inadequately trained to effectively help parents to address behavioral or emotional problems. Providers rated the practices capacity for dealing effectively with parenting difficulties or children's behavior problems as poor (12%), fair (68%), and good (20%).
Across the four sites, we held an average of two sessions (range = 1-3) to orient providers and staff to the project and implementation procedures. In PPC2, we conducted one training session for the staff and nurses; however, we were unable to conduct direct training to the providers. Instead, our site liaison (author LP) introduced the study and process to the providers at a standing meeting and during practice times. In addition to the in-person trainings, we sent reminder e-mails to all providers and staff the day of the implementation launch, and in PPC2 and PPC3, we attended brief morning huddles to remind providers and staff of the start of the program.
Implementation feasibility was high across providers: 67-100% of staff providers reported that they implemented the procedures (see Table 4). Monthly implementation tracking was used to identify adaptations and methods to improve implementation. We provided e-mails to all providers and staff containing implementation rates, procedure reminders, encouragement, and positive reinforcement. Overall, adaptations in all of the sites were minor (e.g., placing forms in areas more visible, changing location of form collection boxes, and posting reminders in break rooms). In addition, our project staff were on site at the PPCs at least twice weekly to collect parent interest forms and support implementation.
We examined staff providers implementing at least one time (n = 28) to evaluate maintenance over time; of these, 68% maintained implementation for at least 6 months. Among those who implemented three or more times, 79% maintained implementation for 6 months (see Table 4). In addition to monthly tracking, at the end of implementation, providers responded to a postimplementation survey. Sixty-eight providers (32 staff MDs/NPs and 36 resident MDs) responded to the survey. Most (77%) respondents reported that they referred at least one parent to the project. One provider noted, "This was an excellent program and helpful for patients" and another stated that the program contained "very useful information for the patients." One provider noted being motivated by the parent report that "the modules helped them deal with their children better." Over half (59%) of respondents reported that they provided the parents with program materials, 57% reported that they explained the opportunity to parents, and 27% reported that they answered parents' questions regarding the program. Providers commented that barriers to implementation related to inconsistencies in maintaining protocols ("materials were inconsistently part of the well-child packets" and "patients did not bring the forms into the exam room"). One provider commented that "more information about the interventions themselves would be helpful." Another recommended that "MDs do not have reasonable time to perform necessary introduction and stay efficient." Finally, several residents commented that it was easier to implement when their supervisor promoted the program.
Despite the majority of providers (69%) endorsing the importance and appropriateness of providing parenting resources in PPC, nearly 25% reported they did not participate in any implementation activities. This is consistent with daily tracking (53% of providers implemented the procedures more than once). Of the providers (23%) who reported that they did not participate in the implementation activities, the reasons (multiple endorsements allowed) included unaware of the program (19%), inadequate time (31%), did not understand procedures (50%), forgot (38%), the parent did not receive materials (63%), and not a priority (13%). As one provider noted, "Within a 20-minute time period...I am to take a history, do an exam...fill out WIC and school forms, log into the computer, enter orders, explain vaccines and make an attempt to be civil and engaging. On many days, we are running behind, so I don't even really have my full 20 minutes. Therefore, I had no time to add more to my work."
In this section, we review relevant findings from the RE-AIM evaluation and discuss factors that contributed to implementation success, challenges, and recommendations for future efforts.
A compelling finding was that we did not adequately reach the target parents for this program. On average, only 4% of potentially eligible parents enrolled in the study. One factor to this low level of reach is related to implementation breakdowns and that providers introduced only 14% of eligible parents to the program. Despite careful planning, we encountered challenges consistent with other studies of brief intervention in PPC, including time, competing priorities, and lack of information/communication about the program (King, Muzaffar, & George, 2009; Rahm et al., 2015).
Practices (PPC2, PPC3, and PPC4) that conducted more monthly WCVs for children aged 2-5 years had lower rates of program introduction than PPC1 (M = 20 visits/month). One explanation is that smaller practices have fewer moving parts and greater ability to integrate new procedures. Relative to the other sites, PPC1 had fewer residents in the practice, the RN director provided support to providers with verbal reminders prior to a WCV, and the providers documented adherence to implementation in the EHR. Successful implementation requires this type of holistic approach to alter provider habits and increase implementation (Johnson & May, 2015). As identified by a resident, if the attending physician supervising them did not promote the program, they were unlikely to do so. This further complicates implementation maintenance as residents rotate frequently through practice settings. Furthermore, 60% of providers indicated that materials were not in the packet or provided to the patient. Thus, even if providers wanted to introduce the program, the lack of materials may have prohibited it. Reminders to include this information for the providers could increase reach.
PPC1 was the only practice that was able to use the EHR to document implementation. We were not able to track whether providers in the other PPCs forgot to fill out the paper tracking logs, making it possible that we have underestimated provider implementation (e.g., providers introduced the study but failed to report that they did). This highlights the benefit of having the cue to integrate the introduction to the program as part of anticipatory guidance procedures in the EHR. Offering universal PT as part of anticipatory guidance aligns with American Academy of Pediatrics recommendations for promoting family support and child development (Duncan et al., 2015). It is also possible that administrative support to amend the EHR for this project signals a higher level of support and buy-in for interventions.
Another implementation challenge was the context of an RCT. Because providers introduced the study prior to randomization, we did not provide comprehensive information to the providers regarding ezParent, and there were multiple steps required for parents to enroll (e.g., complete interest form, set up baseline appointment) that may have created provider and parent barriers. During training sessions, staff and providers received minimal information about ezParent and the control condition (e.g., parents would receive either tablet-based parenting support or health promotion content). Therefore, with limited information and uncertainty about which program would be delivered, providers may have been less motivated to provide information to families. In practice, with specific information regarding content and outcomes of the PT, providers may be more likely to endorse the program and maintain implementation of the procedures. Further, with multiple competing quality improvement and research projects, it is possible that providers were less motivated/able to integrate another new activity into their practice. If ezParent delivery were a permanent initiative integrated into the practice, it would become a normative feature and might have higher provider engagement and allow parents immediate access to the intervention. Despite these limitations, we believe that collecting implementation data while conducting an RCT informs the external validity of the program, uptake, and how to support practice site implementation.
Implementation was low despite providers' survey responses that highlight acceptability and appropriateness of the intervention in PPC (Proctor et al., 2011). One potential explanation is that ezParent was delivered as a universal prevention program and was appropriate for all parents in the practice with 2- to 5-year-old children and not focused on specific behavior challenges. We intentionally designed the delivery to have few criteria to lessen stigma and increase ease of identification and delivery. However, given the limited time and providers' responsibility to address requirements for the WCV and parent concerns, providers may be less likely to promote programs or studies that are not indicated for a given patient. For example, if parents are asking questions about their child's behavior, the provider may be apt to use the time to explain a program relevant to their concerns. While always relevant during the WCVs, parenting is not always one of the main concerns addressed during a WCV. Successful implementation may require a stepped-care approach in which the providers direct parents to a universal parenting program for all parents and a more targeted intervention or referral for parents who have a specific behavioral concern. This may be a more feasible and acceptable method for providers to use existing infrastructure to reach all parents in the practice. This aligns with findings that primary care may be a critical point of entry for prevention interventions but not the sole intervention site (Rojas et al., 2019).
Our findings are consistent with the literature that identifies barriers to implementation, including time, lack of information, and full practice buy-in and engagement. Therefore, to support successful implementation, we will use these findings and the Agency for Healthcare Research and Quality (2013) toolkit guidelines to develop an implementation toolkit to guide pediatric practices to deliver ezParent. The toolkit will include education and training of the interdisciplinary team, clear messaging regarding ezParent purpose and content, defining roles within the team, development of practice champions, parent education and advertising, use of the EHR, data from the RCT and literature to support intervention effectiveness and implementation, strategies for implementation, and ideas for local adaptations.
Using preimplementation findings from a Hybrid Type I Effectiveness-Implementation trial provides important information to identify challenges in delivering PT in primary care. Application of these findings and the existing literature will inform our ability to leverage existing infrastructure to integrate mental and behavioral health prevention in PPC.
Agency for Healthcare Research and Quality. (2013, February 15). Section 6: Toolkit guidance. Retrieved November 15, 2018, from https://www.ahrq.gov/research/publications/pubcomguide/pcguide6.html
Berkout, O. V., & Gross, A. M. (2013). Externalizing behavior challenges within primary care settings. Aggression and Violent Behavior, 18, 491-495. http://dx.doi.Org/10.1016/j.avb.2013.07.004
Breitenstein, S. M., Brager, J., Ocampo, E. V., & Fogg, L. (2017). Engagement and adherence with ezParent, an mHealth parent-training program promoting child well-being. Child Maltreatment, 22, 295-304. http://dx.doi.org/10.1177/1077559517725402
Breitenstein, S. M, Fogg, L., Ocampo, E. V., Acosta, D. I., & Gross, D. (2016). Parent use and efficacy of a self-administered, tablet-based parent training intervention: A randomized controlled trial. JMIR mHealth and uHealth, 4, e36. http://dx.doi.org/10.2196/mhealth.5202
Brown, C. M., Raglin Bignall, W. J., & Ammerman, R. T. (2018). Preventive behavioral health programs in primary care: A systematic review. Pediatrics, 141, e20170611. http://dx.doi.org/10.1542/peds.2017-0611
Chen, M., & Chan, K. L. (2016). effects of parenting programs on child maltreatment prevention: A metaanalysis. Trauma, Violence, & Abuse, 17, 88-104. http://dx.doi.org/10.1177/1524838014566718
Curran, G. M, Bauer, M, Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50, 217-226. http://dx.doi.org/10.1097/MLR.0b013e3182408812
Duncan, P. M, Pirretti, A., Earls, M. F., Stratbucker. W., Healy, J. A., Shaw, J. S., & Kairys, S. (2015). Improving delivery of Bright Futures preventive services at the 9- and 24-month well child visit. Pediatrics, 135, el78-el86. http://dx.doi.org/10.1542/peds.2013-3119
Forgatch, M. S., Patterson, G. R., & Gewirtz, A. H. (2013). Looking forward: The promise of widespread implementation of parent training programs. Perspectives on Psychological Science, 8, 682-694. http://dx.doi.org/10.1177/1745691613503478
Glasgow, R. E., & Estabrooks. P. E. (2018). Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Preventing Chronic Disease, 15, E02. http://dx.doi.org/10.5888/pcd15.170271
Johnson, M. J., & May, C. R. (2015). Promoting professional behaviour change in healthcare: What interventions work, and why? A theory-led overview of systematic reviews. British Medical Journal Open, 5, e008592. http://dx.doi.org/10.1136/bmjopen-2015-008592
Keith, R. E., Crosson, J. C, O'Malley, A. S., Cromp, D., & Taylor, E. F. (2017). Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: A rapid-cycle evaluation approach to improving implementation. Implementation Science, 12, 15. http://dx.doi.org/10.1186/s13012-017-0550-7
King, T. M., Muzaffar, S., & George, M. (2009). The role of clinic culture in implementation of primary care interventions: The case of Reach Out and Read. Academic Pediatrics, 9, 40-46. http://dx.doi.org/10.1016/j.acap.2008.10.004
Leslie. L. K., Mehus, C. J., Hawkins, J. D., Boat, T., McCabe, M. A., Barkin, S.,...Beardslee, W. (2016). Primary health care: Potential home for family-focused preventive interventions. American Journal of Preventive Medicine, 51(Suppl 2), S106-S118. http://dx.doi.org/10.1016/j.amepre.2016.05.014
McLearn, K. T., Strobino, D. M., Minkovitz, C. S., Marks, E., Bishai, D., & Hou, W. (2004). Narrowing the income gaps in preventive care for young children: Families in healthy steps. Journal of Urban Health, 81, 556-567. http://dx.doi.org/10.1093/jurban/jthl40
Metzler, C, Sanders, M., Rivara, F., Christakis, D., & Rusby, J. (2014). Parenting help online study: Pediatrician questionnaire. Eugene, OR: Oregon Research Institute.
Perrin, E. C., Leslie, L. K., & Boat, T. (2016). Parenting as primary prevention. Journal of the American Medical Association Pediatrics, 170, 637-638. http://dx.doi.org/10.1001/jamapediatrics.2016.0225
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M.,...Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 21. http://dx.doi.org/10.1186/s13012-015-0209-1
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A.,...Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65-76. http://dx.doi.org/10.1007/s10488-010-0319-7
Rahm, A. K., Boggs, J. M., Martin, C, Price, D. W., Beck, A., Backer, T. E., & Dearing, J. W. (2015). Facilitators and barriers to implementing Screening, Brief Intervention, and Referral to Treatment (SBIRT) in primary care in integrated health care settings. Substance Abuse, 36, 281-288. http://dx.doi.org/10.1080/08897077.2014.951140
Rojas, L. M., Baham6n, M., Wagstaff, R., Ferre, I., Perrino, T., Estrada, Y.,...Prado. G. (2019). Evidence-based prevention programs targeting youth mental and behavioral health in primary care: A systematic review. Preventive Medicine: An International Journal Devoted to Practice and Theory, 120, 85-99. http://dx.doi.Org/10.1016/j.ypmed.2018.12.009
SAMHSA's National Registry of Evidence-Based Programs and Practices. (2017). Parent training programs, evidence summary. Rockville, MD: Substance Abuse and Mental Health Services Administration. Retrieved from https://nrepp-learning.samhsa.gov/sites/default/files/documents/Topics_Behavioral_Health/pdf_07_2017/Parent%20Training%20Programs_7.2017.pdf
Sege, R. D., Siegel, B. S., & Council on Child Abuse and Neglect, Committee on Psychosocial Aspects of Child and Family Health. (2018). Effective discipline to raise healthy children. Pediatrics, 142, e20183112. http://dx.doi.org/10.1542/peds.2018-3112
Tanner-Smith, E. E., Durlak, J. A., & Marx, R. A. (2018). Empirically based mean effect size distributions for universal prevention programs targeting school-aged youth: A review of meta-analyses. Prevention Science, 19, 1091-1101. http://dx.doi.org/10.1007/s11121-018-0942-1
Received January 28, 2019
Revision received August 7, 2019
Accepted September 6, 2019
Susan M. Breitenstein, PhD, RN, FAAN
The Ohio State University
Laura Pabalan, MD
Rush University Medical Center
Pamela Roper, MD, MPH
University of Illinois at Chicago
Stacy Laurent, DO
University of Illinois at Chicago
Heather J. Risser, PhD
Mary T. Saba, MS, RN
Rush University Medical Center
Michael Schoeny, PhD
Susan M. Breitenstein, PhD, RN, FAAN, College of Nursing, The Ohio State University; Stacy Laurent, DO, Department of Pediatrics, College of Medicine, University of Illinois at Chicago; Laura Pabalan, MD, Department of Pediatrics, College of Medicine, Rush University Medical Center; Heather J. Risser, PhD, Department of Psychology and Behavioral Sciences at Feinberg School of Medicine, Northwestern University; Pamela Roper, MD, MPH, Department of Pediatrics, College of Medicine, University of Illinois at Chicago; Mary T. Saba, MS, RN, Ambulatory Nursing. Rush University Medical Center, Michael Schoeny, PhD, College of Nursing, Rush University.
The authors gratefully acknowledge the support of the providers and staff at our primary care implementation sites and the research assistance of Alethea Callier, Raquel Real, and Katherine Rosemeyer.
This study is supported by a grant from the Agency for Healthcare Research and Quality (R01 HS024273). Trial Registration: NCT02723916.
Correspondence concerning this article should be addressed to Susan M. Breitenstein, PhD, RN, FAAN, College of Nursing, The Ohio State University, 1585 Neil Avenue, Columbus, OH 43210. E-mail: email@example.com
Table 1 Implementation Approach in PPC Implementation strategy Description Plan Assess readiness, identify barriers, build relationships Educate Informing stakeholders of implementation initiative Finance Incentives Restructure Alterations in roles, structures, and data systems Quality management Audit and provide feedback Implementation strategy Implementation approach Plan * Developed relationships of study team and PPC team (written communications and face-to-face meetings). * Reviewed workflow and organizational climate. * Developed tailored implementation strategy based on agreement from study and PPC teams. * Identified potential barriers to implementation and tailored strategies to overcome barriers and honor PPC site preferences. * Identified and prepared champions (one provider per site). * Developed staff and provider training materials. Educate * Conducted educational sessions for staff, providers, and administration on implementation activities. Finance * Provided snacks at educational sessions and periodically through implementation period as thank you for participation. Restructure * One site adapted the electronic health record where providers could select if they implemented the study introduction; the remaining sites used a paper tracking system to monitor implementation. Quality management * Collected monthly performance data (see RE-AIM measures) and provided reports to PPC providers and staff via e-mail updates. Used this feedback to provide reminders and clarify messaging and adjust implementation procedures as needed. Note. PPC = pediatric primary care; RE-AIM = Reach, Effectiveness, Adoption, Implementation, and Maintenance. Table 2 REACH Metrics for Implementation in Pediatric Primary Care Well-child visits for Parents expressing Recruitment children aged 2-5, interest in study, period, M (range) M (range) Site months per month per month 1 18 19.9 (7-54) 7.9 (3-16) 2 11 269.3 (209-348) 19.6 (12-33) 3 14 81.7 (37-153) 12.6 (5-20) 4 11 200.5 (137-315) 19.0 (7-32) Total 123.9 (7-348) 13.8 (3-33) Parents enrolled in the study, Total M (range) parents Site per month enrolled, n 1 3.1 (1-7) 56 2 8.8 (2-14) 100 3 3.6 (1-6) 52 4 7.2 (2-16) 79 Total 5.2 (1-16) 287 Table 3 Description of Primary Care Clinics Sites That Adopted the Implementation Plan Practice site Site description Patient population PPC1 Combined internal medicine Newborn through geriatric and pediatrics PPC2 General pediatrics Newborn through young adult PPC3 Federally Qualified Health Newborn through adults Center PPC4 General and specialty Newborn through young pediatrics adult Practice Allotted time for site Providers well-child visit PPC1 4 attending MDs; 1 NP; 30 min (MDs) 16 resident MDs 40 min (NP) PPC2 15 attending MDs; 33 20 min resident MDs PPC3 5 family medicine 15 min MDs; 2 pediatric MDs; 24 family medicine residents PPC4 8 attending MDs; 1 NP; 30 min (first year 39 resident MDs residents only) 20 min (all other) Note. PPC = pediatric primary care; NP = nurse practitioner. Table 4 Provider Implementation and Maintenance of Procedures Total staff Times implemented Maintained Site providers Never 1-2 times 3+ times 6+ months (a) PPC 1 5 0 (0) 0 (0) 5 (100) 5 (100) PPC 2 15 4 (27) 0 (0) 11 (73) 7 (64) PPC 3 7 1 (14) 3 (43) 3 (43) 3 (50) PPC 4 9 3 (33) 1 (11) 5 (56) 4 (67) Total 36 8 (22) 4 (11) 24 (67) 19 (68) (a) The denominator for the maintenance percentages is the number of staff providers who implemented one or more times.
|Printer friendly Cite/link Email Feedback|
|Author:||Breitenstein, Susan M.; Pabalan, Laura; Roper, Pamela; Laurent, Stacy; Risser, Heather J.; Saba, Mar|
|Publication:||Families, Systems & Health|
|Date:||Dec 1, 2019|
|Previous Article:||Evaluating Time in Health Care: What Are We Busy About?|
|Next Article:||Looking Beyond the Individual: How Family Demands and Capabilities Affect Family Adjustment Following Pediatric Solid Organ Transplant.|