Printer Friendly

Implementation Findings From an Effectiveness-Implementation Trial of Tablet-Based Parent Training in Pediatric Primary Care.

Parent training (PT) is the gold standard for helping parents develop skills to effectively manage problem behaviors, promote positive behaviors, and prevent maltreatment (Chen & Chan, 2016; SAMHSA's National Registry of Evidence-Based Programs and Practices, 2017). Unfortunately, system and logistic barriers make PT largely unavailable to families (Forgatch, Patterson, & Gewirtz, 2013). Mobile delivery approaches can increase access and availability to parents, as well as improve the quality and translation of PT in pediatric primary care (PPC; Breitenstein, Brager, Ocampo, & Fogg, 2017; Leslie et al., 2016). This article reports implementation findings from a Hybrid Type I Effectiveness-Implementation trial evaluating a tablet-based PT intervention in PPC.

ezParent Program

The ezParent program is a six-module, tablet-based PT program adapted from the group-based Chicago Parent Program (Breitenstein, Fogg, Ocampo, Acosta, & Gross, 2016). ezParent was developed to be culturally and contextually relevant for low-income, ethnic minority families of children ages 2-5 years. ezParent helps parents develop positive and effective parenting skills and decrease physical punishment through use of behavioral strategies (e.g., routines, labeled praise), videos of parents using the strategies, activities, quizzes, and assignments. In a previous randomized controlled trial (RCT; n = 79 parents), 85% completed all six program modules, 88% reported that ezParent was very helpful, and 82% would highly recommend the program (Breitenstein et al., 2016). Modest improvements in parenting and child outcomes (Cohen's d = .14-.31) are consistent with universal primary prevention program effects (Tanner-Smith, Durlak, & Marx, 2018).


PPC is an ideal setting for providing PT because it offers a consistent and supportive context (Perrin, Leslie, & Boat, 2016). PPC is accessible to parents and has an existing infrastructure for disseminating information, and parents view PPC providers as trusted sources of information (McLearn et al., 2004). PPC providers are often the first professionals that families approach regarding parenting concerns or child behavior problems (Berkout & Gross, 2013). Recent American Academy of Pediatrics guidelines for decreasing corporal punishment highlight the need for accessibility of resources and programs like ezParent in PPC (Sege, Siegel, & Council on Child Abuse and Neglect, 2018).

Expanding PT in PPC can maximize the public health impact of these interventions by improving access and reach. However, interventions need to be easily implemented to avoid adding to an already burdened system (Leslie et al., 2016; Perrin et al., 2016). There is a need identify implementation processes to create organizational change (Brown, Raglin Bignall, & Ammerman, 2018), and studying PT implementation in PPC provides critical information to identify and address barriers and facilitators and promote sustainability efforts.

Hybrid Type I Effectiveness-Implementation Design

The three types of hybrid effectiveness-implementation designs vary on the emphasis placed on effectiveness testing and implementation evaluation (Curran, Bauer, Mittman, Pyne, & Stetler, 2012). This Type I study design employs a rigorous test of intervention effectiveness while collecting implementation data for feasibility and acceptability of implementation (Curran et al., 2012). In this article, we report on the implementation processes in PPC using

RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance; Glasgow & Estabrooks, 2018). We report encountered barriers and facilitators to implementation. This implementation evaluation will inform future efforts to add PT to standard practice in primary care.


Study Design and Context

A descriptive design using RE-AIM guided the evaluation of implementation in PPC. We evaluated provider practice of introducing the study to parents of children ages 2-5 years during a well-child visit (WCV). This introduction served as proxy for recommending the intervention. The implementation preceded individual-level randomization to ezParent or the control program. We report on reach, adoption, implementation, and maintenance components that were assessed using quantitative measures.

Implementation occurred between April 2016 and April 2018 in four urban PPC sites that served predominantly low-income Chicago communities. All sites report a diverse and a large Medicaid/underserved population base. PPC1 is a medicine-pediatrics practice, PPC3 a family medicine practice, and PPC2 and PPC4 are general pediatric clinics.

Implementation Approach and Procedures

The implementation approach for introduction to the study was tailored to each of the PPC sites. As part of implementation agreements with each site, we identified key stakeholders (providers, staff, and administration). We worked closely with the stakeholders to understand individual practice environments and workflows and to develop tailored implementation plans. Across all sites, PPC providers introduced the study to parents as part of a WCV, allowing integration into the workflow and approximate the process that would occur in practice. The Consolidate Framework for Implementation Research (Keith, Crosson, O'Malley, Cromp, & Taylor, 2017) and a review of strategies for implementation in health care (Powell et al., 2015) guided our implementation approach.

Once the implementation approach was finalized (see Table 1), we developed scripts and training materials. All sites provided parents with written information describing the study and an interest form with contact information for parents to complete. Two sites (PPC1 and PPC4) included the study material in the packets given to parents at their WCV. In the two other sites (PPC2 and PPC3), the staff who brought the family to the exam room provided the study information sheet. Therefore, all parents should have the study information when the provider entered the room. The providers briefly (< 2 min) presented the opportunity to receive parent support via tablet-based apps and to participate in the study. After the visit, the providers completed tracking logs (paper or electronic health record [EHR]) to report compliance with the implementation procedures. Implementation was planned for 10 months at each site, and start times were staggered by 4 months. Study procedures and protocols were approved by institutional review boards at the two primary institutions of the four PPC sites.

Measures and Data Sources

We collected data at three points: preimple-mentation, during implementation, and post-implementation.

Reach. To evaluate that we were reaching our intended population (i.e., parents of children ages 2-5 years), we measured the characteristics of parents from the practices as compared to those who enrolled in the study. All parents who presented with their 2- to 5-year-old child for a WCV were eligible. All data collected from the EHR were stripped of identifiers and aggregated.

Adoption. To evaluate adoption of the implementation procedures, we examined the characteristics of the PPC sites. We tracked the number of educational sessions to orient providers and staff to the project and implementation procedures. We assessed providers' current practices for responding to parent and child behavior concerns. Prior to the start of implementation, providers responded to the Primary Provider Survey (Metzler, Sanders, Rivara, Christakis, & Rusby, 2014), demographic questions, and 32 items assessing their current practice and attitudes in helping parents with managing child behavior problems. Item responses were on a 5-point scale assessing level of difficulty in addressing children's behavior problems, parenting concerns, confidence in managing these problems, and rating of the practice's capacity for dealing with parenting difficulties or child behavior problems.

Implementation. To track provider delivery, three sites used paper tracking logs and one site used the EHR to record whether the provider discussed and gave the study information to parents. If not discussed, providers recorded the reason. Variations in tracking procedures (e.g., paper tracking logs or EHR) occurred due to preference and administrative support to amend the EHR. We also tracked and monitored any adaptations made during delivery. Given variations in resident rotations, we were unable to identify the proportion of residents who implemented or maintained procedures; therefore, proportions are presented for staff providers.

Maintenance. To evaluate the maintenance of procedures, we developed a postimplementation survey to evaluate providers' knowledge and implementation of the procedures and assess barriers and facilitators to implementation. Providers responded to seven questions assessing their role and ease or difficulty of implementation of the procedures and rated their perception of implementing the study introduction into their day-to-day activities on a 5-point scale: not at all, only slightly, somewhat, quite, or very. A final open-ended question queried respondents if they had other thoughts or suggestions. We invited providers to complete the survey after the end of implementation.


For all analyses, descriptive statistics were calculated by clinic and overall. Given inherent differences between clinics, we did not assess between-clinic differences. However, we provide descriptive data by clinic to illustrate variability across sites. To evaluate reach, chi-square tests were used to compare the demographic characteristics of the study participants to the patient population.

For maintenance, we assessed only those providers who had five or more WCVs in at least 1 month during the implementation period. This approach was taken to ensure adequate exposure to the study and opportunities for implementation of the study protocol. All providers with fewer than five WCVs were either part-time staff or medical residents.



Across sites, the monthly number of WCVs for children ages 2-5 years ranged from 20 to 269. On average, providers introduced 14% (18/124) of eligible parents to the study each month. Of the parents introduced to the study, 78% (14/18) expressed interest in participating (see Table 2). In total, 759 parents signed forms indicating their interest in the study; however, the research staff never reached 29% (217/759) to screen for eligibility. Of those screened for eligibility, 3% (14/542) were ineligible, 44% (241/542) did not show up for their baseline appointment, and 53% (287/542) enrolled.

Although race/ethnicity of enrolled parents did not differ between sites, in two sites, the race distribution of those who enrolled differed from the race distribution of the practice. Specifically, in PPC2, enrolled parents were less likely to be White (12% vs. 23% in the practice) and more likely to be Black/African American (57% vs. 47%) or other (31% vs. 24%; p = .008). In PPC4, participants were more likely to be White (17% vs. 8%) and Black/African American (61% vs. 48%) and less likely to be other (23% vs. 37%; p = .002). There were no significant child gender differences between sites, however, PPC3 had fewer enrolled females (39% vs. 52%; p = .05).


All sites adopted their individualized implementation plan (see Table 3). We report responses from the staff MDs and nurse practioners (NPs) as they were primarily responsible for implementation. Across the four sites, 24 (67%) providers responded to the Primary Provider Survey. They reported an average of 14.94 (SD = 10.07) years of practice, 83% reported that it was slightly (58%) to very (25%) difficult to address parents' concerns about their children's behavior problems during WCVs, 79% reported that it was very difficult to make an appropriate referral for children's behavior problems, and 92% reported that they felt inadequately trained to effectively help parents to address behavioral or emotional problems. Providers rated the practices capacity for dealing effectively with parenting difficulties or children's behavior problems as poor (12%), fair (68%), and good (20%).

Across the four sites, we held an average of two sessions (range = 1-3) to orient providers and staff to the project and implementation procedures. In PPC2, we conducted one training session for the staff and nurses; however, we were unable to conduct direct training to the providers. Instead, our site liaison (author LP) introduced the study and process to the providers at a standing meeting and during practice times. In addition to the in-person trainings, we sent reminder e-mails to all providers and staff the day of the implementation launch, and in PPC2 and PPC3, we attended brief morning huddles to remind providers and staff of the start of the program.


Implementation feasibility was high across providers: 67-100% of staff providers reported that they implemented the procedures (see Table 4). Monthly implementation tracking was used to identify adaptations and methods to improve implementation. We provided e-mails to all providers and staff containing implementation rates, procedure reminders, encouragement, and positive reinforcement. Overall, adaptations in all of the sites were minor (e.g., placing forms in areas more visible, changing location of form collection boxes, and posting reminders in break rooms). In addition, our project staff were on site at the PPCs at least twice weekly to collect parent interest forms and support implementation.


We examined staff providers implementing at least one time (n = 28) to evaluate maintenance over time; of these, 68% maintained implementation for at least 6 months. Among those who implemented three or more times, 79% maintained implementation for 6 months (see Table 4). In addition to monthly tracking, at the end of implementation, providers responded to a postimplementation survey. Sixty-eight providers (32 staff MDs/NPs and 36 resident MDs) responded to the survey. Most (77%) respondents reported that they referred at least one parent to the project. One provider noted, "This was an excellent program and helpful for patients" and another stated that the program contained "very useful information for the patients." One provider noted being motivated by the parent report that "the modules helped them deal with their children better." Over half (59%) of respondents reported that they provided the parents with program materials, 57% reported that they explained the opportunity to parents, and 27% reported that they answered parents' questions regarding the program. Providers commented that barriers to implementation related to inconsistencies in maintaining protocols ("materials were inconsistently part of the well-child packets" and "patients did not bring the forms into the exam room"). One provider commented that "more information about the interventions themselves would be helpful." Another recommended that "MDs do not have reasonable time to perform necessary introduction and stay efficient." Finally, several residents commented that it was easier to implement when their supervisor promoted the program.

Despite the majority of providers (69%) endorsing the importance and appropriateness of providing parenting resources in PPC, nearly 25% reported they did not participate in any implementation activities. This is consistent with daily tracking (53% of providers implemented the procedures more than once). Of the providers (23%) who reported that they did not participate in the implementation activities, the reasons (multiple endorsements allowed) included unaware of the program (19%), inadequate time (31%), did not understand procedures (50%), forgot (38%), the parent did not receive materials (63%), and not a priority (13%). As one provider noted, "Within a 20-minute time period...I am to take a history, do an exam...fill out WIC and school forms, log into the computer, enter orders, explain vaccines and make an attempt to be civil and engaging. On many days, we are running behind, so I don't even really have my full 20 minutes. Therefore, I had no time to add more to my work."


In this section, we review relevant findings from the RE-AIM evaluation and discuss factors that contributed to implementation success, challenges, and recommendations for future efforts.

A compelling finding was that we did not adequately reach the target parents for this program. On average, only 4% of potentially eligible parents enrolled in the study. One factor to this low level of reach is related to implementation breakdowns and that providers introduced only 14% of eligible parents to the program. Despite careful planning, we encountered challenges consistent with other studies of brief intervention in PPC, including time, competing priorities, and lack of information/communication about the program (King, Muzaffar, & George, 2009; Rahm et al., 2015).

Practices (PPC2, PPC3, and PPC4) that conducted more monthly WCVs for children aged 2-5 years had lower rates of program introduction than PPC1 (M = 20 visits/month). One explanation is that smaller practices have fewer moving parts and greater ability to integrate new procedures. Relative to the other sites, PPC1 had fewer residents in the practice, the RN director provided support to providers with verbal reminders prior to a WCV, and the providers documented adherence to implementation in the EHR. Successful implementation requires this type of holistic approach to alter provider habits and increase implementation (Johnson & May, 2015). As identified by a resident, if the attending physician supervising them did not promote the program, they were unlikely to do so. This further complicates implementation maintenance as residents rotate frequently through practice settings. Furthermore, 60% of providers indicated that materials were not in the packet or provided to the patient. Thus, even if providers wanted to introduce the program, the lack of materials may have prohibited it. Reminders to include this information for the providers could increase reach.

PPC1 was the only practice that was able to use the EHR to document implementation. We were not able to track whether providers in the other PPCs forgot to fill out the paper tracking logs, making it possible that we have underestimated provider implementation (e.g., providers introduced the study but failed to report that they did). This highlights the benefit of having the cue to integrate the introduction to the program as part of anticipatory guidance procedures in the EHR. Offering universal PT as part of anticipatory guidance aligns with American Academy of Pediatrics recommendations for promoting family support and child development (Duncan et al., 2015). It is also possible that administrative support to amend the EHR for this project signals a higher level of support and buy-in for interventions.

Another implementation challenge was the context of an RCT. Because providers introduced the study prior to randomization, we did not provide comprehensive information to the providers regarding ezParent, and there were multiple steps required for parents to enroll (e.g., complete interest form, set up baseline appointment) that may have created provider and parent barriers. During training sessions, staff and providers received minimal information about ezParent and the control condition (e.g., parents would receive either tablet-based parenting support or health promotion content). Therefore, with limited information and uncertainty about which program would be delivered, providers may have been less motivated to provide information to families. In practice, with specific information regarding content and outcomes of the PT, providers may be more likely to endorse the program and maintain implementation of the procedures. Further, with multiple competing quality improvement and research projects, it is possible that providers were less motivated/able to integrate another new activity into their practice. If ezParent delivery were a permanent initiative integrated into the practice, it would become a normative feature and might have higher provider engagement and allow parents immediate access to the intervention. Despite these limitations, we believe that collecting implementation data while conducting an RCT informs the external validity of the program, uptake, and how to support practice site implementation.

Implementation was low despite providers' survey responses that highlight acceptability and appropriateness of the intervention in PPC (Proctor et al., 2011). One potential explanation is that ezParent was delivered as a universal prevention program and was appropriate for all parents in the practice with 2- to 5-year-old children and not focused on specific behavior challenges. We intentionally designed the delivery to have few criteria to lessen stigma and increase ease of identification and delivery. However, given the limited time and providers' responsibility to address requirements for the WCV and parent concerns, providers may be less likely to promote programs or studies that are not indicated for a given patient. For example, if parents are asking questions about their child's behavior, the provider may be apt to use the time to explain a program relevant to their concerns. While always relevant during the WCVs, parenting is not always one of the main concerns addressed during a WCV. Successful implementation may require a stepped-care approach in which the providers direct parents to a universal parenting program for all parents and a more targeted intervention or referral for parents who have a specific behavioral concern. This may be a more feasible and acceptable method for providers to use existing infrastructure to reach all parents in the practice. This aligns with findings that primary care may be a critical point of entry for prevention interventions but not the sole intervention site (Rojas et al., 2019).

Our findings are consistent with the literature that identifies barriers to implementation, including time, lack of information, and full practice buy-in and engagement. Therefore, to support successful implementation, we will use these findings and the Agency for Healthcare Research and Quality (2013) toolkit guidelines to develop an implementation toolkit to guide pediatric practices to deliver ezParent. The toolkit will include education and training of the interdisciplinary team, clear messaging regarding ezParent purpose and content, defining roles within the team, development of practice champions, parent education and advertising, use of the EHR, data from the RCT and literature to support intervention effectiveness and implementation, strategies for implementation, and ideas for local adaptations.

Using preimplementation findings from a Hybrid Type I Effectiveness-Implementation trial provides important information to identify challenges in delivering PT in primary care. Application of these findings and the existing literature will inform our ability to leverage existing infrastructure to integrate mental and behavioral health prevention in PPC.


Agency for Healthcare Research and Quality. (2013, February 15). Section 6: Toolkit guidance. Retrieved November 15, 2018, from

Berkout, O. V., & Gross, A. M. (2013). Externalizing behavior challenges within primary care settings. Aggression and Violent Behavior, 18, 491-495. http://dx.doi.Org/10.1016/j.avb.2013.07.004

Breitenstein, S. M., Brager, J., Ocampo, E. V., & Fogg, L. (2017). Engagement and adherence with ezParent, an mHealth parent-training program promoting child well-being. Child Maltreatment, 22, 295-304.

Breitenstein, S. M, Fogg, L., Ocampo, E. V., Acosta, D. I., & Gross, D. (2016). Parent use and efficacy of a self-administered, tablet-based parent training intervention: A randomized controlled trial. JMIR mHealth and uHealth, 4, e36.

Brown, C. M., Raglin Bignall, W. J., & Ammerman, R. T. (2018). Preventive behavioral health programs in primary care: A systematic review. Pediatrics, 141, e20170611.

Chen, M., & Chan, K. L. (2016). effects of parenting programs on child maltreatment prevention: A metaanalysis. Trauma, Violence, & Abuse, 17, 88-104.

Curran, G. M, Bauer, M, Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50, 217-226.

Duncan, P. M, Pirretti, A., Earls, M. F., Stratbucker. W., Healy, J. A., Shaw, J. S., & Kairys, S. (2015). Improving delivery of Bright Futures preventive services at the 9- and 24-month well child visit. Pediatrics, 135, el78-el86.

Forgatch, M. S., Patterson, G. R., & Gewirtz, A. H. (2013). Looking forward: The promise of widespread implementation of parent training programs. Perspectives on Psychological Science, 8, 682-694.

Glasgow, R. E., & Estabrooks. P. E. (2018). Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Preventing Chronic Disease, 15, E02.

Johnson, M. J., & May, C. R. (2015). Promoting professional behaviour change in healthcare: What interventions work, and why? A theory-led overview of systematic reviews. British Medical Journal Open, 5, e008592.

Keith, R. E., Crosson, J. C, O'Malley, A. S., Cromp, D., & Taylor, E. F. (2017). Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: A rapid-cycle evaluation approach to improving implementation. Implementation Science, 12, 15.

King, T. M., Muzaffar, S., & George, M. (2009). The role of clinic culture in implementation of primary care interventions: The case of Reach Out and Read. Academic Pediatrics, 9, 40-46.

Leslie. L. K., Mehus, C. J., Hawkins, J. D., Boat, T., McCabe, M. A., Barkin, S.,...Beardslee, W. (2016). Primary health care: Potential home for family-focused preventive interventions. American Journal of Preventive Medicine, 51(Suppl 2), S106-S118.

McLearn, K. T., Strobino, D. M., Minkovitz, C. S., Marks, E., Bishai, D., & Hou, W. (2004). Narrowing the income gaps in preventive care for young children: Families in healthy steps. Journal of Urban Health, 81, 556-567.

Metzler, C, Sanders, M., Rivara, F., Christakis, D., & Rusby, J. (2014). Parenting help online study: Pediatrician questionnaire. Eugene, OR: Oregon Research Institute.

Perrin, E. C., Leslie, L. K., & Boat, T. (2016). Parenting as primary prevention. Journal of the American Medical Association Pediatrics, 170, 637-638.

Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M.,...Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 21.

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A.,...Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65-76.

Rahm, A. K., Boggs, J. M., Martin, C, Price, D. W., Beck, A., Backer, T. E., & Dearing, J. W. (2015). Facilitators and barriers to implementing Screening, Brief Intervention, and Referral to Treatment (SBIRT) in primary care in integrated health care settings. Substance Abuse, 36, 281-288.

Rojas, L. M., Baham6n, M., Wagstaff, R., Ferre, I., Perrino, T., Estrada, Y.,...Prado. G. (2019). Evidence-based prevention programs targeting youth mental and behavioral health in primary care: A systematic review. Preventive Medicine: An International Journal Devoted to Practice and Theory, 120, 85-99. http://dx.doi.Org/10.1016/j.ypmed.2018.12.009

SAMHSA's National Registry of Evidence-Based Programs and Practices. (2017). Parent training programs, evidence summary. Rockville, MD: Substance Abuse and Mental Health Services Administration. Retrieved from

Sege, R. D., Siegel, B. S., & Council on Child Abuse and Neglect, Committee on Psychosocial Aspects of Child and Family Health. (2018). Effective discipline to raise healthy children. Pediatrics, 142, e20183112.

Tanner-Smith, E. E., Durlak, J. A., & Marx, R. A. (2018). Empirically based mean effect size distributions for universal prevention programs targeting school-aged youth: A review of meta-analyses. Prevention Science, 19, 1091-1101.

Received January 28, 2019

Revision received August 7, 2019

Accepted September 6, 2019

Susan M. Breitenstein, PhD, RN, FAAN

The Ohio State University

Laura Pabalan, MD

Rush University Medical Center

Pamela Roper, MD, MPH

University of Illinois at Chicago

Stacy Laurent, DO

University of Illinois at Chicago

Heather J. Risser, PhD

Northwestern University

Mary T. Saba, MS, RN

Rush University Medical Center

Michael Schoeny, PhD

Rush University

Susan M. Breitenstein, PhD, RN, FAAN, College of Nursing, The Ohio State University; Stacy Laurent, DO, Department of Pediatrics, College of Medicine, University of Illinois at Chicago; Laura Pabalan, MD, Department of Pediatrics, College of Medicine, Rush University Medical Center; Heather J. Risser, PhD, Department of Psychology and Behavioral Sciences at Feinberg School of Medicine, Northwestern University; Pamela Roper, MD, MPH, Department of Pediatrics, College of Medicine, University of Illinois at Chicago; Mary T. Saba, MS, RN, Ambulatory Nursing. Rush University Medical Center, Michael Schoeny, PhD, College of Nursing, Rush University.

The authors gratefully acknowledge the support of the providers and staff at our primary care implementation sites and the research assistance of Alethea Callier, Raquel Real, and Katherine Rosemeyer.

This study is supported by a grant from the Agency for Healthcare Research and Quality (R01 HS024273). Trial Registration: NCT02723916.

Correspondence concerning this article should be addressed to Susan M. Breitenstein, PhD, RN, FAAN, College of Nursing, The Ohio State University, 1585 Neil Avenue, Columbus, OH 43210. E-mail:
Table 1
Implementation Approach in PPC

Implementation strategy  Description

Plan                     Assess readiness, identify
                         barriers, build relationships
Educate                  Informing stakeholders of
                         implementation initiative
Finance                  Incentives
Restructure              Alterations in roles, structures,
                         and data systems
Quality management       Audit and provide feedback

Implementation strategy  Implementation approach

Plan                     * Developed relationships of study team and PPC
                         team (written communications and face-to-face
                         * Reviewed workflow and organizational climate.
                         * Developed tailored implementation strategy
                         based on agreement from study and PPC teams.
                         * Identified potential barriers to
                         implementation and tailored strategies to
                         overcome barriers and honor PPC site
                         * Identified and prepared champions (one
                         provider per site).
                         * Developed staff and provider training
Educate                  * Conducted educational sessions for staff,
                         providers, and administration on implementation
Finance                  * Provided snacks at educational sessions and
                         periodically through implementation period as
                         thank you for participation.
Restructure              * One site adapted the electronic health record
                         where providers could select if they
                         implemented the study introduction; the
                         remaining sites used a paper tracking
                         system to monitor implementation.
Quality management       * Collected monthly performance data (see
                         RE-AIM measures) and provided reports to PPC
                         providers and staff via e-mail updates. Used
                         this feedback to provide reminders and clarify
                         messaging and adjust implementation procedures
                         as needed.

Note. PPC = pediatric primary care; RE-AIM = Reach, Effectiveness,
Adoption, Implementation, and Maintenance.

Table 2
REACH Metrics for Implementation in Pediatric Primary Care

                   Well-child visits for  Parents expressing
      Recruitment  children aged 2-5,     interest in study,
         period,      M (range)             M (range)
Site     months       per month             per month

1     18            19.9 (7-54)            7.9 (3-16)
2     11           269.3 (209-348)        19.6 (12-33)
3     14            81.7 (37-153)         12.6 (5-20)
4     11           200.5 (137-315)        19.0 (7-32)
Total              123.9 (7-348)          13.8 (3-33)

       Parents enrolled
       in the study,       Total
        M (range)         parents
Site    per month        enrolled, n

1      3.1 (1-7)          56
2      8.8 (2-14)        100
3      3.6 (1-6)          52
4      7.2 (2-16)         79
Total  5.2 (1-16)        287

Table 3
Description of Primary Care Clinics Sites That Adopted the
Implementation Plan

site      Site description            Patient population

PPC1      Combined internal medicine  Newborn through geriatric
            and pediatrics
PPC2      General pediatrics          Newborn through young
PPC3      Federally Qualified Health  Newborn through adults

PPC4      General and specialty       Newborn through young
            pediatrics                  adult

Practice                           Allotted time for
site      Providers                well-child visit

PPC1       4 attending MDs; 1 NP;  30 min (MDs)
          16 resident MDs          40 min (NP)
PPC2      15 attending MDs; 33     20 min
          resident MDs
PPC3       5 family medicine       15 min
          MDs; 2 pediatric
          MDs; 24 family
          medicine residents
PPC4       8 attending MDs; 1 NP;  30 min (first year
          39 resident MDs          residents only)
                                   20 min (all other)

Note. PPC = pediatric primary care; NP = nurse practitioner.

Table 4
Provider Implementation and Maintenance of Procedures

       Total staff          Times implemented            Maintained
Site   providers    Never   1-2 times          3+ times   6+ months (a)

PPC 1     5         0 (0)   0 (0)               5 (100)   5 (100)
PPC 2    15         4 (27)  0 (0)              11 (73)    7 (64)
PPC 3     7         1 (14)  3 (43)              3 (43)    3 (50)
PPC 4     9         3 (33)  1 (11)              5 (56)    4 (67)
Total    36         8 (22)  4 (11)             24 (67)   19 (68)

(a) The denominator for the maintenance percentages is the number of
staff providers who implemented one or more times.
COPYRIGHT 2019 American Psychological Association, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2019 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Breitenstein, Susan M.; Pabalan, Laura; Roper, Pamela; Laurent, Stacy; Risser, Heather J.; Saba, Mar
Publication:Families, Systems & Health
Date:Dec 1, 2019
Previous Article:Evaluating Time in Health Care: What Are We Busy About?
Next Article:Looking Beyond the Individual: How Family Demands and Capabilities Affect Family Adjustment Following Pediatric Solid Organ Transplant.

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters