The use of technology to improve staff performance.
Staff performance is one of the critical components of the effective programming for learners with special needs (Jahr, 1998; Parsons, & Reid, 1995; Salmento & Bambara, 2000; Parsons, Reid, & Green, 1993; Baker, Foxx, & Albin, 1995; Schepis, Reid, Ownbey, & Parsons, 2001). Didactic trainings in forms of lectures and workshops are often used to teach staff knowledge and to improve staff performance. However, the didactic teaching does not always translate to the application of the clinical or educational interventions because "knowing" and "doing" are different repertoires. For example, knowing the difference of various prompting procedures does not make the instructor competent to implement prompting procedures effectively (Parsons & Reid, 1995; Jahr, 1998; Schepis, Ownbey, Parsons, & Reid, 2000; Smith, 1995; Plavnick, Ferreri, & Maupin, 2010; DiGennaro, Marrtens, & Kleinman, 2007)
In our verbal world, it is very natural to call learner's name or asking the learner, "What's next?" and staff may not think these as verbal prompts. However, students may develop prompt dependency within tasks or for transition. Just making faces or eye contact can be also a prompt for learners, but instructors may not realize they are even using those prompts while they are working with learners. Didactic teaching does not effectively address these issues.
On-site staff training, including the frequent on-site supervision and feedback, is proven to be effective in order to improve staff performance (Smith, 1995; Arco, 2008; Green, Rollyson, Passante, & Reid, 2002; Parsons, Reid, & Crow, 2003; Salmento & Bambara, 2000; Langeland, Johnson, & Mawhinney, 1998; Reid, Rotholz, Parsons, Morris, Braswell, Green, & Schell, 2003; Guercio, Dixon, Soldner, Shoemaker, Zlomke, Root, & Small, 2005). Despite the effectiveness of on-site training, there are some barriers to implement this type of training consistently and frequently. First, the on-site staff training including staff observation and providing feedback is time consuming, especially for community based programs since supervisors need to visit all training sites and they lose driving time between sites. Secondly, implementation of the on-site training is costly. It requires many supervisors to implement sufficient amount of training on-going basis and within reasonable amount of time period. Thirdly, the presence of supervisor can be intrusive to staff, learners, and the environment, especially in the community. In addition to these barriers to implement on-site training, reactivity of staff to the presence of their supervisors could affect the assessment of staff performance (Brackett, Reid, & Green, 2007; Mowery, Miltenberger, & Weil, 2010). Without having accurate data on staff performance, the supervisors cannot provide appropriate training for staff. This will impact the quality of service provided to learners.
These barriers of providing on-site training can be minimized with utilization of available technology. The advancement of technology has made the various modes of environmental adaptations including many electronic devices available and accessible with significantly reduced cost. The implementation of those devices became easier due to the improved portability. Nepo (in press) and Satriale, Chance, and Nepo (2007) demonstrated that the Bluetooth[R] technology can be effectively utilized for interventions to teach learners with Autism. This concept can be also applied for the on-site staff training. In the present study, Bluetooth[R] technology was implemented to collect data and provide immediate feedback remotely thereby time and cost for driving will be saved and reactivity will be decreased.
Beside the on-site observation and feedback, the effective staff training package often include self-monitoring. Self-monitoring procedure consists of goal setting and recording own target behaviour has been proved to be effective to improve staff performance, especially when combined with other procedures (Petscher & Bailey, 2006; Richman, Riordan, Reiss, Pyles, & Baily, 1988; Baker, Fox, & Albin, 1995; Plavnick, Ferreri, & Maupin, 2010). The author incorporated the self-monitoring in this study not only to improve staff performance but also to monitor their awareness of own behaviours. The purpose of this study was to examine the effectiveness of the intervention package, including immediate feedback, self-monitoring, and delayed feedback with videotaped sessions and graphs, with utilization of commonly available technology on staff performance. A multiple baseline experimental design was used across participants and it was hypothesized that the intervention package incorporating technology will improve staff performance.
Materials and Methods
Bluetooth[R]: Motorola 807L, Verizon 06329N Cell phone with Bluetooth[R] capabilities: LG 810, IPhone-3G with 8GB
Participants were Ricky, Eric, and George who worked at a community based program for adolescents and adults with autism as a direct care staff. Ricky was 31 years old male instructor who had experience working with adolescents with autism for over 5 years. He had participated in initial and on-going didactic training for the basic ABA strategies prior to the current study. Eric was 27 years old male instructor who held a teaching certificate but did not have previous work experience with adolescents with autism prior to the current position. He received the initial and on-going basic ABA training in forms of lectures and workshops. George was 34 years old male instructor who had experience working with adolescents with autism for over 5 years. He received initial and on-going didactic training for the basic ABA strategies prior to the study.
All participants agreed to participant in the study to improve their performance prior to the study. However, the details of the study regarding the target behaviors were not disclosed until their intervention phases.
The settings for this study were vocational sites for adolescents with autism, including local convenience store, hotel, and restaurant.
Target Behavior/Data Collection
The number of unnecessary verbal prompts, including calling students' names when students needs to be prompted to keep working, asking students', "What's next?" while the goal is students to check their schedule and transition independently, and providing verbal directions between steps of tasks where the instructional plan indicates to use physical guidance to shape the sequence, were collected for 10 minutes via Bluetooth[R] and a remote cell phone 1-2 times per week.
IOA (Inter Observer Agreement) data were collected 31 % of the total sessions by a second instructor from the participants' school. The number of agreements was divided by the total number of sessions (the number of agreements plus the number of disagreement) and multiplied by 100. 100 % agreement on IOA was obtained.
A multiple baseline experimental design was used across participants to examine the effectiveness of the intervention package utilizing the commonly available technology, including self monitoring, immediate behavior specific feedback, delayed feedback with video and graphs on staff performance.
During the baseline, participants wore Bluetooth[R] and kept a cell phone with them (in their pockets or clipped to their hips). The number of verbal prompts was collected through the Bluetooth[R] remotely except the videotaped sessions. The verbal consent was attained prior to the study but participants were not provided information regarding their target behaviors.
Participants wore Bluetooth[R] and kept a cell phone in their pockets or clipped to their hips, and the number of verbal prompts was collected remotely via Bluetooth[R] technology. The participants were reminded of the target behaviors with examples prior to each session with adolescents with autism. During the session, the participants monitored the number of verbal prompts they used and the immediate feedback was provided during and immediately after the session from the supervisor via Bluetooth[R]. Videotaped sessions and graphs of their behaviors were used for feed back at the end of their day.
The number of verbal prompts was collected via Bluetooth[R] successfully and remotely. The immediate feedback via Bluetooth[R], delayed feedback with graphs and video clips, and self monitoring significantly reduce the unnecessary verbal prompts for all participants during the sessions with adolescents with autism. Initially the videotaped sessions were planned to provide corrective feedback as well as positive feedback for the target behavior. However, reactivity of all participants to being videotaped and the presence of the supervisor were very high, and the number of verbal prompts was low across phases. Thus, the video clips of sessions were used mainly for positive feedback during the intervention. Nonetheless, all participants responded positively to the intervention package. Ricky's verbal prompts were decreased from an average of 12.25 (8-15) times per 10 minutes to an average of 1.9 (0-10) per 10 minutes with implementation of the intervention package. For Eric, the number of verbal prompts was decreased from an average of 14.6 (9-17) times per 10 minutes to an average of 1.22 (0-8) per 10 minutes after the intervention package was implemented. George's verbal prompts were decreased from an average of 16.7 (9-23) times per 10 minutes to an average of 1.5 (0-12) per 10 minutes with implementation of the intervention.
[FIGURE 1 OMITTED]
The increasing number of research studies shows that the use of technology can support the effective ABA programming (Mechiling, Gast, & Scid, 2009; Goldsmith & LeBlanc, 2004; Stromer, Kimball, Kinney, & Taylor, 2006; Parsons & Mitchell, 2002; Delano, 2007; Goodwin, 2008; Mechling, & Cronin, 2006). The results of this study indicated that the commonly available technology can be utilized to collect data effectively and provide on-site staff training remotely. All participants responded to the current intervention package positively and the unnecessary verbal prompts were eliminated to teach skills to adolescents with autism. It is also important to note several advantages of utilizing Bluetooth[R] within the intervention package here. First of all, reactivity to the presence of their supervisor was eliminated. Compared to videotaped sessions, all participants exhibited significantly more unnecessary verbal prompts during their baseline sessions with data collection via Bluetooth[R] which reflected their typical staff performance in absence of their supervisors. Thus, Bluetooth[R] supported to collect accurate data in order to provide proper training for staff performance (Bracket!, Reid, & Green; 2007). Secondly, cost of maintain enough supervisors and driving, especially for community based programs or programs with multiple sites, was reduced. Thirdly, this data collection method made on-site staff training for supervisors easier to provide since the supervisors could collect data remotely from their office. Fourthly, by incorporating Bluetooth[R] technology, feedback was delivered immediately which was proved to be more effective than delayed feedback to improve staff performance (Daniels, 1989; Schepis, & Reid, 1994). In addition, intrusiveness to the environments, students, and staff were reduced since the supervisors were not physically present during the sessions. As we all agree, having extra person in the environments can be stigmatizing or at least not appropriate, especially in the community or in the inclusion classrooms. The Bluetooth technology[R] had made the implementation of the intervention possible without the presence of supervisor.
Despite of positive outcome of the current intervention package on staff performance, there are some limitations in this study need to be noted. Through Bluetooth[R] data collection, the possible use of other prompts such as gestures, facial expressions, or eye contacts, cannot be monitored even the staff may have used during sessions. Those prompts may require attention for further improvement of staff performance. All participants displayed high reactivity to being videotaped and the presence of the supervisor, thus the videotaped sessions were not used as corrective feedback tool as intended originally. Those video clips were used rather as positive feedback for the target behaviors. Self monitoring can be difficult at times, especially during dyads and triads instructions in the community on top of collecting data for learners' behaviors. The long term effects of the intervention also need to be investigated. The follow up data need to be collected continuously to examine the maintenance of their behaviors. The network connectivity can affect the reliability of data collections. For example, if there are too many dropped called during the session would affect the results. Additionally, there may be individual difference in the reactivity to the Bluetooth[R], video feedback, immediate feedback, as well as visual inspection. Although all participants in the current study were motivated by feedback and their own progress, others may feel the intervention somewhat intrusive.
The replication studies across larger number of participants and settings are necessary to assess the effectiveness of the staff training utilizing the current technology. The further investigation on the components of this intervention, including self monitoring, immediate feedback, and video feedback, as well as the sequence of these components calls for the future research. The reaction to videotaping, wearing Bluetooth[R], and data collection by others require additional analysis. In addition, the impact on the performance of the learners with autism by reducing the verbal prompts needs to be examined, since the ultimate goal of the staff performance improvement is providing the most effective treatment for consumers.
As the author demonstrated and supported by increasing body of research, technology can support the advancement of behavioural interventions (Mechiling, Gast, & Scid, 2009; Goldsmith & LeBlanc, 2004; Stromer, Kimball, Kinney, & Taylor, 2006; Parsons & Mitchell, 2002; Delano, 2007; Goodwin, 2008; Mechling, & Cronin, 2006). Along with rapid advancement of technology, the more devices or software become readily available to be utilized as a part of behavioural interventions. It is time for more researchers to uncover the ways to incorporate technology into the behavioural interventions.
I would like to acknowledge Ricky Deleguardia, Eric Shindledecker, and George Jennings for participating in this study and continuing to work with our students, and all of PAAL staff members for supporting this study.
I am also grateful to have supportive parents who have made this project possible. At last, it is impossible for me to complete this project without participations of our students, and I am honored that I had given opportunities to work with these individuals. * PAAL: Preparing Adolescents for Adult life
Arco, L. (2008). Feedback For Improving Staff Training And Performance In Behavioral Treatment Programs. Behavior Interventions, 23, 39-64.
Baker, D. J., Fox, J., & Albin, R. W. (1995). Staff Support Strategies To Increase The involvement of Individual with Disabilities in Meal Preparation. Behavioral Intervention, 10 (4), 197-210.
Brackett, L., Reid, D. H., & Green, C. W. (2007). Effects of Reactivity to Observations on Staff Performance. Journal of Applied Behavior Analysis, 40, 191-195.
Daniels, A. C. (1989). Performance Management. Tucker, GA: Performance Management Publications.
Delano, M. E., (2007). Video Modeling Interventions For Individuals With Autism. Remedial and Special Education, 28, 33-42.
DiGennoro, F. D., Martens, B. K. & Kleinman, A. E. (2007). A Comparison of Performance Feedback Procedures On Teachers' Treatment Implementation Integrity And Students' Inappropriate Behavior In Special Education Classrooms. Journal of Applied Behavior Analysis, 40, 447-461.
Goldsmith, T., R., & LeBlanc, L., A. (2004). Use of Technology in Interventions for Children with Autism. Journal of Early and Intensive Behavioral Intervention, 1(2), 166-178.
Goodwin, M. S. (2008). Enhancing And Accelerating The Pace Of Autism Research And Treatment: The Promise Of Developing Innovative Technology. Focus On Autism And Other Developmental Disabilities. 23, 125-128.
Green, C. W., Rollson, J. H., Passante, S. C., & Red D. H. (2002). Maintaining Proficient Supervisor Performance With Direct Support Personnel: An Analysis of Two Management Approaches. Journal of Applied Behavior Analysis, 35, 205-208.
Guercio, J. M., Dixon, M. R., Sholner, J., Shoemaker, Z., Zlomke, K., Root, S., & Small, S. (2005). Enhancing Staff Performance Measures in a Acquired Brain Injury Setting: Combating The Habituation to Organizational Behavior Interventions. Behavioral Intervention, 20, 91-99.
Jahr, E. (1998). Current Issues in Staff Training. Research in Developmental Disabilities, 19(1), 73-87.
Langeland, K. L., Johnson, C. M., & Mawhinney, T. C. (1998). Improving Staff Performance in a Community Mental Health Setting: Job Analysis, Training, Goal Setting, Feedback, and Years of Data. Journal of Organizational Behavior Management, 18 (1), 21-42.
Mechling, L. C., & Cronin, B. (2006). Computer-Based Video Instruction To Teach The Use of Augmentative And Alternative Communication Devices For Ordering At Fast-Food Restaurants. The Journal of Special Education. 39, 234-245.
Mechling, L. C., Gast, D. L., & Scid, N. H. (2009). Using a Personal Digital Assistant to Increase Independent Task Completion by Students with Autism Spectrum Disorder, Journal of Autism and Developmental Disorder, 39, 1420-1434.
Mowery, J. M., Miltenberger, R. G., & Weil, T. M. (2010). Evaluating the Effects of Reactivity to Supervisor Presence on Staff Response to Tactile Prompts and Self-Monitoring in a Group Home Setting. Behavior Intervention, 25, 21-35.
Nepo, K. G. The Use of Bluetooth[R] Technology To Increase Independence and Social Acceptance of Adolescents With Autism. In Press.
Parsons, M. B., & Reid, D. H., & Crow, R. E. (2003). Best And Worst Ways To Motivate Staff In Community Agencies: A Brief Survey of Supervisors. Mental Retardation, 41, 96-102.
Parsons, M. B., Reid, D. H., & Green, C. W. (1993). Preparing Direct Staff to Teach People with Severe Disabilities: A Comprehensive Evaluation of an Effective and Acceptable Training Program. Behavioral Residential Treatment, 8 (3), 163-185.
Parsons, S., & Mitchell, P. (2002). The Potential Of Virtual Reality In Social Skills Training For People With Autistic Spectrum Disorders. Journal of Intellectual Disability Research. 46 (5), 430-443
Petscher, E. S., & Bailey, J. S. (2006). Effects Of Training, Prompting, And Self-Monitoring On Staff Behavior In A Classroom For Students With Disabilities. Journal of Applied Behavior Analysis.39, 215-226.
Plavnick, J. B., Ferreri, S. J., & Maupin. A. N. (2010). The Effects Of Self-Monitoring On The Procedural Integrity Of A Behavioral Intervention For Young Children With Developmental Disabilities. Journal of Applied Behavior Analysis. 43, 315-320.
Reid, D., H., Rotholz, D.A., Parsons, M. B., Morris, L. Braswell, B., A., Green, C. W., & Schell, R. M. (2003). Training Human Service Supervisors In Aspects Of PBS: Evaluation of A Statewide, Performance-Based Program. Journal of Positive Behavior Interventions, 5 (1), 35-46.
Richman, G. S., Riordan, M. R., Reiss, M. L., Pyles, D. A. M., & Bailey, J. S. (1988). The Effects of Self-Monitoring and Supervisor Feedback on Staff Performance in a Residential Setting. Journal of Applied Behavior Analysis, 21, 401-409.
Salmento, M., & Bambara, L. M. (2000). Teaching Staff Members to Provide Choice Opportunities for Adults with Multiple Disabilities, Journal of Positive Behavior Interventions, 2(1), 12-21.
Schepis, M. M., Ownbey, J. B., Parsons, M. B., & Reid, D. H. (2000). Training support staff to teach young children with disabilities in an inclusive preschool settings. Journal of Positive Behavior Interventions, 2, 170-178.
Schepis, M. M., & Reid, D. H. (1994). Training Direct Service Staff in Congregate Settings to Interact with People with Severe Disabilities: A Quick, Effective And Acceptable Program. Behavioral Intervention, 9 (1), 13-26.
Schepis, M. M., Reid, D. H., Ownbey, J. & Parsons, M. B. (2001). Training Support Staff To Embed Teaching Within Natural Routines Of Young Children With Disabilities In An Inclusive Preschool. Journal of Applied Behavior Analysis, 34, 313-327.
Smith, D. M. (1995). A System of Program Evaluation and the Use of Feedback as a Means of Improving Service Delivery. Behavioral Intervention, 10 (4), 225-236.
Satriale, M. G., Chance, L., & Nepo, K (2007). The Use of Bluetooth[R] Technology to Promote Independent Responding in the Community: The Reduction of the Stigma of Prompting. Poster Presentation at ABAI Autism 2009.
Stromer, R., Kimball, J. W., Kinney, E. M., & Taylor, B. A. (2006). Activity Schedules, Computer Technology, and Teaching Children With Autism Spectrum Disorders. Focus On Autism And Other Developmental Disabilities. 21 (1), 14-24.
Kaori G. Nepo
503 S. Old Middletown Road Media, Pa 19063
|Printer friendly Cite/link Email Feedback|
|Author:||Nepo, Kaori G.|
|Publication:||The International Journal of Behavioral Consultation and Therapy|
|Date:||Jun 22, 2010|
|Previous Article:||Therapist's directive and nondirective behavior: analysis of their effects in a parent training group.|
|Next Article:||The use of biofeedback, CES, brain mapping and neurofeedback with youth who have sexual behavior problems.|