Impact of a computer-based case study on outbreak investigation skills.

Abstract

Distance learning has been promoted as an effective training tool for the public health workforce. We developed a self-instructional computer-based case study, "Botulism in Argentina," that builds on the strengths of distance learning while avoiding some of its shortcomings. The case study was designed to teach public health practitioners skills in investigating infectious disease outbreaks. In the case study, students work through a real-life outbreak and must intermittently answer questions to progress. Students can review basic epidemiologic and public health concepts or explore topics of interest in greater detail. The case study was field tested on 32 members of the target audience. Field testers believed that the case study was easy to undertake, informative, and enjoyable. Of the 28 field testers who completed both a pre-test and post-test, scores increased by an average of 29%. Field testers who had NOT participated in an outbreak investigation had a significantly greater increase in scores. This computer-based case study appears effective in teaching skills in outbreak investigation and could be a model for other training materials.

Background

Training and retraining of the estimated 450,000 public health employees at the local, state, and federal level is a challenging task. (HRSA, 2001) As with all adult learners, training of public health employees must focus on knowledge and skills relevant to job activities, ideally in the setting in which those activities are performed. To facilitate translation to the workplace, training of public health employees needs to provide the learner with ample opportunity to synthesize knowledge and practice skills. Because public health employees come to their jobs from varying backgrounds (i.e., education and professional experiences) and learning styles, training must be tailored to meet individual needs to be most effective and efficient. In addition, training must be repeated at irregular and unpredictable intervals due to continual workforce turnover. Finally, due to limited funding and employee time, training must be affordable and able to accommodate the busy schedule of the public health employee. These last, perhaps more practical, constraints may be most influential in public health training efforts. Researchers estimated that 80% of public health practitioners were unable to attend desired training due to cost, time, and travel. (Hirano and Dillenberg, 1998)

The traditional student/instructor classroom model often proves too costly, time-consuming, and inefficient to address most public health training needs. As a result, distance learning has been promoted as an effective and increasingly popular training tool for the public health workforce. (Hinman, 1996; Friede et al., 1995; Maibach and Holtgrave, 1995; Umble et al., 2000; Berman and Novotny, 1999; Farel et al., 2001; Rosner et al., 1996)

Distance learning is broadly defined as an educational situation in which the instructor and learner are separated by time, location, or both. (Kaplan-Leiserson, 2001) Formats of distance learning can be as simple as print-based training materials and as complex as web-based courses that incorporate live events and follow-up discussion groups. Additional distance learning formats include audiotape, videotape, computer and web-based instruction, audio- and videoconference, satellite broadcast, and webcast.

Distance learning has several advantages over the traditional classroom model. Distance learning can be designed so that learners can fit the instruction into their schedules at their convenience. It can be designed to allow learners to progress at their own rates; they can skip over topics they already know (or don't need to know) or repeat topics they are having difficulty with. In addition, distance learning can be used to provide the same information and training simultaneously across a geographically dispersed audience or repetitively over time so that some level of consistency and quality can be achieved.

Distance learning, however, also has its shortcomings. Some distance learning can seem remote and impersonal to the learner, particularly when there is limited or no human contact as provided by an instructor or other students. Opportunities for the student to interact intellectually with the content, such as through individualized feedback, can also be limited and hinder the student's assimilation of the material. In some formats of distance learning, spontaneity is lacking and strategies used to present the instruction are uninteresting to the learner. Furthermore, distance learning that is synchronous in nature, such as satellite broadcasts and webcasts, requires students to be in a specific place at a specific time.

We developed "Botulism in Argentina," a computer-based epidemioiogic case study, as a new approach for distance learning in public health practice. In this model we try to build on the strengths of distance learning while avoiding some of its shortcomings, and address the special needs of the public health workforce.

Description Of Innovative Training Methods

"Botulism in Argentina" is an interactive computer-based exercise developed to teach public health employees skills for investigating infectious disease outbreaks. It was designed primarily for use by individuals for self-instruction. "Botulism in Argentina" was based on a real-life outbreak investigation (Villar R, 1999) and a traditional print-based classroom case study of the same name. (Stehr-Green, 2000) The classroom version of the case study, created for use in a group setting with a knowledgeable instructor, was developed and tested before work on the computer-based version was initiated, Development and testing of the classroom version helped clarify the structure of the computer-based version, including placement of questions and identification of topics for which some students might need remedial explanations or desire further exploration.

In "Botulism in Argentina," students work through an outbreak of botulism from beginning to end. Information about the outbreak is slowly revealed through written text. Intermittently, students answer questions and then receive feedback tailored to their answers. Question format includes multiple choice (single and multiple answer), drag-and-drop activities, yes/no questions, and fill-in-the-blanks. Students can access hints and reference materials, included in the case study, before answering the questions. Students needing more remedial work can review didactic lessons about basic epidemiologic and public health concepts (i.e., MORE ABOUT lessons). Curious students can explore topics of interest in greater detail through vignettes presented by an epidemiologist, laboratorian, or sanitarian (i.e., one of three professionals frequently involved in investigating infectious disease outbreaks) (i.e., EXPLORE WITH AN EXPERT vignettes). Students can move at their own pace and decide what supplemental learning activities to undertake. They can complete the case study in one sitting (which takes about 3 hours) or spread it over several sessions.

The interface for the case study is a day-planner type notebook placed on a wooden desktop. The notebook includes dividers that represent the steps in a routine outbreak investigation. The background information and questions are presented in a consistent manner as if on the pages of the notebook. To support the notebook metaphor, other interface elements are placed on the desktop or clipped to the notebook, including:

* mocked-up photographs that depict investigation team activities

* an epidemic curve that graphically depicts the outbreak and investigation and changes as the case study progresses

* an investigation outline that corresponds to the six steps of the outbreak investigation and contains a record of the student's progress through the case study

* other investigation items such as press releases and questionnaires.

Where possible we tried to maintain the notebook metaphor so the student would remain immersed in the case study. See issue's website <http://rapidintellect.com/AEQweb/fa102.htm>

Evaluation of Training Method

Methods

After revising the product based on one-on-one observations of students working on the case study and in-depth review by content and instructional design experts, "Botulism in Argentina" was tested on members of the target audience. Field testers were identified through colleagues in local and state health departments. To participate, field testers met the following criteria:

* be from the intended target audience which includes epidemiologists, nurses, sanitarians, MPH students in academic institutions, and other persons with knowledge of basic epidemiologic concepts

* have access to a personal computer with a Windows 95 or higher operating system and a CD drive

* understand English

Field testers were required to provide information about their education and job experience, complete a 23-question pre-test, do the case study, and complete a post-test (identical to the pre-test). Field testers were also asked to rate (on a scale of 1 [strongly disagree] to 5 [strongly agree]) their level of agreement to a standardized set of positive statements about the case study (e.g., "The language used in the case study was clear and understandable." "The feedback received for answers was informative.") and report any problems they had with the case study. To be included in the evaluation, participants had to complete the field test within 3 weeks.

Results

Thirty-eight persons from the target audience agreed to participate in the field test; 32 (84%) completed the case study and evaluation, 3 (8%) began the case study but did not complete the field test by the deadline, 3 (8%) withdrew. Individuals who withdrew were asked why they did not complete the field test. All withdrew for reasons unrelated to the case study (e.g., vacation, upcoming meeting for which they had to prepare). None reported that problems with the software or case study prevented them from completing the field test.

Twenty-three (72%) field testers had completed an introductory epidemiology course; 20 (63%) had participated in a foodborne disease outbreak investigation. Seven field testers were from outside the United States (i.e., Colombia and Egypt).

On a set of standardized statements, the field testers were positive about the training module. They believed the case study was easy to undertake, informative, and enjoyable. They also believed the case study improved their understanding of outbreak investigations. All field testers said they would be interested in doing similar training modules in the future and would recommend this case study to colleagues and coworkers. See issue's website <http://rapidintellect.com/AEQweb/fa102.htm>

Twenty-eight field testers completed both the pre-test and post-test with average scores of 65% and 78% respectively. The average increase in post-test scores was 29%. Changes in scores did not vary by field tester job/position description (e.g., epidemiologist, public health nurse, sanitarian). MPH students, however, had a higher increase in post-test scores compared with all other field testers (52% vs. 22%). Field testers who had previously completed an introductory epidemiology course improved more on the post-test compared with persons who had not completed such a course (32% vs. 23%). Neither of these findings was statistically significant. Field testers who had NOT participated in an outbreak investigation before the field test had a significantly greater increase in post-test scores compared with those who had previously participated in an investigation (54% vs. 13%).

Discussion

Although this evaluation of the "Botulism in Argentina" computer-based case study was somewhat limited, we did demonstrate that members of the target audience had an increase in knowledge about outbreak investigations and enjoyed the training experience. After completing the case study, field testers performed better on a posttest that focused on knowledge and skills related to outbreak investigation. In addition, field testers most in need of the training (i.e., those who had not participated in an outbreak investigation previously) improved the most. Furthermore, the high field test completion rate, given the short turn-around time, and the positive comments about the case study suggest that the case study was easy to fit into the schedules of busy public health practitioners and fun to undertake. Although we were unable to explore whether field testers improved work performance as a result of completing the case study, it is likely that "Botulism in Argentina" will also have significant impact on practice due to its interactive nature (Davis, et al. 1999) and other unique strategies (see below). As a result, we think "Botulism in Argentina" could serve as a successful model for other distance learning efforts.

"Botulism in Argentina" has several advantages over the traditional student/instructor classroom model of training and some formats of distance learning (e.g., audio conferences, satellite broadcasts). Since it is designed primarily for self-instruction, students can complete the case study when it is convenient to their schedule and do so at their own pace. They do not need an instructor or special meeting place; they do not need an extended block of time, but can break the case study into sessions to fit the time they have. Because students can select which learning activities to undertake in the computer-based case study, their training time can be used more efficiently to focus on areas in which they are deficient and those most relevant to their job activities. In addition, it is likely that some students will be less inhibited by the computer-based case study than training in the classroom because peers or colleagues will not influence their performance. (Davis et al., 1999) As a result they may be willing to take chances and make mistakes, both of which can be instructive.

"Botulism in Argentina" provides some unique strategies not available in more didactic training, such as classroom lectures or through satellite broadcasts. The outbreak on which "Botulism in Argentina" was based is not unlike work situations public health employees encounter on the job. This reference point serves as an "anchor" for students to retrieve, collect, and construct knowledge. (Cognition and Technology Group at Vanderbilt, 1992) As a result, it is likely that learning can be more meaningful and resulting skills more easily transferred to such job activities as detection of infectious disease outbreaks, collection and interpretation of information for the ensuing investigation, and development of control and intervention strategies. Completion of "Botulism in Argentina" requires more than simply regurgitating facts and formulas. Students must process information; they must apply concepts, perform calculations, and make decisions. As such, the case study allows the student to practice the complex action-oriented skills required of public health employees and is, therefore, likely to have a greater impact on job performance. (Davis et al., 1999) Finally, the graphic user interface of the case study results in a welcoming and engaging computer screen with which field testers feel comfortable. The case study comes alive for many students, resulting in what is repeatedly reported to be an enjoyable experience.

"Botulism in Argentina" does have some shortcomings. First, it was difficult to address complex issues through such a format. Open-ended questions could not be included because they could not be judged using the computer algorithms available within our budget. To solve this, we broke complex issues into component parts using segments of text and a series of simpler questions. For instance, in a section on developing a study questionnaire, we first ask students about the steps of developing a questionnaire and allow them to review a didactic lesson on developing questions for an epidemiologic study. We then introduce the categories of information typically collected in such a study and ask students to select items (in a "drag-and-drop" question) that they would include in a questionnaire for this particular investigation. We give students an opportunity to write a question (not judged) for the investigation questionnaire and then have them compare their question with several we created. In the end, students get to examine the questionnaire that was used in the original investigation. Of course, these steps do not exactly resemble real life (e.g., being asked to create a full questionnaire), but they cover the desired information and help students develop good thought processing habits and work methodically through a problem.

Secondly, the "Botulism in Argentina" format may not be appropriate for all audiences or subject materials. Some students may not have access to computers meeting the system requirements or may lack the skills needed to use the computer. An informal survey of public health practitioners from 26 state health departments in 1999 suggests that most workers (93%) have access to computer equipment adequate to run the case study. (CDC, unpublished data, 1999) A substantial proportion of the current generation of public health employees may not, however, feel comfortable with computers. For these students, we attempted to create an intuitive interface they could feel comfortable with and developed an extensive, easily accessible help tool including answers to frequently asked questions (largely identified from the field test). Similarly, some types of training may not be well suited to the case study approach or computer-based technology. (Spitzer, 1991) These include efforts that teach basic concepts or facts for which simpler approaches would suffice or those that teach advanced interpersonal skills that require human interactions. We believe that training which requires application of knowledge and skills and problem solving would best be suited to this case study approach.

Finally, "Botulism in Argentina" does not incorporate all three types of interactions ideal for any distance learning effort: learner-content, learner-instructor, and learner-learner. (Moore, 1989) The case study is rich with opportunities for the student to interact intellectually with the content, but it lacks benefits that result from learner-instructor and learner-learner interactions. An instructor can modify the learning process on site by providing feedback, responding to questions, restating issues, posing follow-up questions, and inserting topics of interest as the need arises. An instructor can also play a critical role in motivating students and acting as a role model. Although learners can sometimes be intimidated by other students, the benefits of learner-learner interactions in training should not be overlooked. (Brown and Duguid, 2000) By working collectively, students develop a shared understanding of the training materials. In the presence of others, students learn how to articulate their ideas and defend their decisions. Furthermore, by listening and responding to answers given by their classmates, students learn how to deal with other people and gain experience in the team approach to problem solving. For this case study there was no way we could overcome this disadvantage; however, we are still looking.

Conclusion

Early evaluation and use of the "Botulism in Argentina" computer-based case study suggest that it may be effective in teaching skills in outbreak investigation and could be a model for other public health training products. Although lacking important benefits of instructor-led training, this product appears to offer some unique advantages compared with classroom training and has utility in selected settings, such as those with limited training resources. The long-term impact of "Botulism in Argentina" and effectiveness of the model with different subject matter are unknown. As a result, we would encourage further examination and evaluation of the model by public health practitioners, instructors, instructional designers, and students. The case study can be downloaded or purchased through the Public Health Training Network (PHTN) Case Studies website at http://www.phppo.cdc.gov/phtn/casestudies.

Acknowledgements

This work is funded by the Centers for Disease Control and Prevention. The Authors wish to thank their colleagues that served as field testers and Phil Thompson who edited this manuscript.

References

Bureau of Health Professions, Health Resources and Services Administration. The public health work force. Enumeration 2000. 2001. http: //cpmcnet./columbia.edu/dept/nursing/institute-centers/chphsr/enum2000.pdf (8 Apr. 2002)

Hirano D, Dillenberg J. Transforming public health through distance learning. J Public Health Manag Pract, 1998, 4(5), 42-5.

Hinman AR. Distance learning and distance education: a personal perspective. Am J Prey Med 1996 Jan-Feb;12(1):5-8

Friede A, Blum HL, McDonald M. Public health informatics: how information-age technology can strengthen public health. Annu Rev Public Health 1995;16:239-52

Maibach E, Holtgrave DR. Advances in public health communication. Annu Rev Public Health 1995;16:219-38.

Umble KE, Cervero RM, Yang B, Atkinson WL. Effects of traditional classroom and distance continuing education: A theory-driven evaluation of a vaccine-preventable diseases course. Am J Public Health 2000;90:12-18-24.

Berman SJ, Novotny TE. Extended degree and continuing education preferences of California public health professionals. J Public Health Manag Pract 1999 May;5(3):20-4.

Farel A, Umble K, Polhamus B. Impact of an online analytic skills course. Eval Health Prof 2001 Dec;24(4):446-59.

Rosner E, Gould B, Gaschler L, Howard S, Rarick B. Evaluation of a satellite educational program. Clin Lab Sci 1996 Jan-Feb;9(1):30-4.

Kaplan-Leiserson, E. E-Learning glossary. 2001. http://www.learningcircuits.org/glossary.html (9 Apr. 2002).

Villar RG, Shapiro RL, Busto S, et al. Outbreak of type A botulism and development of a botulism surveillance and antitoxin release system in Argentina. JAMA 1999; 281:1334-40.

Stehr-Green J. Foodborne disease outbreak investigations: classroom epidemiologic case studies. Botulism in Argentina. 2000. http://www.phppo.cdc.gov/phtn/casestudies/classroom.

Davis, D., O'Brien, M. A. T., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-Vaisey, A. (1999) Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 282 (9), 867-74.

Cognition and Technology Group at Vanderbilt. The Jasper experiment: An exploration of issues in learning and instructional design. Educational technology research and development 1992;40(1), 65-80.

Spitzer D. Training: What it is and how to use it appropriately. Performance and Instruction 1991;30:20-3.

Moore MG. Editorial: Three types of interaction. Am J Distance Ed 1989;3(2).

Brown JS and Duguid P. The Social Life of Information. Boston: Harvard Business School Press, 2000. pp. 221-5.

Jeanette Stehr-Green, Public Health Practice Program Office, Centers for Disease Control and Prevention Nancy Gathany, Public Health Practice Program Office, Centers for Disease Control and Prevention

Dr. Stehr-Green was trained in internal medicine at the University of Texas Medical School at San Antonio and Stanford University. She completed a fellowship with the Epidemic Intelligence Service at the Centers for Disease Control and Prevention in 1986. Gathany is a Senior Instructional Designer at the Centers for Disease Control and Prevention for the Public Health Training Network. She received her Masters in Education Degree in Instructional Technology from the University of Georgia in 1987.