Optimizing environmental health training outcomes: a case study of tribal and nontribal trainees.

Introduction

The essential public health services include education and empowerment, development of community partnerships, and the development of policies to ensure healthy people in healthy communities both on a daily basis and in times of crisis (Public Health Functions Steering Committee, 1995). Millions of dollars are spent every year to educate and empower the public health workforce (Herring, 2006); evaluation efforts, however, have focused primarily on knowledge transfer. Few studies have attempted to quantify changes in workplace practices as a function of training attendance (Boatright, 2005).

The purpose of our study was to measure the efficacy of training provided at a crisis and risk communication conference aimed at enhancing environmental health workforce skills. To accomplish this goal, we measured the differences in knowledge gained and implementation of new skills among Native-American and non-Native-American conference attendees. The findings of our study will serve as a foundation for the development of curricula for future environmental public health training programs to ensure that training materials are prepared and presented in a manner that will maximize knowledge retention and ensure relevance to practice.

Methods

Study Participants

The authors selected study participants from registrants at the Crisis and Risk Communication Conference, which was convened at the Loma Linda University School of Public Health in March 2006. The program was attended by 165 individuals, the majority of whom were employed in the western United States in the public-sector environmental health workforce or in the sovereign-nation workforce. Conference attendees were initially assembled into Native-American and non-Native-American cohorts for the purpose of measuring changes in knowledge, skills, and abilities (KSA). We then recruited 11 subjects from the Native-American group and 24 from the non-Native-American group. Of the 24 in the non-Native-American group, three were excluded because of nonresponse, and 12 were excluded because they attended less than 60 percent of the conference. Of the 11 in the Native-American group, two were excluded following the pre-conference questionnaire because they did not attend the conference, five were excluded because the individual who registered for the conference was not the individual who attended the conference, and one was excluded because of nonresponse. The final sample size was 12, with three in the Native-American group and 9 in the non-Native-American group.

Questionnaire

Pre- and post-conference questionnaires were developed to assess demographic data, as well as participant KSAs related to workplace crisis and risk communication strategies. (The preconference questionnaire can be seen at https://www.llu.edu/llu/sph/ophp/nahi/documents/llu_risksurvey.pdf). Our aim was to assess the effectiveness of training in Domain 2 (Learning) and Domain 3 (Behavior) as outlined in published literature (Sarpy, Chauvin, & Anderson, 2003). The questionnaire was divided into three major sections. The first section elicited demographic data on age, race/ethnicity, education, and the entity that the participant was representing at the conference (e.g., private industry, public health agency, sovereign nation, etc.). The remaining two sections measured KSAs in crisis and risk communication by way of Likert scale and true-false questions. Questions measuring participant knowledge were based on content presented throughout the conference curriculum. The post-conference questionnaire also had questions about the applicability of the conference material, potential barriers to applying the course materials, teaching styles of the different presenters, and personal preferences of materials presented.

Interviews were conducted via telephone. The interviewer was prepared before the study through formal training and simulations of questionnaire administration. The pre-conference questionnaire was administered one week before the program. By contrast, the post-conference questionnaire was administered 90 days following the close of the workshop.

Data Analysis

The data were analyzed with the SAS Version 9.1 statistical package, Minitab Version 14, and StatXact. Non-parametric Wilcoxon exact and Fischer exact tests were used to determine if there were significant differences in the responses of the Native-American participants and the non-Native-American participants.

Results

Select professional characteristics of Native-American and non-Native-American subjects are presented in Table 1. The two groups were comparable in all characteristics related to environmental health risk communication experience, past experience managing a crisis situation, and existing availability of support for communication functions. The two groups were not comparable, however, with respect to education level (p = .03). The average education level in the Native-American group was some college education, while on average the non-Native-American group possessed a graduate degree.

Table 2 summarizes select data collected from the two study groups 90 days after the training program. For each of the measured attributes, we found statistically significant differences in the implementation of the materials presented at the conference. The two groups differed significantly in the development and implementation of 1) an appropriate communication network with media in case of emergency (p = .0330), 2) the ability to communicate with an incident command center (ICC) or emergency personnel (p = .0330), 3) appropriate communication systems within the organization, and 4) application of the course material to current crisis and risk communication protocols (p=0.0110). In each case above, the non-Native group was more likely to implement the systems provided during the training program than its Native counterpart.

While we did not detect a significant difference in knowledge gained (Native and non-Native knowledge gain and retention was approximately the same), qualitative questions on the post-survey indicated that there was a difference in how the two groups reacted to the two presenters teaching styles. Table 3 summarizes the training program attendee perceptions of the presenters teaching style for each day of the two-day program. The Native-American attendees clearly had a preference for Presenter 1, while non-Native attendees preferred Presenter 2.

Discussion

The goals of the Crisis and Risk Communication Conference were to identify different levels of fear associated with a health crisis, to teach skills necessary to effective communication with the public during an environmental health crisis; to characterize common communication mistakes; to identify effective communication channels; to implement specific social marketing techniques for working with print, radio, and television; and to develop a social-marketing and prevention campaign (Loma Linda University, 2006). Our results suggest an overall difference in the way information from the conference was translated into practice among Native-American and non-Native-American groups. The difference in the mean responses on the pre-conference and post-conference surveys showed that 90 days following the conference, non-Native-American participants generally had implemented at least some of the strategies and tactics provided in the conference materials. By contrast, the Native-American group had generally not implemented the course materials, although they reported that the information was important. There was not a significant difference in the availability of funds or barriers to prevent implementation of the course materials to explain the difference in implementation. Both groups showed improved knowledge about crisis and risk communication in the post-conference questionnaire.

Differences in implementation of conference material cannot be explained by differences in gender or age of conference attendees. Statistical analysis showed that implementation of conference materials was not significantly different among males and females. Age was not a significant predictor; however, the age range of conference attendees was 20 years, substantially less than the age range of the environmental health workforce as a whole, which is greater than 30 years (U.S. Environmental Protection Agency, 2006).

A possible explanation for the difference in the application of the materials may be the relative difference in education level between the Native-American and non-Native-American groups. That is, the non-Native study subjects generally possessed a graduate degree, while the Native-American cohort generally had some college education but not a full degree. While a graduate degree is not requisite to effective implementation of communications programs, it is possible that graduate education provides complementary skills that contribute to the launching of new initiatives within an organization. Many other possible explanations also exist, such as organizational priorities, cultural factors, and job tasks.

The conference had two major speakers, Presenter 1 and Presenter 2, both of whom were nationally recognized authorities with very different tactics for communicating and interacting with the audience. Presenter 1 was very informal and conversational, providing few handouts or formal presentation aids. Conversely, Presenter 2 had a very traditional teaching style, using a slide show with accompanying handouts. Answers to qualitative questions suggested that Native Americans found the information provided by Presenter 1 to be more beneficial and applicable than that provided by Presenter 2. In fact, many of the Native Americans reported that Presenter 2 was difficult to follow and felt overwhelmed by the abundance of information. On the contrary, those in the non-Native-American group preferred Presenter 2. The non-Native-American group requested handouts or other aids to accompany the presentation materials of Presenter 1.

It is possible that preferences for one speaker over another are due to other factors such as content or gender of the presenter. The qualitative questions were broad, asking for conference attendees' opinions about their experiences on Day 1 and Day 2 of the conference. The comments returned all dealt with presenter style rather than content. Further questions investigated the difference in presenter style. Unfortunately our data will not support an evaluation of the impact of gender on attendees' perceptions of the presenters.

Another difference between the Native-American and non-Native-American groups was in perceptions of how to build on the foundational materials provided at the conference. In general, the non-Native-American group recommended that simulation drills and role playing be provided to help attendees visualize how to implement risk communication principles and skills. The Native-American group felt that the information presented in the conference was ample but requested additional information in areas specific to their spheres of concern, such as wildfires and earthquakes.

A weakness of our study is its small sample size. We used non-parametric statistical techniques customarily employed with small study population analysis, which, while defensible, should be considered in the context of the limited number of individuals who completed the pre- and post-conference questionnaires. While the authors were very pleased with overall Native-American conference attendance, many of the tribes sent alternates to attend the program, thus negating the pre-program interviews, which were essential for our pre-program and post-program analyses.

Anecdotally the authors were delighted with the level of social and professional exchange between the Native-American and non-Native-American workshop attendees. The conference provided a safe environment for an exchange of ideas and perspectives related to public health communications. Our initial results suggest, however, that customized training and follow-up may be the most effective approach for Native American environmental health professionals. In the future, the authors will consider convening general environmental health training programs for all environmental health professionals, with subsequent breakout sessions committed to the unique Native-American issues. In this fashion we aim to preserve the value of having sovereign-nation and non-Native-American training participants exchange ideas and perspectives, while retaining the opportunity to drill down into particular issues of significance.

This study is the first in a series in which the authors hope to quantifiably identify independent and dependent variables associated with the assessment of positive changes in knowledge, skills, and abilities related to professional-development courses. Our long-term aim is to employ the lessons learned from this effort to scale up for a larger study that would prospectively follow trainees over time to determine how training is being used in the workplace and to quantify benefits (e.g., efficiencies in service delivery and reductions in illness and injury) directly linked with training program attendance.

Conclusion

The results of our study indicate that Native Americans and non-Native Americans equally benefited from a gain in knowledge associated with attendance at a risk communication program. Ninety days after the training program, however, non-Native Americans applied new skills more frequently than did their Native-American counterparts. While teaching styles may partially contribute to the disparities in implementation rates, additional research is needed to identify barriers to implementation and enabling mechanisms that could enhance training effectiveness. Our study is the first step in the assessment of environmental health training outcomes--that is, in measuring changes in workplace performance as a result of attendance at professional-development courses.

Corresponding Author: David T. Dyjack, Interim Dean, Loma Linda University School of Public Health, 10970 Parkland Avenue, Loma Linda, CA 92350. E-mail: [e-mail].

References

Boatright, D. (1995). Environmental health education: What's wrong with this picture? Journal of Environmental Health, 58(1), 33-35.

Loma Linda University, School of Public Health. (2006). Crisis and risk communication conference. Retrieved October 14, 2006, from http://www.llu.edu/llu/sph/cpe/healthy/crc06.html.

Herring, M.E. (2006). Developing the environmental health workforce. Journal of Environmental Health, 68(10), 56-57.

Public Health Functions Steering Committee. (1995). Public health in America. Retrieved October 20, 2006, from http://www.health.gov/phfunctions/public.htm.

Sarpy, S.A., Chauvin, S.W, & Anderson, A.C. (2003) Evaluation of the effectiveness of the south central center for public health preparedness training. Public Health Reports, 118, 568-572.

U.S. Environmental Protection Agency. (2006, April). EPA's workforce--Age distribution. Retrieved April 17, 2007, from http://www.epa.gov/ohr/age.htm.

Sarah Hess, M.P.H.(C)

David T. Dyjack, Dr.P.H., C.I.H.

Jesse Bliss, M.P.H.

TABLE 1 Demographic Characteristics

                                             Native         Non-Native
Characteristic                               American (%)   American (%)

Education > college degree                   33.3 (1 of 3)  90.1
Risk communication experience > novice       33.3 (1 of 3)  30.0
Received prior crisis management training    66.7 (2 of 3)  30.0
Past experience managing a crisis situation  66.7 (2 of 3)  30.0
Availability of funds for crisis planning >  66.7 (2 of 3)  60.0
  available but hard to access

TABLE 2 Significance of the Differences Between Questionnaire Responses
of the Two Groups 90 Days After the Training

Questionnaire Item                             p-Value

Appropriate communication network in place     .0330 (Fischer's exact)
Communication with ICC or emergency personnel  .0330 (Fischer's exact)
Appropriate communication protocol within the  .0330 (Fischer's exact)
  organization in case of crisis situation
Applicability of material                      .0110 (Fischer's exact)
Presenter 1 Content Knowledge Question 1       .3599 (Wilcoxon exact)
Presenter 1 Content Knowledge Question 2       .2088 (Wilcoxon exact)
Presenter 2 Content Knowledge Question 1       .0852 (Wilcoxon exact)
Presenter 2 Content Knowledge Question 2       .2088 (Wilcoxon exact)

TABLE 3 Positive Impressions of Conference Presenters by Native-American
and Non-Native-American Attendees

             Positive Impressions Among  Positive Impressions Among
             Native-American Attendees   Non-Native-American Attendees
Presenter    (%)                         (%)

Presenter 1  100.0                       27.3
Presenter 2    0.0                       81.8