A comparison of food safety knowledge among restaurant managers, by source of training and experience, in Oklahoma County, Oklahoma.

Introduction

The Centers for Disease Control and Prevention (CDC) has estimated that each year 76 million illnesses, 325,000 hospitalizations, and 5,000 deaths are due to foodborne illnesses, with a majority of the reported cases being traced to public eating establishments (Council for Agriculture Science and Technology, 1994; Economic Research Service, 1994; Riben et al., 1994). The increase in the number of people patronizing restaurants has been accompanied by an increased risk of illnesses being transmitted by food handlers (Hedberg, MacDonald, & Osterholm, 1994). Human susceptibility to foodborne disease is universal, and most cases occur through consumption of food or water contaminated with microorganisms, primarily bacteria or viruses (McSwane, Rue, & Linton 2000; Nadakavukaren, 1995). In an era of frequent travel and international distribution of food products, along with increased emphasis on streamlining trade through minimizing import restrictions, safe food-handling practices are imperative given the potential for widespread outbreaks of foodborne illness.

The two practices of retail establishments that are most commonly reported as contributing to foodborne illness are improper holding and storage temperatures and poor personal hygiene among food handlers (Collins, 2001). Recognizing these areas of concern, the National Restaurant Association (NRA) has asserted that the most powerful weapon to combat food-handling problems is training directed toward the approximately, eight million employees in the fourth largest industry in the United States (Lynn, 1996). The Food and Drug Administration (FDA), with support from enforcement agencies and the food industry, has endorsed food service worker training since 1976; however, since that time, the retail food service industry (in conjunction with regulatory agencies), has intensified efforts to improve retail food safety through training of restaurant managers and employees.

Several studies have examined the effectiveness of food safety education in reducing the frequency of violations associated with foodborne illness; they have produced inconsistent and inconclusive results (Casey & Cook, 1979; Clingman, 1976; Kneller & Bierma, 1990; Riben et al., 1994; Wright & Feun, 1986). A study that compared food handler education with restaurant inspection scores in several Canadian provinces demonstrated better inspection scores when food handlers and supervisors had completed education courses (Mathias, Sizto, Hazlewood, & Cocksedge, 1995). A comparison of hazard and performance ratings between a group that had undergone a food safety program and a group that had not showed that completion of the food safety program improved performance (Stevenson, 1987). While many studies showed similar results (Clingman, 1976; Nabali, Bryan, Ibrahim, & Atrash, 1986; Palmer, Hatlen, & Jackson, 1975), others indicated no reduction in inspection violations as a result of food safety training (Addis, Yashuk, Clapp, & Blake, 1989; Teel, Fowler, & Jacobs, 1981). Mathias et al. (1994) found no significant association between the number of individuals trained in food safety and reported inspection violations or reported cases of foodborne illness; they concluded with a recommendation that the effectiveness and efficiency of the education program be further evaluated.

Education/Training in Oklahoma

In Oklahoma, as in other states, regulatory agencies and industry share the burden of preventing and managing foodborne illness. Oklahoma state regulations mandate that 1) a food service establishment license holder be the person in charge of the food service establishment, or designate a person in charge, and ensure that the person in charge is present at the food service establishment during all hours of operation; and 2) during inspection, the person in charge of the food service establishment be able to demonstrate knowledge of foodborne-disease prevention, application of hazard analysis critical control point (HACCP) principles, and knowledge of Oklahoma's food service establishment regulations (Oklahoma Administrative Code, 2002). The person in charge may demonstrate required knowledge through compliance with the department's rules, by being a certified food protection manager who has passed an accredited test, or by responding correctly to an inspector's questions relating to the specific food operation at the time of inspection. In Oklahoma County, the department of health follows the manager training plan and criteria endorsed by FDA (City-County Health Department of Oklahoma County, 1998). Nine municipalities in Oklahoma County require each food establishment to have at least one properly trained food service operator (FSO) available. FSO certificates can be obtained in three ways.

1: Operators may take a course currently offered at two area colleges.

2: Experienced operators may take a "challenge" examination at the City-County Health Department; a passing score is 75 percent.

3: Operators may obtain an FSO certificate through reciprocity. A current certificate from another food service sanitation course could be approved for reciprocity (Educational Foundation of the National Restaurant Foundation, 1992).

The food service manager training course and certification exam cover the following topics:

* sanitary facilities,

* food protection,

* foodborne disease,

* cleaning/sanitizing,

* personal hygiene,

* safe food-handling practices,

* operational problems,

* self-inspection,

* motivation, and

* personnel training.

According to the 1998 City-County Health Department of Oklahoma County Consumer Protection Annual Report, reports of foodborne illness in Oklahoma County have increased each year lot the past several years (Table 1). A specific example of a foodborne-illness outbreak involved a cluster of Campylobacter jejuni infections that occurred during the period of August 16-20, 1996, among people who ate lunch in an Oklahoma City restaurant (Graves, Bradley, & Crutcher, 1998). The ensuing investigation indicated that the C. jejuni infection was most likely acquired from eating lettuce cross-contaminated with raw chicken. The report concluded that ensuring that restaurant employees receive training in food safety could reduce risks of foodborne illness in restaurants.

Despite considerable progress in training, foodborne illness continues to be a public health problem. Oklahoma County does not have an education follow-up plan designed to determine the current level of knowledge of food service workers or gauge the effectiveness of the FSO certification training program. To date, no evaluation of food service managers' baseline knowledge of food safety criteria has been conducted.

The study reported here therefore sought to evaluate basic food safety knowledge among those in charge of facility operations. In addition, the analysis looked at levels of knowledge among individual FSOs with different levels of experience and trained by different sources. Finally,; recognizing that the FSO of a facility is not required to be present during operations, the authors examined differences in knowledge between FSOs and their designees.

Methods

The study team developed a questionnaire designed to measure a food service manager's general knowledge of food safety principles and criteria. Thirteen questions on basic food safety principles and the spread and prevention of foodborne illnesses were derived from food safety criteria and from Essentials of Food Safety and Sanitation (2nd edition), a text used in Oklahoma City/County food safety courses (McSwane et al., 2000) (Figure 1). The questionnaire asked each participant to identify the length of employment in the food industry, estimate the number of hours of food safety training received, estimate the time lapsed since food safety training had been received, specify whether the training was from a government health agency or a corporate training source, identify the type of training last received, and indicate whether the participant was a certified FSO.

Over a three-month period, health inspectors supplied questionnaires during routine inspections to the person identified as manager at food service establishments. Oklahoma City/County health inspectors randomly inspect approximately 1,000 medium- and high-risk food service establishments per month. Upon completion, the questionnaires were placed in a sealed envelope and returned to the study group; managers not willing to participate were asked to return blank questionnaires.

Questionnaires were coded by a single individual and entered into a database; questions left blank were coded as missing information. After the data were reviewed for entry errors, individual scores were determined as the percentage of questions answered correctly, and descriptive statistics were generated with Microsoft Excel[TM]. Results from different groups of respondents were compared with the Wilcoxon ranked sum test, with [alpha] = .05 established as the level of significance.

Results

A total of 231 surveys were returned (193 completed, 38 blank), yielding an 83.5 percent response rate. Survey scores were determined (calculated) as the percentage of questions answered correctly (Table 2). The mean score for completed surveys was 87.2 percent.

The results indicate that, overall, food safety knowledge was high (87.2 percent), although in some subject areas there were significant deficiencies of knowledge. Different groups gave statistically different levels of performance (Table 3). The respondents who had received both health department and corporate training had the highest scores (average score = 92.8), while the group that had received no formal training had the lowest scores (average score = 79.5 percent). Managers with health department training performed almost 10 points better than those who had received corporate training. Individuals holding an FSO certification significantly outscored uncertified respondents.

The number of hours of training had little effect on scores. The mean score of the group with more than 40 hours of training was not significantly different than that of the group that had received less than 10 hours of training. It is encouraging to note that almost 22 percent of participants had received more than 40 hours of training and that 76 percent had received more than 10 hours.

Although it is widely accepted that the average age and years of experience are low among food service workers, the managers participating in this study had a significant base of experience; 75 percent of respondents had five or more years of experience in the industry, while 15 percent had more than 20 years experience. Mean scores increased with experience, except in the case of the group with tenures in the industry of more than 24 years, and those with one to four years of experience had a significantly lower score than those with longer tenures (more than 20 years).

Training appeared to be an ongoing process, with 56 percent of respondents having received training within 15 months of the survey and 28 percent within the last five months. Only 16 percent of the respondents had not received training within the previous 2.5 years. A comparison of the knowledge base of those recently trained (less than six months prior to the survey) with those trained more than 25 months prior to the survey indicated no significant difference in scores (p = .0970).

Groups were also compared according to the percentage of correct responses given to individual questions (Table 4). Not surprisingly, the group that had not received formal training performed poorest on many survey questions. Half of this group could not identify proper handwashing techniques (as a whole, only 75 percent of study participants could identity proper handwashing methods). This group also had much less understanding of temperature and food storage requirements. For all survey participants, Question 8, which dealt with recognizing hepatitis A symptoms, was the question most commonly answered incorrectly (31 percent gave incorrect answers). Questions dealing with temperatures and food storage were also often answered incorrectly by more than one group. The group with corporate-only training actually fared worse than the untrained group on Question 1, which dealt with identification of the temperature danger zone. This group also did not have a good understanding of appropriate storage of meats in refrigeration units (Question 11).

Discussion and Conclusions

Because of the nature of regulations in Oklahoma, people in charge of food service establishments may not have received formal food safety training, may not be certified, and may have limited or extensive experience. Although the survey discussed in this paper was designed to assess only basic food safety knowledge, it was apparent that overall knowledge was generally good for those in charge of facilities. In addition, the survey demonstrated the benefits of both training and certification. Results also indicated that food service managers with food safety training had more knowledge of food safety criteria than did managers without formal training, and that food service managers with FSO certification had greater knowledge of food safety, criteria than did non-FSO-certified managers.

Health department training appeared to be most effective in delivering food safety education. Food service managers with health department training demonstrated a greater knowledge of food safety criteria than did food service managers with corporate training. Food service managers with health department training scored just slightly below managers with both health department training and corporate training, a result suggesting that additional corporate training was not effective in increasing knowledge (or that the training was redundant).

A relationship was observed between years of food service experience and knowledge of basic food safety, indicating that food service managers with more years of experience tended to have a greater knowledge of food safety criteria. The retention of knowledge was not significantly affected by the number of months elapsed since training or the number of hours of training received. The lack of improvement associated with increased hours of training does not mean that training was not helpful; it may rather indicate that the survey was too basic to be sensitive to such improvements. Given that nearly 70 percent of respondents had received some training within the past two years, in a context of relatively long tenure among most managers surveyed, it appears that training occurs with some regularity.

The survey question that dealt with recognizing hepatitis A symptoms was frequently missed, with all groups demonstrating a poor knowledge of this subject. Given the epidemic state of this disease in Oklahoma, it is extremely important that food service managers have a robust knowledge of this subject. Clearly, the training being provided is not sufficient to adequately inform the intended audience.

Food service managers also had limited knowledge of optimal temperature ranges for bacterial growth and appropriate temperatures for heating and cooling foods. Since inadequate cooling and heating of foods have been identified as the most common source of foodborne-disease outbreaks, it seems axiomatic that managers should have accurate, memorized knowledge of these factors.

Recognizing the various possibilities for food service manager qualifications, the authors designed the study to assess the basic food safety knowledge of food service managers in charge at randomly selected facilities. Since the survey was conducted as part of the routine restaurant inspection process, some degree of random sampling was accomplished. In addition, the response rate was high; therefore, results should be generalizable to facilities throughout Oklahoma County. One potential source of error is that individual inspectors were not required to identify themselves on the surveys they distributed to their areas of jurisdiction; therefore, it is not possible to ascertain the level of participation by facilities visited or by inspector. Given the length of time over which the survey was distributed and the average workload of inspectors, it may be inferred that some sanitarians did not participate, possibly biasing the outcome toward more educationally oriented inspectors. Other potential errors may have been introduced if inspectors did not understand the principle of distribution to only high- and medium-risk facilities. Finally, the analysis is limited by the small number of respondents in some groups.

With the range of qualification of food service managers, it was not surprising that a wide range of responses was received. The overall knowledge of participants was high; however, a concern is the large number of food service managers who indicated that they had received no formal food safety training. With he growing importance of food safety, it seems crucial that a minimum level of training, qualifications, or competencies be achieved by those responsible for the operation of retail food establishments. To determine the effects of any changes in training and educational criteria, future surveys could be conducted in Oklahoma County and compared against the survey data, or the results of this study could be judged against those from a similar survey in a jurisdiction with different educational or qualification requirements.

TABLE 1

Number of Foodborne-Illness
Reports Received in Oklahoma
County, Fiscal Years 1995-1998

               Number of Reports
 Fiscal Year       Received

    1995             176
    1996             194
    1997             242
    1998             265

Source: City-County Health Department of Oklahoma County, 1998.

TABLE 2

Survey Scores for Different Groups

                                                 Standard
Group                                 N   Mean   Deviation

All participants                     193  87.2    13.0
Any training                         169  88.5    11.9
No training                           24  79.5    17.2
FSO certification                    133  89.0    11.6
No FSO certification                  59  82.8    14.8
Health department training            64  90.1     9.4
Health department/company training    48  92.8     7.0
Company training                      59  82.1    16.1
1-10 hours of training                46  88.1    11.0
11-20 hours of training               40  86.9    11.7
21-30 hours of training               16  82.2    17.0
31-40 hours of training               24  89.7    11.1
>40 hours of training                 42  91.0    10.8
1-4 years of experience               49  82.5    14.4
5-8 years of experience               43  86.1    14.5
9-12 years of experience              22  87.2    12.8
13-16 years of experience             34  89.5     9.9
17-20 years of experience             16  94.4     6.6
21-24 years of experience             12  94.4    15.4
>24 years of experience               17  89.9     9.5
Time since training (1-5 months)      55  85.0     8.3
Time since training (6-10 months)     29  90.1    10.1
Time since training (11-15 months)    24  84.2    17.1
Time since training (16-20 months)     7  86.8    15.2
Time since training (21-25 months)    20  90.4    12.5
Time since training (26-30 months)     4  88.5     9.9
Time since training (>30 months)      30  89.0     9.9

TABLE 3

Summary Analysis of Group Comparison (Wilcoxon Rank-Sum Test)

Comparison                                          p Value

Training versus no training                          .0129
FSO certified versus non-FSO certified               .0034
Health department training versus company
 training                                            .0026
Health department training versus health
 department and company training                     .1563
1-10 hours training versus >40 hours training        .1071
Recent training versus >25 months since training     .0974
1-4 years experience versus >24 years experience     .0194

TABLE 4

Percentage of Correct Answers to Individual Survey Questions,
by Survey group

                               Survey Group

               Total      With     Without   Health Dept.
                        Training   Training    Training

Question 1      83.9       85.7        75        93.8
Question 2      91.7       93.5      79.2        93.8
Question 3      94.8       95.8      87.5         100
Question 4      75.1       78.6        50        79.7
Question 5      83.9       84.5      79.2        85.9
Question 6      92.2         94      79.2        96.9
Question 7      94.8       96.4      87.5        95.3
Question 8      69.4       70.2      62.5        68.8
Question 9      84.4       83.3      91.7        90.6
Question 10     94.3       95.8      83.3        96.9
Question 11     78.2       79.8      66.7        81.2
Question 12     93.8       93.4      95.8        92.2
Question 13     96.4       96.4      95.8        95.3

             n = 193    n = 169    n = 24      n = 64

                          Survey Group

              Company     Health      FSO     Non-FSO
              Training  & Company

Question 1      72.9      85.4       84.9        81.4
Question 2      89.8      95.8       93.2        88.1
Question 3      86.4      97.9       96.2        91.5
Question 4      67.8      87.5       78.9        66.1
Question 5      83.1      83.3       85.7        79.7
Question 6      88.1      97.9       96.2        93.1
Question 7      94.9      95.8       95.5        93.2
Question 8      71.2      70.8       69.9        67.8
Question 9      69.5      87.5       86.5        79.7
Question 10     93.2      95.8       96.2        89.8
Question 11     69.5      91.7       81.9        69.5
Question 12     88.1       100       95.5        89.8
Question 13     94.9      97.9       96.2        96.6

              n = 59    n = 48    n = 133      n = 59

n = Total number of surveys filled out by each group.

FIGURE 1

Food Safety Knowledge Questionnaire

1. What is the temperature range over which bacteria
grow most quickly?

a. 80[degrees]F to 100[degrees]F
b. 98.6[degrees]F
c. 40[degrees]F to 140[degrees]F
d. Higher than 140[degrees]F

2. What is the safest way to defrost meat, poultry,
and fish products?

a. By heating in an oven
b. At room temperature
c. In a refrigerator below 41[degrees]F

3. Which of the following should you choose to
prevent the passing of bacteria from one food
to another:

a. Sanitize cutting boards between raw and
cooked foods
b. Turn cutting boards over between raw and
cooked foods
c. Rinse cutting boards with hot water between
raw and cooked foods
d. Use wood cutting boards instead of synthetics

4. Proper hand washing technique includes:

a. Soaking hand in hot, soapy water for at least
a minute
b. Briskly scrubbing hands, forearm, and between
fingers
c. Scrubbing hands with a disinfectant wipe
(example Wet-Ones[R])

5. Cold foods must be held at or below what temperature?

a. 32[degrees]F
b. 41[degrees]F
c. 56[degrees]F
d. Room temperature

6. Cooked foods must be held at or above what
temperature?

a. 45[degrees]F
b. 65[degrees]F
c. 100[degrees]F
d. 140[degrees]F

7. Dry and canned foods can be stored on the
floor if in original containers.

a. True
b. False

8. A worker infected with hepatitis A:

a. Will usually not report to work because of
illness
b. Will appear to have fever and chills
c. Often has no symptoms

9. Workers must wear gloves:

a. When delivering food to customers
b. When handling read-to-eat-foods
c. When handling money
d. When taking out the garbage

10. Gravy is cooked to 165[degrees]F in a 4-gallon container.
How should it be cooled?

a. On the counter in the original container
b. Keep in original container and place in
refrigerator
c. Transfer gravy to shallow pan and place in
refrigerator

11. Uncooked meat in the refrigerator should be
stored:

a. On the floor
b. On high shelves to prevent contamination
c. Below other foods
d. Doesn't matter where it's located

12. It is not necessary to wash hands after handling
raw meat to prepare a sandwich if you put
gloves on before making the sandwich.

a. True
b. False

13. Chemicals should be stored:

a. On a shelf over the dishwashing sink
b. In the dry food storage area
c. In a cabinet away from food handling areas

Acknowledgements: This project was approved by the University of Oklahoma Health Sciences Center Institutional Review Board (IRB#09239). The authors gratefully acknowledge staff from the Consumer Protection Division of the Oklahoma City-County Health Department (OCCHD) for their assistance in preparation and distribution of the survey. Special thanks go to Ms. Marla Sweaney of OCCHD for her assistance in all aspects of the project.

REFERENCES

Addis, D.G., Yashuk, J.C., Clapp. K.E., & Blake. P.A. (1989). Outbreaks of diarrheal illness on passenger cruise ships. Epidemiology and Injection, 63(1), 63-72.

Casey, R., & Cook, C. (1979) Assessment of a food service management saturation course. Journal of Environmental Health, 41(5), 281-284.

City-County Health Department of Oklahoma County. (1998). Consumer protection annual report. Oklahoma City, OK: Author.

Clingman, C.D. (1976) Ohio evaluates effects of food certification training. Journal of Environmental Health 38(4), 235-236.

Collins, J.E. (2001). Impact of changing consumer lifestyles on the emergence/reemergence of foodborne pathogens. Emerging Infectious Diseases, 3(4), 1-13.

Council for Agriculture Science and Technology. (1994) Foodborne pathogens: Risks and consequences: Task force report (Report No. 22), Ames. IA: Author.

Economic Research Service (1996). Bacterial foodborne disease: Medical costs and productivity losses (Report No. 741). Washington. DC: Author.

Educational Foundation of the National Restaurant Association. (1992). Applied foodservice sanitation: A certification coursebook (4th ed.). New York: John Wiley and Sons. Inc.

Graves, T.K., Bradley, K.K., & Crutcher, J.M. (1998). Outbreak of Campylobacter enteritis associated with cross-contamination of food--Oklahoma, 1996. Morbidity & Mortality Weekly Report, 47(7), 129-131.

Hedberg. C.W., MacDonald, K.L., & Osterholm, M.T. (1994). Changing epidemiology of food-borne disease: A Minnesota perspective. Clinical Infectious Diseases, 18(5), 671-682.

Kneller. P, & Bierma, T. (1990). Food service certification measuring the effectiveness of a state program. Journal of Environmental Health, 52(5), 292-294.

Lynn, J. (1996, September). Better safe than sorry: Training food handlers protects everyone. Restaurants USA. http://www.restaurant.org/rusa/magArticle.cfm?ArticleID=182 (15 Mar. 2001).

Mathias. R.G., Riben, PD., Campbell, E., Wiens, M., Cocksedge, W., Hazelwood, A., Kirshner, B., & Pelton, J. (1994) The evaluation of the effectiveness of routine restaurant inspections and education of food handlers: Restaurant inspection survey. Canadian Journal of Public Health, 85(Suppl. 1), S61-S66.

Mathias. R.G., Sizto, R., Hazlewood, A., & Cocksedge, W. (1995). The effects of inspection frequency and food handler education on restaurant inspection violations. Canadian Journal of Public Health, 86(1), 46-50.

McSwane, D., Rue, N., & Linton, R. (Eds.). (2000). Essentials of food safety and sanitation (2nd ed.). Upper Saddle River, NJ: Prentice Hall.

Nabali, H., Bryan, F.L., Ibrahim, J., & Atrash, H. 1986). Evaluation of training food service managers in Bahrain. Journal of Environmental Health, 48(6), 315-318.

Nadakavukaren, A. (1995). Our global environment: A health perspective (4th ed.). Prospect Heights. IL: Waveland Press.

Oklahoma Administrative Code, (2002). Food service establishment regulations Subchapter 3: Management and personnel, Section 2: Demonstration. http://www.health.state.ok.us/program/cpd/256.pdf (27 Apr. 2003).

Palmer, B.J., Hatlen, J.B., & Jackson, B.B. (1975). The implementation and evaluation of management training in a fast food restaurant chain. Journal of Environmental Health, 37(4), 364-368.

Riben, P.D., Mathias, R.G., Wiens, M., Cocksedge, W., Hazelwood, A., Kirshner, B., & Pelton, J. (1994). Routine restaurant inspections and education of food handlers: Recommendations based on critical appraisal of the literature and survey of Canadian jurisdictions on restaurant inspections and education of food handlers. Canadian Journal of Public Health, 85(Suppl. 1), S67-S70.

Stevenson, R.K. (1987). A food service establishment evaluation program procedure for the 1980's and 1990's. Journal of Environmental Health, 50(1), 25-28.

Teel, J.H., Fowler. P., & Jacobs. D. (1981). An assessment of the efficacy of a comprehensive plan review process in food service establishments. Journal of Environmental Health, 44(1), 15-18.

Wright, J., & Feun, L. (1986). Food service manager certification: An evaluation of its impact. Journal of Environmental Health, 49(1), 12-15.

Corresponding Author: Robert Lynch, Associate Professor, Department of Occupational and Environmental Health, College of Public Health, University of Oklahoma Health Sciences Center, 801 N.E. 13th Street, Oklahoma City, OK 73104. E-mail: [e-mail].