Recurrent critical violations of the food code in retail food service establishments.

Introduction

Restaurant inspection is a public health activity intended to prevent foodborne illness. In addition to its primary function as a public health intervention, the inspection process generates a written record of conditions observed in food service establishments. Inspection records have been used retrospectively in studies to assess the effect that increased inspection frequency and food handler training have on inspection scores (Campbell et al., 1998) and the effect that altered enforcement consequences have on inspection scores (Fielding, Aguirre, & Palaiologos, 2001). Inspection records have also been used in case-control studies of the antecedents of foodborne-illness outbreaks in restaurants in the United States and the United Kingdom. Studies in Seattle & King County (Irwin, Ballard, Grendon, & Kobayashi, 1989) and Los Angeles County (Buchholz, Run, Kool, Fielding, & Mascola, 2002) found a significant relationship between the occurrence of foodborne-illness outbreaks from food service establishments and violations, low inspection scores, or both; in contrast, a case-control study of Chinese restaurants in Scotland did not find a significant relationship between inspection scores and outbreaks of infection with two rare strains of Salmonella enteritidis (Mullen, Cowden, Cowden, & Wong, 2002).

[ILLUSTRATION OMITTED]

Although inspection reports present a rich source of information on operations at retail food service operations, the completeness and consistency of inspections are open to question. In addition, even the most thorough inspection by a public health agency represents only a "snapshot" of operations at a food service establishment. Day-to-day protection of public health relies upon the implementation of food safety practices by the food service establishments themselves. The study reported here was based on the premise that by viewing these "snapshots" sequentially, investigators could identify differences among establishments in routine food safety performance.

Recurrent violations of selected Food Code items during a defined period were used as a key measure of food safety performance. The occurrence of repeat violations at an establishment is direct evidence that the establishment has not responded effectively to a previous citation. It must be recognized that the inverse is not true; because the inspection record is only a series of "snapshots," the failure of the record to show a recurrent violation in a given establishment does not imply that the establishment was completely effective at preventing recurrences. As the study showed, however, nonrandom patterns in the distribution of violations can be used to identify relative differences in performance among establishments by category, if not individually.

Food service establishments in Oklahoma are classified by the health department as "high risk," "medium risk," or "low risk" according to unstandardized subjective criteria. "Low-risk" establishments are typically food stores or snack bars, with very limited food service. The distinction between "medium-risk" and "high-risk" establishments is not so clear. Therefore, another objective of the study was to determine whether there was a difference in food safety performance between "high-risk" and "medium-risk" establishments.

Another question of potential interest to the public is whether there are differences in food safety performance between chain restaurants and local or independent restaurants. It might be hypothesized that chain restaurants would demonstrate better food-handling practices than local or independent restaurants because regional or national chains may have more standardized practices and a wider reputation at stake. On the other hand, it could be argued that all restaurants face similar challenges in ensuring proper food-handling practices, and therefore local, regional, and national chain restaurants would have similar violation rates. This hypothesis was tested in the study.

In Oklahoma, three different public health departments are responsible for inspection of retail food service establishments. The Oklahoma City-County Health Department (OCCHD) has jurisdiction over the most populous urban county in the state, which has a population of 660,000, according to the year 2000 census. Tulsa (county population 560,000) is under the jurisdiction of Tulsa City-County Health Department. The remaining 75 counties in Oklahoma, with a combined population of 2,230,000, are under the jurisdiction of the Oklahoma State Department of Health (OSDH). OSDH sanitarians are assigned to counties in rough proportion to the populations of the counties. Thus, at any given time, several sanitarians may be assigned to each of the more populous counties, while a single sanitarian may be assigned to cover all inspections in one or more rural counties. Another objective of the study was to assess consistency of inspection practices between jurisdictions and among inspectors.

Inspections are conducted in enforcement of the state food code (Food Service Establishments, 2004), which is based on the U.S. model Food Code (U.S. Public Health Service, 2001) and is revised periodically. Between 1990 and 1999, the inspection report forms used by Oklahoma sanitarians listed 44 numbered items that could be cited as violations, including 13 items that were designated as critical. A concise description of the critical items is provided in Table 1. A major revision of the OSDH inspection form was made in 1999; items were renumbered, and the designation "critical" was removed. An analysis of trends in violation rates between 1990 and 2000 indicated that the frequency with which some violations previously designated as "critical" were cited, including pest and personnel violations, decreased in 1999-2000, possibly in response to the change in the inspection form; other critical violations, including holding temperature and hygiene practices, apparently were not affected (Elledge, Basara, Boatright, Lynch, & Phillips, 2002). Clearly, when inspection data were being analyzed for differences in the food safety practices of retail establishments, it was necessary to control for inconsistencies arising from changes in health department policy and practice over time, as well as possible inconsistencies in inspection practices among individual inspectors.

Methods

A random sample of inspection files for food service establishments was collected from Oklahoma County and from 30 counties under the jurisdiction of OSDH. Each food service establishment had its own file, consisting of its health department inspection reports. Ten percent of establishment files were sampled in each county. A minimum of five years of inspection data were required for an establishment to be included in the sample. Collectively, the files in the sample spanned the years 1986-2000 for Oklahoma County and 1989-2001 for OSDH.

Data from the files were originally entered into a Microsoft Excel 1998 spreadsheet. For each inspection, the abstracted data included the name, identifier number, and address of the establishment; the date of the inspection; the inspector's identification number; the reason for the inspection (i.e., initial, complaint, follow-up, routine, or emergency); the risk category of the establishment; and whether a citation was recorded under any of 44 items of the state food code. For OSDH inspections recorded on the revised inspection sheets that did not explicitly list the 44 items, the authors categorized each cited violation as one of the 44 items according to the description of the violation on the sheet. Citation for violation of a given item (e.g., proper food-holding temperature) in a given inspection was a dichotomous (yes/no) variable.

Establishments were classified as local, regional chain, or national chain according to the following criteria:

[FIGURE 1 OMITTED]

* Local establishments were either independent businesses or had sister establishments within the state of Oklahoma only.

* Regional chains had outlets in more than one state but in no more than two quadrants of the continental United States.

* National chains had outlets in at least three quadrants of the continental United States.

The "local" category included outlets of small chains found only in Oklahoma.

Because of known differences in administration and training between OCCHD and OSDH, the data from these jurisdictions were analyzed separately. To control for trends over time (because of the changes in the food code, the inspection form, and sanitarian training mentioned in the introduction), the analysis was restricted to inspection records from 1996-2000 for establishments that had records spanning at least 3.5 years of that five-year period. The adequacy of this control was checked by an examination of the consistency in the proportion of establishments that fell into each category over time. The year 1996 was chosen as the starting point for the study because OSDH implemented the FDA Level 1 Training Plan for its sanitarians in 1995. A total of 178 medium- and high-risk establishments, with 2,297 total inspections, were included from OSDH files; they comprised 116 local establishments, 20 regional-chain establishments, and 42 national-chain establishments. In Oklahoma County, 140 medium- and high-risk establishments, with 1,747 inspections, were included, of which 85 were local establishments, 14 belonged to regional chains, and 41 belonged to national chains. The establishments were mostly restaurants, but included some food stores, canteens, and caterers.

Statistical Analyses

All data manipulations and calculations were conducted in Microsoft Excel 2002.

The authors defined the violation rate as the number of times a violation was cited for a given category of establishment divided by the number of inspections for that category. All inspections were used in the denominator regardless of the reason for the inspection. The total critical-violation rate was defined as the total number of critical violations cited for a given category divided by the number of inspections for that category. Recurrent violations were defined as violations for which a given establishment was cited more than once in the period 1996-2000.

[FIGURE 2 OMITTED]

To determine whether the violations of the most frequently cited critical items were randomly distributed among establishments, the authors compared a theoretical random distribution of violations among the establishments to the observed distributions. The theoretical distribution was based on a null hypothesis that violations of a specific critical item were random, independent events across all establishments. Under these circumstances, each inspection functions as an independent trial with a dichotomous outcome (citation/no citation) for any particular critical item, and the probability of any establishment receiving a citation for that item in any given inspection is equal to the overall violation rate, [theta], for that item. The results of independent trials with dichotomous outcomes can be modeled with a binomial distribution. If a given establishment was inspected a total of n times during the study period, it could theoretically receive any number of citations from zero to n. The probability of the establishment receiving a total of k citations in n inspections would be given by the binomial distribution:

n!/[k! (n - k)!] [[theta].sup.k] (1 - [theta])[.sup.n - k]

A binomial distribution was calculated for each establishment on the basis of the actual number of inspections conducted at the establishment. Then the probability of each establishment having k = 0 violations was added up over all establishments to yield the number of establishments expected to have zero violations under the null hypothesis. Likewise, the probability of each establishment having k = 1 violations was added up over all establishments to yield the number of establishments expected to have one violation, and similarly for k = 2, k = 3, and so forth, to produce the expected distribution of violations among establishments under the null hypothesis. The expected distribution of violations was then compared with the observed distribution through a Chi-square goodness-of-fit test. For the Chi-square test, cells representing establishments with multiple (i.e., two or more) violations were collapsed as needed to obtain expected values of five or more in each cell. The null hypothesis was rejected if the p-value of the Chi-square statistic was less than .05. (A Microsoft Excel worksheet showing how the expected distributions were calculated and compared with the observed distributions can be obtained from the corresponding author upon request.)

Differences in citation rates among inspectors were evaluated by a contingency-table analysis. To ensure sufficient numbers of expected violations for the Chi-square method, this analysis was limited to the most frequently cited critical items and to inspectors with a minimum of 3.5 years of data encompassing at least 75 inspections across 10 or more establishments. Eight inspectors from OSDH and 10 inspectors from OCCHD met these inclusion criteria.

The effect of variability among inspectors on the distribution of violations by establishment was evaluated through the correlation between the actual violation rate and the predicted violation rate for each establishment inspected by the selected inspectors. This analysis included 105 establishments with 1,273 total inspections in the OSDH jurisdiction and 151 establishments with 1,251 inspections in Oklahoma County. Failure to control for the effect of each individual establishment on an inspector's citation rate would lead to overestimation of the correlation. Therefore, the authors calculated the predicted number of violations for an establishment by multiplying the number of inspections at the subject establishment by the individual inspector's citation rate averaged over his inspections at all other establishments. The number of violations predicted for the establishment by inspector was then summed over all inspectors and divided by the total of number inspections at that establishment to yield the predicted violation rate.

Results

Overall critical-violation rates estimated from the sample of inspection records from OSDH and OCCHD during the period 1996-2000 are presented in Table 1, together with the percentage of violations that were repeat violations of the same critical item. Recurrent violations accounted for more than half the violations in the OSDH sample and 44 percent of violations in the OCCHD sample. The most frequently cited critical violations from OSDH involved food-holding temperatures, pests or protection of openings to the outdoors, and hygiene practices such as handwashing. The most frequently cited critical violations from OCCHD involved personnel, food-holding temperatures, and pests or protection of openings to the outdoors. Differences between the two jurisdictions in violation rates for these items were statistically significant, as were differences in the overall critical violation rates. The data from the two jurisdictions therefore were treated separately in subsequent analyses.

The distributions of violations among establishments in Oklahoma County and in the OSDH jurisdiction are shown in Figure 1 and Figure 2, respectively, for the most frequently cited items. In general, fewer than half the establishments were cited for any particular critical item during the study period, although only 8.6 percent of establishments in Oklahoma County and 10.7 percent of establishments in the OSDH jurisdiction received no citations at all for critical violations in this period. The number of inspections conducted at each establishment during the study period ranged from 4 to 28 in the OSDH jurisdiction and from 6 to 25 in Oklahoma County. If violations of a given critical item occurred randomly with uniform frequency across all establishments, some repeat citations still would be expected in some establishments. As described in the methods section of this article, a Chi-square test was performed for the most frequently cited critical items, comparing the observed distributions to the distributions expected under the null hypothesis of a uniform violation rate across all establishments. The selected items were holding-temperature, pest, and hygiene-practice violations in the OSDH jurisdiction and holding-temperature, pest, and personnel violations in Oklahoma County. The observed distributions of violations were found to be significantly different (p < .05) from the expected distributions for all of the selected critical items. All of the observed distributions showed an excess number of establishments with no violations, reduced numbers of establishments with one or two violations, and an excess number of establishments with six or more violations. Because the total number of violations was fixed in this model, a higher proportion of violations tended to occur as multiple repeat violations than would be expected under the null hypothesis.

Medium Risk Versus High Risk

Violation rates for critical items, by risk category, are given in Table 2. High-risk establishments had significantly higher violation rates than medium-risk establishments for nearly all critical items. One exception was that violations of the rule concerning personnel (requiring a "Food Safety Officer" and restricting personnel with infections) were more frequent in medium-risk establishments than in high-risk establishments in Oklahoma County. In both jurisdictions, high-risk establishments were inspected more frequently on average than medium-risk establishments.

Local, Regional-Chain, and National-Chain Establishments

In the OSDH jurisdiction, the total critical-violation rates by establishment type were 0.986 for regional chains, 0.551 for local establishments, and 0.424 for national chains. The differences among the three establishment types were all statistically significant (p < .001). In Oklahoma County, the total critical-violation rates were 0.556 for regional chains, 0.516 for local establishments, and 0.509 for national chains; these differences by establishment type were not statistically significant. The percentages of each type of establishment that had recurrent violations on selected critical items are shown graphically in Figure 3 and Figure 4 for Oklahoma County and OSDH, respectively. In the OSDH sample, the national chains and local establishments appeared to have significantly better performance than regional chains in food-holding temperature, hygiene practices, sanitization, and hygiene facilities. A similar though weaker disparity with respect to hygiene facilities was also observed in the Oklahoma County sample. In Oklahoma County, the regional- and national-chain establishments tended to receive more recurrent citations for personnel violations than did local establishments.

[FIGURE 3 OMITTED]

[FIGURE 4 OMITTED]

In both jurisdictions, the average number of inspections per regional-chain restaurant during the study period was about 14, which was slightly higher than the number for other restaurants in Oklahoma County (about 12 inspections per establishment) and outside Oklahoma County (about 13 inspections per establishment). The proportions of national-chain, regional-chain, and local establishments varied slightly over time; the largest variation was in the proportion of regional-chain establishments in the OSDH jurisdiction, which started as 9 percent of all establishments inspected January through June 1996, ended as 12 percent of all establishments inspected July through December of 2000, but was nearly constant otherwise.

Variability Among Inspectors

In counties under OSDH jurisdiction, typically just one sanitarian performed all of the inspections at any given establishment. Citation rates for holding temperature, hygiene practices, and pest violations differed significantly (p < .05) among inspectors. The largest range of citation rates was 0.005-0.324, for holding-temperature violations cited by selected OSDH inspectors. The tendency of individual inspectors to cite certain critical items at other establishments was modestly predictive of the violation rate at any given establishment; differences in inspectors' citation rates accounted for 30 percent of the variance among 105 establishments in holding-temperature violation rates, 11 percent of the variance in hygiene-practice violation rates, and 6 percent of the variance in pest violation rates.

In Oklahoma County, most establishments were inspected by two or three different sanitarians over the five-year period of the study. Citation rates for personnel, sanitization, and pest violations differed significantly among inspectors (p <.05). Differences in citation rates for holding-temperature violations were not significant. There was no significant correlation between the violation rates predicted on the basis of inspectors' citation rates and the actual violation rates for personnel, sanitization, and pests at 151 establishments.

Discussion

The authors found a nonrandom distribution of recurrent violations among food service establishments that could not be attributed primarily to differences in inspection practices.

Significant differences were found in citation rates among inspectors, but the significant differences in citation rates among Oklahoma County inspectors appeared to be the result, not the cause, of differences in violation rates among the establishments they inspected. In counties under the jurisdiction of OSDH, inconsistencies among inspectors accounted for a modest portion of the observed differences in violation rates among establishments, but even in this jurisdiction, the analysis suggested that real differences existed among establishments.

Regional-chain restaurants in the OSDH jurisdiction were found to have a significantly higher total-critical-violation rate than other restaurants and were more likely to have recurrent violations of some critical items. The magnitude of the differences in recurrent violations among restaurant types was too great to be accounted for by the fact that regional-chain restaurants were inspected slightly more often on average than were other restaurants. Relative to their overall proportion of establishments in the study, the regional chains were modestly under-represented in the first six months of the study period and over-represented in the last six months. Thus, time trends in enforcement were not completely controlled for across types of establishments; again, however, the differential effect of these time trends among categories would have been too small to account for the higher overall percentage of regional-chain restaurants with recurrent violations.

Conclusions

The statistical analysis of restaurant inspection data found that recurrent violations were indicative of real differences among conditions in establishments. The large number of repeat violations observed in the study reported here suggests that restaurant inspections alone have not been effective at promoting fundamental change in establishments with substandard food-handling practices. Yet restaurant inspection records contain useful data that can be exploited in the designing of interventions. The statistical methods demonstrated in this article can be used to identify individual establishments or types of establishments that can be targeted with enhanced food safety education, enforcement efforts, or both. Improvement in practices in establishments with a record of recurrent violations would dramatically reduce the overall rate of critical violations.

One encouraging finding of this study was that 8 to 10 percent of establishments had no critical violations in 1996-2000. This result suggests that some restaurants do consistently apply good food safety practices. These restaurants could be studied further to identify the factors that led to their good performance.

Acknowledgments: The authors thank the Oklahoma State Department of Health and the Oklahoma City-County Health Department for providing inspection records. This work was funded through the Environmental Health Services Branch of the National Center for Environmental Health at the Centers for Disease Control and Prevention (CDC) by a CDC/ASPH Cooperative Agreement (S1107-19/19).

Corresponding Author: Margaret L. Phillips, Associate Professor, University of Oklahoma Health Sciences Center, College of Public Health, Department of Occupational and Environmental Health, 801 Northeast 13th Street, Oklahoma City, OK 73104. E-mail: [e-mail].

REFERENCES

Buchholz, U., Run, G., Kool, J.L., Fielding, J., & Mascola, L. (2002). A risk-based restaurant inspection system in Los Angeles County. Journal of Food Protection, 65(2), 367-372.

Campbell, M.E., Gardner, C.E., Dwyer, J.J,. Isaacs, S.M., Krueger, P.D., & Ying, J.Y. (1998). Effectiveness of public health interventions in food safety: A systematic review. Canadian Journal of Public Health, 89(3), 197-202.

Elledge, B., Basara, H., Boatright, D., Lynch, R., & Phillips, M. (2002). Analysis of food safety policy implementation between urban and rural establishments [Abstract #49872]. Paper presented at the 130th Annual Meeting of the American Public Health Association, Philadelphia, PA.

Fielding, J.E., Aguirre, A., & Palaiologos, E. (2001). Effectiveness of altered incentives in a food safety inspection program. Preventive Medicine, 32(3), 239-244.

Irwin, K., Ballard, J., Grendon, J., & Kobayashi, J. (1989). Results of routine restaurant inspections can predict outbreaks of foodborne illness: The Seattle-King County experience. American Journal of Public Health, 79(5), 586-590.

Mullen, L.A., Cowden, J.M., Cowden, D., & Wong, R. (2002). An evaluation of the risk assessment method used by environmental health officers when inspecting food businesses. International Journal of Environmental Health Research, 12(3), 255-260.

Food Service Establishments, Okla. Admin. Code [section] 310-256 (2004).

U.S. Public Health Service, Food and Drug Administration. (2001). Food Code (NTIS No. PB 2002-100819). Washington, DC: U.S. Public Health Service.

Margaret L. Phillips, Ph.D

Brenda L. Elledge, Dr.P.H.

Heather G. Basara, M.S.

Robert A. Lynch, Ph.D.

Daniel T. Boatright, Ph.D.

TABLE 1 Critical Violation Rates and Contribution of Recurrent
Violations in Oklahoma Food Service Establishments, 1996-2000

Critical                                    Violation Rate (b)
Item (a)  Description                       OCCHD (d)  OSDH (e)

 #1       Approved food source              0.007      0.010
 #3       Holding temperature correct       0.087      0.151***
 #4       Enough facilities for hot- and    0.010      0.019**
            cold-holding
 #7       Cross-contamination prevented     0.039      0.033
#11       Personnel: certified food safety  0.090      0.005***
            officer present, infected
            personnel restricted
#12       Hygienic practices (handwashing,  0.045      0.077***
            etc.)
#20       Sanitization of equipment,        0.052      0.042
            utensils
#27       Approved water source             0.020      0.019
#28       Approved sewage disposal          0.009      0.010
#30       No cross-connections, backflow    0.017      0.011
#31       Hygiene facilities: toilets,      0.039      0.054*
            sinks adequate
#35       Pests; outer openings protected   0.065      0.096***
#41       Toxics properly stored, labeled   0.039      0.047
          All critical violations           0.519      0.573***

Critical                                    Repeat Violations (c)
Item (a)  Description                       OCCHD (d)  OSDH (e)

 #1       Approved food source               8.3%      13.0%
 #3       Holding temperature correct       55.9%      70.5%
 #4       Enough facilities for hot- and    35.3%      37.2%
            cold-holding
 #7       Cross-contamination prevented     49.3%      39.5%
#11       Personnel: certified food safety  52.9%       8.3%
            officer present, infected
            personnel restricted
#12       Hygienic practices (handwashing,  37.2%      53.7%
            etc.)
#20       Sanitization of equipment,        48.4%      44.3%
            utensils
#27       Approved water source             25.7%      32.6%
#28       Approved sewage disposal          18.8%      26.1%
#30       No cross-connections, backflow    41.4%      28.0%
#31       Hygiene facilities: toilets,      21.7%      42.7%
            sinks adequate
#35       Pests; outer openings protected   50.4%      56.1%
#41       Toxics properly stored, labeled   35.3%      44.9%
          All critical violations           44.4%      51.9%

(a) Item number from inspection report.
(b) Total number of violations divided by total number of inspections.
(c) Percentage of violations that were repeat violations.
(d) OCCHD = Oklahoma City-County Health Department.
(e) OSDH = Oklahoma State Department of Health.
*p < .05.
**p < .01.
***p < .001.

TABLE 2 Violation Rates and Inspection Frequency, by Risk Category

                               OCCHD (a)              OSDH (b)
                         Medium Risk  High Risk  Medium Risk  High Risk
Critical Item            (n = 92)     (n = 48)   (n = 141)    (n = 37)

Approved food source     0.007        0.006      0.010        0.011
Holding temperature      0.045        0.139***   0.138        0.186**
  correct
Enough facilities for    0.004        0.017**    0.017        0.023
  hot- and cold-holding
Cross-contamination      0.024        0.059***   0.021        0.065***
  prevented
Personnel: certified     0.105        0.071**    0.003        0.011**
  food safety officer
  present, infected
  personnel restricted
Hygienic practices       0.032        0.060**    0.067        0.106***
  (handwashing, etc.)
Sanitization of          0.027        0.083***   0.036        0.059**
  equipment, utensils
Approved water source    0.020        0.021      0.018        0.021
Approved sewage          0.004        0.015**    0.006        0.021***
  disposal
No cross-connections,    0.012        0.022      0.010        0.015
  backflow
Hygiene facilities:      0.036        0.044      0.055        0.051
  toilets, sinks
  adequate
Pests; outer openings    0.056        0.076*     0.088        0.117*
  protected
Toxics properly stored   0.025        0.056***   0.038        0.070***
  labeled
All critical violations  0.398        0.669***   0.507        0.757***
Average number of        2.6          4.1        2.7          3.7
  inspections per year

(a) OCCHD = Oklahoma City-County Health Department.
(b) OSDH = Oklahoma State Department of Health.
*p < .05.
**p < .01.
***p < .001.