Scores and grades: A sampling of how college students and food safety professionals interpret restaurant inspection results. (Features).

Introduction

Health departments must contend with complex questions: What is the best way to determine if restaurant food is being prepared safely? How can inspection results be used to protect the public from unsafe restaurant conditions? This study considers whether posting of inspection results as numerical scores or letter grades is a useful tool for health departments.

Numerous controversies must be addressed before a widely accepted policy on this issue can be formulated. There are disagreements about whether a scoring system is better than a pass-fail system, whether a numerical score is preferable to a letter grade ("Using Numbers," 1999), whether a standard national code is advisable (Food and Drug Administration [FDA], 1999; Frable, 1997), and the impact of grading disparity among inspectors. Food safety professionals also have considered whether the number of times a restaurant is inspected correlates with its ratings (Allwood, Lee, & Borden-Glass, 1999), and how to determine optimal sanctions for code violations (e.g., education versus restaurant closure). Many of these issues affect the potentially adversarial restaurant--local health department relationship (Straus, 1993).

One topic of particular concern to restaurants involves the posting of inspection scores. There is variation by jurisdiction in whether posting of inspection results is required and in the manner in which results must be displayed (Boehnke & Graham, 2000). As of 1998, some unincorporated areas of Los Angeles County require posting of grades in the windows of restaurants (Dulen, 1998). Other areas (e.g., Farmington Valley, Connecticut) plan to require restaurants to post inspection ratings in the future (Silverman, 1999). As of January 2001, Toronto uses a color-coded system; a green sign in a restaurant window denotes a passing inspection grade, a yellow sign represents significant infractions resulting in a conditional pass, and a red sign means that a restaurant has had major violations and has been closed. There is controversy about whether a yellow sign will scare off diners even though yellow signifies that a restaurant has passed inspection--albeit conditionally (Honywill, 2001; Solyom, 2001). A central concern about inspection signs is how well a scoring system truly conveys risk. A score of 95 could reflect several critical temperature violations while another restaurant might receive an 88 for multiple, but noncritical infractions (Koeune, 2000). Furthermore, there is no evidence that a particular food safety violation prompts a consistent reaction by health inspectors (Rajapaksa & Dundes, 2001).

In addition to disagreement about how a score should be derived, there are still questions about the tremendous variation in how health departments across the United States make the results of restaurant inspections readily available to the general public. Although a growing number of jurisdictions are making such information accessible online (see, for example, <www.restaurantsafety.com/inspections.html>), it still is not known whether posting (or publicizing, in the case of Web sites) a letter grade or score effectively informs consumers about restaurant food safety or merely jeopardizes the relationship between health departments and restaurateurs without conferring any benefit to either party While it could be argued that posting serves a more important purpose--to provide an incentive for a restaurant to comply with codes--this hypothesis, too, is untested (except for anecdotal accounts) (Anderson, 2000; Hahn, 2000).

To assess the posting of restaurant inspection results, it is necessary first to determine whether the public correctly interprets a posted letter or score (Wiant, 1999). In addition, it should be ascertained whether the public wants to know detailed information or if individuals simply prefer to have a restaurant be deemed safe--with doors open if it is safe or closed if there are infractions severe enough to pose a risk to patrons.

Methods

To determine how college students and food safety professionals view posting of restaurant inspection reports by health departments, a survey was devised in which respondents were asked the meaning of signs with either a letter grade or a score posted. The survey showed small, computerized renditions of sample signs posted by a "Department of Health Services" (as is done in Los Angeles County). The signs had a printed "86%" or a printed "C" on them and also included the following clues to context: date, signature, and state seal. During the piloting phase of the study, when the sample sign had been posted by the "Department of Sanitation," many respondents had focused on issues of restaurant cleanliness to the exclusion of other food safety concerns. As a result, the authors changed the sponsor of the sign to "Department of Health Services."

After respondents were asked to interpret posted signs, they were asked if they wanted to be in a position of having to decipher such information. Or did they think that instead of relying on the public's ability to respond appropriately to such data, health departments should simply close a restaurant when an infraction places the public at risk? Finally, respondents were asked about the degree of regulatory involvement they preferred, and background attitudinal and demographic information was solicited.

In the first week of May 2000, the survey was administered to 374 (out of 1,500) undergraduate college students at a small, private, liberal arts college, Western Maryland College, located 40 minutes west of Baltimore. Six students distributed and then collected the anonymous surveys from students in classes and dorms. The authors selected this population for the following reasons:

1. The population tends to be accessible and cooperative; the response rate was nearly 100 percent.

2. Members of the population eat out regularly

3. Most members of the population have worked in a restaurant and so may be more likely to have some sense of how to interpret a posted grade or score.

4. The population lacked exposure to the practice of posting results since in Maryland the health department does not post scores or grades. Anyone who wishes to know inspection results in Maryland must go to the health department in person to peruse files (a rare occurrence) (L. Leitch, Carroll County Health Department, personal communication, May 12, 2000).

To strengthen the generalizability of the results, the survey was distributed to a group of food safety professionals attending the Central Atlantic States Association of Food and Drug Officials (CASA) Annual Conference in New York, May 22-25, 2000. Nearly half of the surveys distributed (150) were returned (n = 72, a 48 percent response rate).

Except for the first two survey items, all questions were closed-ended to offset the possibility of suggestivity, prejudice, or bias. Only the limited information that is available to the average restaurant patron who might see a posted sign was offered. The same questions were asked about two signs, one with a large "C" on it, the other reading "86%."

Results

Interpretation of Signs

After viewing a sign that read "86%," followed by "Dept. of Health Services," a date, a signature, and a state seal, respondents were asked the following:

A. What does this tell you about the restaurant?

B. Explain whether such a sign would affect your decision to eat at the restaurant.

Among students, 21 percent said the sign told them nothing or was unclear; 45 percent simply reiterated the score or stated that the number was the result of a health inspection; 10 percent said it meant the restaurant was clean or well kept; eight percent said it meant the restaurant was not clean; and 16 percent gave a miscellany of responses (e.g., "could be better," "caught on a bad day," "trying to get people to eat there," "like all restaurants--not perfect," "restaurant cares about customer') (Table 1).

Among the CASA sample, 14 percent said the sign was unclear or did not tell them anything; 40 percent indicated that they thought 86 percent was a good score; and 20 percent thought the score was average and qualified as passing an inspection. Another 20 percent thought that "86%" indicated the establishment was problematic or out of compliance. Six percent made comments falling into a miscellaneous category such as "the government has failed to educate the restaurant industry," "the restaurant must be inspected regularly," or "I do not agree with posting scores--too much pressure" (Table 1).

When asked whether the sign would affect their decision to eat at the restaurant, 37 percent of students said they would eat there, 38 percent said they would not eat there, and 25 percent said they would have second thoughts about eating there (Table 2). Of the CASA respondents, 54 percent said they would eat at the restaurant, 27 percent said they would not, and 19 percent said they would be hesitant to eat there (Table 2).

When asked about the sign that read "C" followed by "Dept. of Health Services," a date, a signature, and a state seal, 27 percent of students said it was unclear; 27 percent reiterated (without explanation) that the C represented the results; 13 percent said it meant the restaurant was clean; four percent said it meant the restaurant was not clean; 11 percent said it meant the restaurant was not good or had problems; and 18 percent gave miscellaneous explanations (e.g., "restaurant has been reprimanded by the health department," "it did not do a good job on a health test," "worse than the first one," "food quality is a C") (Table 3). Among CASA members, 40 percent thought C was average or indicated the inspection had been passed; 32 percent thought it meant that the restaurant had problems; 21 percent said they did not know what a C meant; and seven percent had miscellaneous responses (e.g., "they don't take regulations seriously," "the general public sees this as indicative of food quality, not food safety," or a combination of factors: "the restaurant is mediocre in its sanitation and its concern for food safety") (Table 3).

Most student respondents said they would not eat at a restaurant with a C grade posted. Only 22 percent said they would eat there, whereas 58 percent said they would not eat there and 20 percent said they would have second thoughts about eating there (Table 2). Of respondents who said they would eat in a restaurant with a posted inspection report of 86 percent, only about half of them (49 percent) would eat at a restaurant that had a C grade posted. Interestingly, some who would be comfortable eating at a restaurant with a grade would either not want to eat at a restaurant with a score of "86%" posted (14 percent) or would have second thoughts about doing so (four percent). This result indicates that even for grade-conscious college students, a C grade may be perceived as safer than "86%." (One respondent, for example, wrote that "C" stood for "Compliant"). Among CASA members, 65 percent would eat at a C grade restaurant, 21 percent would not eat there, and 14 percent would have second thoughts about such a d ecision (Table 2).

When asked about their concern about food safety in restaurants, 42 percent of students were very concerned, 47 percent were somewhat concerned, and 11 percent were not very concerned. Among CASA members, 76 percent were very concerned, 22 percent were somewhat concerned, and two percent were not very concerned (Table 4).

Respondents also selected from among four options that characterized the amount of government involvement they preferred in keeping people from becoming ill from eating in a restaurant. Among students, two percent said they preferred no government involvement because "market forces and competition will have a much bigger impact than government in keeping restaurants safe and clean." No CASA members selected this option. Fourteen percent of students and 11 percent of CASA members said there should be a little government involvement: "Since customers are not allowed 'behind the scenes,' an inspector should periodically check for gross negligence on behalf of all customers. Most correction is brought about by simple discussion." Most students (57 percent) and many CASA members (47 percent) preferred moderate government involvement in which "inspectors should do in-depth investigations several times per year along with microbiological sampling and testing, despite the slight increased cost to taxpayers." About ha lf as many students (27 percent) wanted strong government involvement in which "every consideration of possible hazards should be regulated, including for those with allergies and special diet considerations, despite a greater cost to patrons" and thought that "higher-risk foods such as raw eggs and raw oysters should be banned." Among CASA members, 42 percent wanted strong government involvement (Table 5).

Finally, respondents were asked who should make the food safety determination when people eat out: the health department ("Either allow a restaurant to be open [indicating it has a safe kitchen] or close it [for unsafe conditions]") or the public ("Post health department inspection results in the form of a score or grade in the window of a restaurant"). Most students (71 percent) and most CASA members (81 percent) wanted the health department to make a determination rather than the public to make a judgment based on posted information (Table 6).

Demographics 

All student respondents were undergraduates at Western Maryland College. Most (60 percent) were 19 to 21 years of age. Forty-six percent were male, and 54 percent were female. Nearly 80 percent ate at a restaurant one to three times per week; 57 percent had worked at a restaurant; most thought that at some point they had become ill from eating restaurant food (51 percent once or twice, eight percent three or more times); and 41 percent believed they had never gotten sick from restaurant food (Table 7).

CASA respondents were 60 percent male and an average of 48 years of age (ranging from 24 to 66), and their official duties included food safety work (93 percent). Most (84 percent) worked for a government agency; 13 percent worked for private industry, and three percent designated their employers as "other." Fourteen percent were from areas that post scores in food establishments, 85 percent were from areas that did not do so, and one percent did not know whether their areas followed such a practice (Table 8).

Discussion

According to data collected in this study, college students have a very limited understanding of what posted restaurant inspection results mean. Virtually no respondents linked the score or grade to particular infractions or even mentioned any general type or category of violations that might occur. The only specific interpretation concerned cleanliness. Even in those cases, most respondents had only the vaguest idea of what cleanliness might mean, or whether cleanliness was a positive or negative attribute in a given score.

Even among CASA professionals, many responses were vague, raising questions about their understanding of the signs. The main difference between the CASA and student responses was that students were more likely to think that the signs were based on cleanliness while the CASA members were more likely to interpret the signs as indicating a passed inspection. An equal number from each group thought the sign signaled problems or noncompliance. In the case of the posted grade of C, 32 percent of the CASA sample saw the restaurant with a C grade sign as having low compliance, while another 40 percent indicated a seemingly opposite reaction-that a C grade revealed average inspection results (a nonspecific response).

The question that attempted to gauge respondents' willingness to eat at a restaurant with the graded and scored signs under study revealed their potential to turn away diners. Interestingly, only 37 percent of college students would be comfortable eating at a restaurant with a posted score of 86, indicating that a score only four percentage points away from 90 was considered sufficiently problematic to make the remaining 63 percent likely to decline to patronize the restaurant. In the case of the sign with the C grade, the proportion that would eat at the restaurant dropped to 22 percent. These results indicate that posting scores not only of C quality but also of B quality or 86 percent, might deter customers if they in fact see the signs. It was notable that the respondents were so unaware of the signs' meanings given that most of the respondents (57 percent) had worked in a restaurant.

CASA members were much more comfortable than students about eating at a restaurant with an "86%" sign, with over half (54 percent) being undeterred by a such a sign. The results for this question indicate that there is a lack of consensus among food safety professionals about the ultimate meaning of a sign reading "86%."

Although college students were more comfortable with a sign that read "86%" than with a sign that read "C" (37 percent versus 22 percent), food safety professionals were more likely to eat at a restaurant with a sign displaying a "C" (65 percent versus 54 percent) (Table 2). Nevertheless, about one-third of the professionals might forgo eating at such an establishment.

Students' uncertainty about the signs' meanings may have affected their response as to whether the health department or the public should make the food safety determination for diners. (One possible explanation was that given their own befuddlement, students thought it would be better not to have the public attempt to interpret such signs.) When offered the choice between posted grades and scores (without explanation) and a simple health department determination most respondents want to count on the health department to close a restaurant that poses a health risk.

Like the students, most food safety professionals (81 percent) prefer that a health department, not the public, bear the responsibility of ensuring that a restaurant is sufficiently compliant to have its doors open; they do not expect the public to make a meaningful determination of risk based on a sign posted. This study does not answer the question of how individuals would react if they had access to a more informative sign detailing inspection results or if they had the chance to peruse results on a Web site, It is certainly possible that in such cases respondents might have selected the option that allows the public to make a decision based on posted results.

Conclusion

This survey of undergraduate students and food safety experts indicates that members of the two groups interpret both graded and numerical inspection results in widely varying ways. These data raise serious questions about whether posted reports (with minimal information) are helpful to the public in determining restaurant safety. While these study populations were limited to two distinct voluntary samples and cannot be generalized to the public as a whole, their level of education and familiarity with the restaurant industry should have made them more likely than the general public to understand the meaning of posted results.

Perhaps the respondents' inability to accurately assess scores and grades contributed to the preference of the majority that the health department close down restaurants with unsafe conditions (rather than post inspection results in the form of a score or grade in a restaurant window). According to the authors' data, if health departments require prompt correction of deficiencies or close a restaurant when its conditions threaten diners' health, posting scores or grades (and perhaps even other variants) might be viewed as both unnecessary and of questionable value. Overall, the respondents from this sample did not voice a desire for relevant information to assess a restaurant's degree of compliance; instead, the vast majority want either moderate or considerable government involvement in regulating and ensuring the safety of restaurants. Furthermore, giving individuals limited posted information (that many do not know how to interpret) could lead some to decline to dine at an establishment.

Although one could argue that the system of posting grades or numerical scores pleases restaurants that receive high scores while motivating restaurants that receive lower scores, the system is based on the questionable assumption that the public is being given meaningful, intelligible information. This investigation suggests at least two options for a jurisdiction: 1) Post information that the public understands (achieved through an educational campaign) or 2) do not use signs and assume that if the doors of a restaurant are open, it means that the restaurant is in compliance with food safety standards.

Corresponding Author: Lauren Dundes, Assistant Professor, Department of Sociology, Western Maryland College, 2 College Hill, Westminster, MD 21157-4390. E-mail: <[e-mail]>.

TABLE 1

Responses to a Sign That Read "86" (*)

                                     Students  CASA

Nothing, Unclear                       21%     14%
86% of standards, health inspection    45%     40%
 results
Restaurant is clean or well kept       10%      --
Students: restaurant is not clean;      8%     20%
 CASA: problematic, noncompliant
Average: passed inspection              --     20%
Miscellaneous                          16%      6%

(*)Respondents were asked the following question: "You see a sign posted
in the window of a restaurant stating: 86%, Dept. of Health Services,
date, signature, state seal; what does this tell you about the
restaurant?"
TABLE 2

Impact of Signs on the Decision to Eat at a Restaurant (*)

                                   "86%" Sign                "C" Sign
                            Students      CASA      Students    CASA

Would eat there               37%         54%         22%       65%
Would not eat there           38%         27%         58%       21%
Would have second thoughts    25%         19%         20%       14%

(*)Respondents were asked the following question: "Explain whether the
sign would affect your decision to eat at the restaurant."
TABLE 3

Responses to a Sign That Read "C" (*)

                                    Students  CASA

Nothing, unclear                      27%     21%
These are the results/the score;      27%     40%
 average
Restaurant is clean                   14%      --
Restaurant is not clean                4%      --
Students: restaurant has problems;    11%     32%
 CASA: problems or low compliance
Miscellaneous                         17%      7%

(*)Respondents were asked the following question: "If you saw a sign
posted in the window of a restaurant that stated C, Dept. of Health
Services, date, signature, state seal, what does this tell you about the
restaurant?"
TABLE 4

Level of Concern About Food Safety in Restaurants (*)

                    Students  CASA

Not very concerned    11%      2%
Somewhat concered     47%     22%
Very concerned        42%     76%

(*)Respondents were asked the following question: "In general, how
concerned are you with the safety of food consumed in restaurants?"
TABLE 5

Degree of Government Involvement Preferred (*)

                                    Students  CASA

None--Market forces and competiton     2%      0%
 will have a much bigger impact
 than government in keeping
 restaurants safe and clean.
 Example--McDonald's restaurants
 in countries that have no health
 departments.

A little--Since customers are not     14%     11%
 allowed "behind the scenes," an
 inspector should periodically
 check for gross negligence on
 behalf of all customers. Most
 correction is brought about by
 simple discussion.

Moderate--Inspectors should do        57%     47%
 in-depth investigations several
 times per year along with
 microbiological sampling and
 testing, despite the slight
 increased cost to taxpayers.

Strong--Every consideration of        27%     42%
 possible hazards should be
 regulated including for those
 with allergies and special diet
 considerations, despite a greater
 cost to patrons. Higher-risk
 foods such as raw eggs and raw
 oysters should be banned.

(*)Respondents were asked the following question: "How much involvement
do you want the government to have to keep you from getting ill from
eating in a restaurant?"
TABLE 6

Opinions About Who Should Determine That a Restaurant Is Safe
(*)

                                    Students  CASA

Health department--Either allow a     71%      81%
 restaurant to be open
 (indicating a safe kitchen) or
 close it (for unsafe conditions).
The public--Post health deparment     29%      19%
 inspection results in the form of
 a score or grade in the window of
 a restaurant.

(*)Respondents were asked the following question: "Who do you think
should make the food safety determination when you eat out?"
TABLE 7

Demographic Information for College Student Respondents

Demographic Category              Percentage

Age
  18 or 19                            31
  20                                  28
  21 or 21                            31
  23+                                 10

Sex
  Male                                46
  Female                              54

Time per week respondent eats at
 restaurants, fast-food or
 otherwise
  0                                   1
  1                                   30
  2                                   30
  3                                   20
  4                                   9
  5                                   10

Respondent has worked in a
restaurant
  Yes                                 57
  No                                  43

Number of times respondent has
gotten sick from eating bad food
he or she believes was from a
restaurant
  Never                               41
  Once or twice                       51
  3+ times                            8
TABLE 8

Demographic Information for CASA Respondents

Demographic Category                 Percentage

Age (average = 48 years)
  32-61 years                            96

Sex
  Male                                   60
  Female                                 40

Official duties include food safety
 work
  Yes                                    93
  No                                      7

Employer
  Government agency                      84
  Private industry                       13
  Other                                   3

REFERENCES

Allwood, PB., Lee, PB., & Borden-Glass, P. (1999). The vital role of restaurant health inspections. Journal of Environmental Health, 61(9), 25-28

Anderson, K. (2000). Should restaurant inspection reports be published? Journal of Environmental Health, 62(8), 29.

Boehnke, R.H., & Graham, C. (2000). International survey on public posting of restaurant inspection reports, and/or grade card posting schemes based upon health inspections (Report prepared for Region of Ottawa--Carleton Health Department). Etobicoke, Ontario, Canada: RHB Consulting and Associates.

Dulen, J. (1998). Making the grade. Restaurants and Institutions, 108(6), 92.

Food and Drug Administration. (August 1999). HAACP: A state-of-the-art approach to food safety. FDA Backgrounder. <http://vm.-cfsan.fda.gov/~lrd/bghaccp.html> (22, May, 2000).

Frable, F, Jr. (1997). Industry needs central resource for code, regulation info. Nation's Restaurant News, 31(20), 44-45.

Hahn, M.I. (2000). Should restaurant inspection reports be published? [Letter to the editor]. Journal of Environmental Health, 62(8), 31.

Honywill, B. (2001, January 13). Toronto eateries signing off; Restaurants say food safety signs killing biz. Toronto Sun, p. 2.

Koeune, T.J. (2000). Should restaurant inspection reports be published? [Letter to the editor]. Journal of Environmental Health, 62(8), 27-28.

Rajapaksa, S., & Dundes, L. (2001). Restaurant food safety: Perceptions and validity of assessing violations. The Journal of the Association of Food and Drug Officials, 65(1), 14-22.

Silverman, F. (1999, December 21). Cleanliness counts: Should restaurant inspection ratings be posted? The Hartford Courant, p. El.

Solyom, C. (2001, January 26). Green means eat in Toronto: Coloured restaurant warnings not needed here: Owners. Montreal Gazette, p. A6.

Straus, K. (1993). In-house safety inspections. Restaurants & Institutions, 103(24), 73-76.

Using numbers rather than letter grade: State restaurant rating experiment may expand. (1999, August 19). Wilmington StarNews, p. 5B.

Wiant, C.J. (1999). Scores, grades, and communicating about food safety. Journal of Environmental Health, 61(9), 37-38.

RELATED ARTICLE: Scores and Grades: A Sampling of How College Students and Food Safety Professionals Interpret restaurant Inspection Results

* Controversy surrounds the use of posted restaurant inspection scores and grades.

* To determine how scores and grades are perceived, the authors surveyed a sample of college students and food safety professionals.

* They asked participants the following questions about a posted inspection score of 86 and a letter grade of C:

--- What does this tell you about the restaurant?

--- Explain whether such a sign would affect your decision to eat at the restaurant.

* Students had the following responses to a score of 86:

--- 21 percent said such a sign was unclear,

--- 45 percent simply reiterated information provided in the survey questions,

--- 10 percent said the score indicated a restaurant was clean or well kept,

--- eight percent said the score indicated a restaurant was not clean, and

--- 16 percent gave miscellaneous responses.

* Food safety professionals had the following responses to a score of 86:

--- 14 percent said such a sign was unclear,

--- 40 percent thought 86 was a good score,

--- 20 percent thought 86 was average and qualified as passing an inspection,

--- 20 percent thought the score indicated an establishment was problematic or out of compliance, and

--- six percent gave miscellaneous responses.

* Students had the following responses to a grade of C:

--- 27 percent said such a sign was unclear,

--- 27 percent reiterated information provided in the survey question,

--- 13 percent said the grade meant the restaurant was clean,

--- four percent said the grade meant the restaurant was not clean,

--- 11 percent said the grade meant the restaurant had problems, and

--- 18 percent gave miscellaneous responses.

* Food safety professionals had the following responses to a grade of C:

--- 40 percent thought C was average or indicated the inspection had been passed,

-- 32 percent thought the grade meant the restaurant had problems,

-- 21 percent said they did not know what a C meant, and

-- seven percent gave miscellaneous responses.

* The argument that posting of grades or scores motivates restaurants to improve food safety is based on the questionable assumption that the public is being given meaningful information.

* This investigation suggests two options for health departments:

1. post information that the public understands (through an educational campaign) or

2. do not post scores and allow the public to assume that if the doors of a restaurant are open, it means that the restaurant is in compliance with food safety standards.