San Diego County team Excellence Performance Measurement System.

Introduction/Background

The Food and Housing Division (FHD) of the San Diego County Department of Environmental Health has received the 2005 Samuel J. Crumbine Consumer Protection Award for Excellence in Food Protection. Keys to the success of the San Diego County Food Safety Program are established community partnerships and the diversity of the stakeholders who participated in the design and implementation of the program, which is focused on risk-based intervention strategies. The team approach to intervention strategies contributed to decreasing trends in the occurrence of major risk factor violations in food facilities (e.g., food from unsafe sources, improper holding temperatures, inadequate cooking, contaminated equipment, and poor personal hygiene). It also helped improve food employee behaviors and food preparation practices (as outlined in the Healthy People 2010 objective), and enhance foodborne-illness surveillance methods for food safety risks in the community.

The San Diego County Food Safety Program has rigorously implemented an inspection methodology focused on food safety risk factors identified by the Centers for Disease Control and Prevention (CDC) as the most prevalent contributing factors in foodborne illness or injury. The concepts of our risk-based food safety program are embedded in our model TEAM Excellence Performance Measurement System. We have a framework of assessing risk, communicating risk, managing risk, and verifying quality of service as essential tools in establishing effective intervention strategies. From these concepts we developed and implemented our most successful intervention strategy: well-trained retail food industry members and registered environmental health specialists who are knowledgeable in innovative intervention and risk-based inspection and education processes. This approach has had a direct, measurable impact by significantly reducing the occurrence of major food safety risk factor violations in food facilities.

San Diego County is located at the southwest corner of California, bordering Mexico, and has a current estimated population of three million people. It is an important tourism destination, with nearly 15 million overnight guests each year, and is a favored retirement center for seniors from throughout the United States.

FHD employs 63 professional and support staff. It is also a full-cost-recovery program funded by food facility permit fees. Operating at full-cost recovery ensures the stability of program staffing and resources. The operational budget for the food safety program is over $6 million.

Program Planning

Our primary program-planning effort targeted the reduction of foodborne illness in San Diego County. In addition, because the county's chief administrative officer had established a long-range general management system (GMS) plan and broad strategic initiatives in 2001, we were also challenged to ensure that our programs were aligned with both of these management documents. Two of the county's strategic initiatives--"Kids" and "Ensuring Safe and Livable Communities"--were identified as areas in which FHD could provide positive impacts.

Traditionally, environmental public health programs have been viewed as difficult to evaluate because they are preventive in nature. The conventional wisdom is that you can't measure what you have prevented. In response, environmental public health programs have traditionally relied on counting "things" rather than quantifying outcomes. "Things" have included the number of inspections conducted or the number of enforcement actions taken. This approach cannot quantify "the difference we are making," nor can it monitor ongoing, new, or evolving environmental public health issues in the community. Our challenge was to develop performance measures that identify "how we are doing," answer the question "what's going on here?", and assess whether we are working effectively and on the right things. These performance measures are also intended to demonstrate results that establish program credibility in the community.

To meet the performance measurement challenge, we set food safety program goals of achieving positive public health outcomes, measuring performance, and continuously improving quality of service. In 2001, we began to look globally, nationally, and locally to identify key concerns and initiatives. We found that the Food and Drug Administration (FDA) National Retail Food Program Standards provided guidance on necessary food safety program components and that Healthy People 2010 provided national goals to which we can contribute. We also started to actively seek input from stakeholders.

Our efforts culminated in the establishment of the long-range TEAM Excellence Performance Measurement System Plan. This plan includes strategies for assessing risk, communicating risk, managing risk, and verifying the quality of service. The long-range plan also established the following goals and objectives for FHD:

* Reduce the number of foodborne illnesses caused by key pathogens in San Diego County. Key indicators include reported Campylobacter infections and Salmonella infections and foodborne-illness outbreaks associated with retail food facilities (goal is to achieve Healthy People 2010 objectives of 12.1 cases per 100,000 for Campylobacter and 12.1 cases per 100,000 for Salmonella species).

* Reduce food safety risk factor violations in retail food facilities by 10 percent per year over a five-year period.

* Improve food employee behaviors and food preparation practices (Healthy People 2010 objective).

* Decrease plan review cycle time for new construction and remodeling plans to 10 working days or less.

* Improve customer service.

Our TEAM Excellence Performance Measurement System also helps us answer the difficult questions regarding achievement of goals and helps us obtain long-term results. The "team" in this system means partnerships with epidemiologists, public health nurses, educators from schools of public health, consumer advocates, public schools, food handler trainers, the Farm and Home Advisor Department, and the regulated food industry. Partnerships also contribute an element of commitment to or ownership in food safety initiatives.

Assessing Risks

An important element of the TEAM Excellence Performance Measurement System is the assessment of risks. Assessing risk is an ongoing process that is essential in identifying changing trends, new problems, relationships between lack of knowledge and violations found, and the effectiveness of implemented interventions. Trends in the occurrence of major food safety risk factor violations can also identify training or other intervention strategies needed to reduce risks.

To help assess risk, a digital dashboard report is assembled and reviewed each month. The dashboard report visually displays various food safety data elements and monitors the ongoing occurrence of major risk factor violations at regulated facilities. The report also enables us to easily monitor impacts from significant changes to our program. For example, in fiscal year 2003-2004, we were able to see results from distribution of temperature control logs and calibration procedures to each food establishment during the routine inspection process in the third quarter and implementation of a new risk-based inspection process and report in the fourth quarter (Figure 1).

In fiscal year 2004-2005, we were able to see the results from the second cycle of inspections that used the risk-based inspection process and report and from the use of risk control plans as an assertive intervention strategy for food facilities with repeated major risk factor violations in the second quarter (Figure 1). Another risk assessment activity is the historical review of baseline data for risk factor violations and foodborne-illness outbreaks found at food establishments. In addition, food safety risk factor violations and food handler knowledge of risk factor violations have been compared. Our findings from the historical review indicated that, as in the FDA-initiated baseline survey of risk factors, the most frequently occurring risk factor violations were consistent from year to year. These include improper cold-holding temperatures, poor personal hygiene, and improper cleaning and sanitization of food contact surfaces. The assessments also revealed the need to improve practices in wholesale food warehouses that distribute food to retail food facilities. On the basis of this assessment information, interventions were developed, implemented, and evaluated. The interventions include use of risk control plans, a new risk-based inspection process and report, a wholesale food warehouse inspection and education program, an operator's guidebook in multiple languages, guidance materials for obtaining food from approved food sources, use of wiping cloths, employee health and hygiene, temperature control logs and calibration procedures, and outreach workshops. Our food handler knowledge survey revealed a lack of knowledge about two risk factors--proper cooking and hot-holding temperatures. We also found that corrective actions carried out by a number of food facility operators tended to be short term. That is, in a number of cases, the same violations were found again at subsequent inspections. This rebound phenomenon contributed to the-place-that-is-always-bad syndrome. We realized that we needed to seek interventions that changed behavior, rather than simply "satisfied the inspector," and that our enforcement and compliance process had to change.

Our assessment of foodborne-illness out-breaks revealed that a significant factor contributing to outbreaks was unsafe food sources, often exacerbated by violations such as improper holding temperature, improper cooking temperature, and cross-contamination. Employee health issues were identified as another leading foodborne-outbreak factor.

Communicating Risks

A sound knowledge base is a recognized first step in effective risk communication. The incorporation of strong community partnerships is also an important component in an effective risk communication system.

For over 20 years, San Diego County has had a mandatory food handler training requirement. Every person who handles food in a retail food facility must be educated and pass an examination demonstrating knowledge of microorganisms, foodborne diseases and their transmission, food protection, temperature control, washing and sanitization of food contact surfaces, and health and hygiene of food handlers. Over 1,200 new food handlers are trained and pass a food handler examination every month.

In 2001, San Diego County established a Food Safety Advisory Council (FSAC) as a forum for obtaining community input for long-term planning purposes. FSAC comprises the retail food industry, San Diego County food handler training schools, the San Diego State University Graduate School of Public Health, a public school nutrition director, the Farm and Home Advisor, epidemiologists, public health nurses, FHD, and interested consumers. Its focus is on information sharing, input into program planning, and education of the community.

The FSAC partnering concept was instrumental in the development of FHD's long-range plan in 2001-2002, the retail food facility operator's guidebook in 2003, and the new risk-based inspection report in 2004. In addition, nearly 2,000 people were trained in our risk factor and intervention-based workshops held throughout the county in 2004. These four accomplishments have had significant impacts in reducing the occurrence of food safety risk factors in retail food facilities by increasing knowledge and improving food employee behaviors and food preparation practices.

San Diego County is one of only a few counties in California that have a public-awareness food facility grading system. It was the first county in California to implement the restaurant grading system in 1947. Establishments with open food preparation receive a grade card, while those that carry and sell only prepackaged food items are not graded. FHD also regularly conducts outreach to the various San Diego communities using print, radio, and television media. Annual media events include a Fourth of July barbecue event to assist the public in having safe summer barbecues, a holiday food safety press release, and Food Safety Education Month targeted to kids in the community. Special issues resulting in media coverage since 2001 include foodborne illnesses linked to the consumption of illegal queso fresco (fresh cheese), consumption of raw Gulf Coast oysters, and a foodborne-illness outbreak associated with prewashed lettuce.

In addition, our Web site (www.sdcdeh.org) is one of the main information portals used to communicate with both our internal and external communities. Web traffic has steadily increased since 2001. At present, the site receives more than 4,000 hits per month, and users download more than 9,000 documents per month. We use commercially available off-the-shelf software to accurately track and report statistical information about Web site visitors. This information is used to customize and improve the Web site content and delivery. Future Web site enhancements will include making inspection results available upon request and using surveys to collect data about the public's knowledge, attitudes, and beliefs regarding environmental health topics. These enhancements are part of the planning process for further program development.

Managing Risks

Another critical program feature of the TEAM Excellence Performance Measurement System is the achievement of a balance between program output (inspections conducted) and outcomes (the reduction in risks). Together, these program components assist in ensuring that cost and revenues are prioritized in critical areas.

To improve the management of risks found during the inspection process, inspection reports were revised so that inspection items that relate directly to risk factors and interventions are conspicuous and prominent on the report. A comprehensive Marking Instructions Guidebook was written to ensure consistency in interpretation and application of the report. Output measures were revised to ensure that overall prioritization and distribution of inspections are consistent and uniform. In addition, a consistent enforcement and compliance process has been implemented to confirm that major violations are immediately corrected or that suitable alternatives are implemented until violations are corrected.

In December 2004, a model Risk Control Plan (RCP) guideline was implemented to assist field staff in providing assistance to operators with recurring major risk factor violations. The RCP leads to long-term results because it assists operators in developing and implementing "active managerial control" procedures uniquely designed for their operation so that they can gain control of their own food safety risks.

Another feature of the San Diego food safety program is the illness complaint process. The FHD foodborne- and waterborne-illness complaint process includes active and passive surveillance, environmental assessments, communication, evaluation and management of the data, and quality assurance/training. A foodborne-illness emergency response team and a close working relationship with our public health partners on the state, federal, and local levels have contributed to effective identification and mitigation of foodborne outbreaks. Disaster preparedness, guidance, and training for wholesale food warehouse distributors in the county have also helped food facilities be better prepared to manage a regional outbreak of foodborne illness. Enhancements in information technology have improved the tracking and identification of risk in the community. In particular, the tracking of major risk factor violations assists us in our assessments, and geographic information systems have been used to map diagnosed Salmonella and Campylobacter illnesses and illegal vendor routes in the county. These methods, along with use of pulsed-field-gel-electrophoresis (PFGE) data, have been particularly beneficial in identifying a major foodborne-illness outbreak associated with the consumption of raw-milk queso fresco (fresh cheese). Increased use of PFGE is changing the way foodborne-illness surveillance has been conducted because it reaches beyond the traditional outbreak surveillance associated with a single food establishment or catered event.

The Inspection Report

Numerous studies have shown that the use of positive reinforcement promotes long-term positive changes in human behavior. For this reason, the inspection form is structured to document when an establishment is in compliance with items related to the risk factors and interventions. Operators can see items that are in compliance, out of compliance, not observed, and not applicable. The focus of the inspection is now to determine "compliance" rather than to merely identify violations. This approach is a major departure from traditional inspection methods and ensures that a consistent risk-based inspection process is followed during each inspection. When major violations are found, however, they require immediate corrective action or implementation of a suitable alternative until the violation is corrected. When a major violation cannot be immediately corrected or a suitable alternative found, the food facility may be subject to closure in the affected areas until the violation is corrected. Inspection items that relate directly to public health interventions are also prominent on the new inspection form since incorporating these procedures into the food safety program of a food establishment will help keep food safe and protect the consumer's health.

Implementing the New Inspection Report

In May 2004, the old inspection reports were replaced by the new risk-factor-and-intervention-based inspection reports. FHD staff conducted consultative inspections the first time they used the new inspection report in a facility. This procedure provided them an opportunity to educate the food facility operator about the new inspection report if the operator had not attended one of the many community-based training and outreach workshops. FHD has recently replaced the original risk-factor-and-intervention inspection report with a version that can be scanned by computer. This scanning system collects more data from the inspection reports and provides more comprehensive data, including information on trends. This data management phase will make an electronic copy of each inspection report that is scanned and will eventually allow FHD to discontinue the hard-copy storage of inspection reports.

Verifying the Quality of Service

An Excellence in Service program is another important component of the food safety program and is designed to assess, verify, and improve, as necessary, the inspection program. It includes an evaluation process, annual standardization, and customer service surveys. As a result of the Excellence in Service program, all field staff, supervisors, and the chief of FHD have been trained, and their practices with respect to the risk-factor-and-intervention-based inspection process have been standardized. The program has also helped identify training needs for field staff and has led to customer satisfaction scores of over 97 percent.

Finally, annual reviews of our performance measures and environmental public health indicators assist us in continuously improving our processes, identifying new or emerging issues, and effectively managing our environmental public health resources.

Conclusion

FHD is similar to many successful organizations that are geared toward achieving positive outcomes, measuring performance, and maintaining and improving the quality of service. FHD, however, has additional accomplishments as a results of partnering with the community to develop and implement the model TEAM Excellence Performance Measures System for the San Diego County food safety program. This system, built on the concepts of assessing risk, communicating risk, managing risk, and verifying quality of service, has resulted in identification and reduction of food safety risks in the community. Its main focus is on outcomes and long-term results. The investment of a tremendous amount of time and commitment by FHD staff and stakeholders has contributed to success in enhancing the quality of life for three million residents and nearly 15 million overnight guests who visit the county each year. When the regulated industry, schools, academia, consumers, epidemiologists, public health nurses, and regulators are interested in partnering to ensure the public's health and safety, we all win!

Acknowledgements: The Department of Environmental Health wishes to thank all of its Food and Housing Division staff and stakeholders for their outstanding efforts and contributions to ensuring safe and livable communities in San Diego County.

Gary W. Erbeck, M.P.H., R.E.H.S.

Liz Pozzebon, M.S., R.E.H.S.

Editor's Note: The Samuel J. Crumbine Consumer Protection Award for Excellence in Food Protection, named in honor of one of the most renowned U.S. public health sanitarians, is presented each year to a local public health agency by a jury of leading environmental health officials and public health sanitarians. The Crumbine Award is the most prestigious recognition that a public health agency can receive for excellence and improvement in food protection. Agencies that win the Crumbine serve as models for other public health and safety programs across the nation.

Below, readers will find a description of the agency that won the 2005 award--the San Diego County Department of Environmental Health. The description is adapted from information the department submitted on its Crumbine application.

For more information about the Crumbine Award, please contact Lynn Rosseth, director of market development and programs at the Foodservice & Packaging Institute (FPI), at [phone]. For more information about the San Diego County Department of Environmental Health, contact Gary W. Erbeck, director, San Diego County Department of Environmental Health, at [e-mail], or Liz Pozzebon, chief, San Diego County Department of Environmental Health, Food and Housing Division, at [e-mail]. A copy of San Diego County's entire award-winning application is posted on the FPI Web site at www.fpi.org.

FIGURE 1 Major Risk Factors Observed per Quarter

               Temp Not       Temp Not        Improper  Employee
               <41[degrees]F  >140[degrees]F  Cooling   Hygiene

3rd Qtr 03-04  5.31           1.12            0.67      3.99
4th Qtr 03-04  5.89           1.10            0.59      4.00
1st Qtr 04-05  5.57           0.79            0.54      2.69
2nd Qtr 04-05  3.57           0.98            0.48      2.60
3rd Qtr 04-05  2.40           0.76            0.43      2.09
4th Qtr 04-05  2.84           0.80            0.67      1.81

               Heating/          Contact  Cross-
               Cooking   Source  Surface  Contamination

3rd Qtr 03-04  0.54      0.12    6.12     3.74
4th Qtr 03-04  0.28      0.17    5.54     2.18
1st Qtr 04-05  0.26      0.14    2.41     1.17
2nd Qtr 04-05  0.29      0.00    2.88     1.11
3rd Qtr 04-05  0.17      0.16    1.56     0.53
4th Qtr 04-05  0.15      0.17    1.34     0.48

3rd Quarter 03-04:
Temperature Logs distributed to all food facilities with focus on
temperature control for food safety.
4th Quarter 03-04:
New risk-based inspection process and report initiated. Staff and
industry training and consultative inspections conducted.
2nd Quarter 04-05:
Second cycle of inspections conducted with risk-based inspection report.
Use of risk control plans for facilities with repeat major risk factor
violations.

Note: Table made from bar graph.