Auditing from a citizen's point of view: the citizen's perspective is often overlooked in traditional audit of government programs. Incorporating the citizen's perspective, where relevant, can make audits a more effective tool for change.

Governments provide programs and services. Auditors look at how well those programs and services are delivered. Often overlooked in a traditional audit is the citizen's perspective: how well and how easily do the programs and services meet the needs of the citizens who use them? Incorporating the citizen's perspective, where relevant, can make audits a more effective tool for change.

ACHIEVING OUTCOMES AND PUBLIC TRUST

The task of government is to deliver services to citizens to meet their needs or protect the needs of others. Management sets goals, plans, organizes, and puts these services into operation, hut often has no idea whether those who needed the services received them or if the desired outcomes were achieved. Government-designed programs are required to follow rules, regulations, and laws, and have procedures that ensure fairness. Following these requirements can create services that are complicated or inefficient. As a result, management must find a balance between public sector requirements and delivering services that solve problems.

Initiatives such as performance budgeting, service efforts and accomplishments reporting, citizen surveys, citizen involvement programs, and continuous quality improvement have put more emphasis on understanding the citizen's point of view. Not only do these efforts increase the likelihood that outcomes will be reached, they also improve citizen confidence in government.

The public's level of trust or distrust results from many small experiences. Each time a citizen uses a service, government has the opportunity to make a good or poor impression. Every pothole ignored, hour spent standing in line to renew a license, rude or disinterested response received from a government worker, unanswered telephone call or unreturned message creates a lasting impression. A series of poor experiences leads to the belief that "government doesn't care." Furthermore, it reduces public satisfaction and decreases support for future funding.

Auditing a service from a citizen perspective is a way to address both organizational effectiveness and public distrust. It also provides unique and compelling evidence that can direct organizational change. While quantifying the effect of a problem in terms of dollars or outcomes might illustrate the need to change, it also increases the perception that government is wasteful and affects employee morale. Collecting data and reporting on service access or quality from a citizen perspective creates a climate of understanding and encourages change in a more constructive way.

AUDIT METHODOLOGIES

The scope of a citizen point of view audit varies by level of contribution and subject area as summarized in Exhibit 1. Evidence gathered from the citizen's perspective can augment other analytical methods or contribute the majority of evidence collected. Further, the subject of the audit can focus on the quality of services delivered or on service accessibility To date, auditors have tended to focus on the quality of services delivered rather than evaluating how easy the services are to access. It is important for auditors to remember, however, that if the citizen can't find, understand, or access a service, government will not be able to achieve its goals.

Exhibit 1: Potential Audit Scope

          CONTRIBUTION

SUBJECT   Partial scope-service access    Full scope-service access
          Partial scope-service quality   Full scope-service quality

There are also two methods for obtaining evidence in an audit from the citizen point of view (Exhibit 2). The more widely known and used approach is to collect information through a third-party from citizens, by interviewing, conducting focus groups, or surveying recipients of a government service to determine satisfaction with the services delivered or accessibility. A second method is to collect information directly, essentially putting the auditor in the citizen's shoes. Audit teams gather information by impersonating citizens seeking a government service and collecting data based upon that experience. This is known in the private sector as the "secret shopper" method. Or, the audit team can evaluate the citizen experience by using criteria developed from the citizen's perspective and arrive at a grade, score, or percentage.

Exhibit 2: Potential Methodologies

THIRD PARTY                   DIRECT

* Interview      * Secret shopper
* Focus groups   * Criteria-based grading
* Surveys          (check lists)

AUDITING THE CITIZEN'S EXPERIENCE

The Multnomah County Auditor's Office (Portland, Oregon) has conducted a number of audits that incorporated the citizen's point of view. These audits have been successful in determining how services could be improved so that the citizen or customer experience is more satisfying.

Two groundbreaking audits reviewed the citizen's ability to access services. The first audit, County Services: Help Citizens Find Their Way (Wayfinding Audit), was a comprehensive look at all county services and facilities, while the second audit, Public Records: Eliminate Barriers to Citizen Access (Public Records Audit), audited the citizen's ability to access a specific mandated service--public records. Both audits were conducted exclusively from the citizen's point of view. In the first audit a checklist was used by auditors based on wayfinding criteria. In the second audit, volunteers were assigned public records to locate and acquire.

WAYFINDING AUDIT: ACCESS TO SERVICES

The Wayfinding Audit completed in 1995 was an outgrowth of previous work completed in an audit of library services. While auditing the efficiency of library staffing, the auditor's office discovered a body of work in the field of architecture describing techniques to successfully guide people through buildings. Using these criteria, the audit team surveyed 25 buildings and 35 programs. The audit tested how easy it was to locate the building from the street, and, once in the building, to find the program office.

The findings were persuasive. The report made extensive use of photographs to increase the reader's understanding of the problems that were identified. Auditors discovered that maps were hard to use, directories were confusing or unhelpful, signage was incomplete or nonexistent, and the use of emergency exit identification was inconsistent. In one case, a handicapped entrance led to a stair well with no way beyond for a handicapped person to proceed. Program staff admitted that handicapped persons had to call ahead to have someone meet them in the stair well.

The Wayfinding Audit also looked at telephone access. At that time, there were 59 recorded answering systems in the county. Many were to programs that had very low incoming volume and seemed to have no purpose other than to distance employees from citizen inquiries. Nearly 50 percent had an average of five calls or fewer per hour. This also became a finding in the 2006 audit of access to public records.

PUBLIC RECORD AUDIT: ACCESS TO RECORDS

While the Wayfinding Audit reviewed access to multiple county services, the Public Records Audit focused on one service--the acquisition of public records. Oregon law mandates that anyone, for whatever reason, can access public records with a few exceptions. A traditional audit might have examined the county's level of compliance with the law. Work would have begun with a review of the county's policies and procedures. Pursuing that methodology would most likely have found evidence of noncompliance with the law, but the findings would not have measured citizen frustration.

Multnomah County's Public Records Audit went beyond compliance and looked at the spirit as well as the letter of the law. The objective was twofold: to look at whether the county was prepared to respond to citizen inquiries, and to determine what the citizen's experience was. The Public Records Audit asked three questions:

* How would the public find the place to make an inquiry?

* How would they be treated?

* How responsive would government be?

The audit was designed to maximize the validity of the results. Departing from the usual audit procedure, the audit was unannounced. However, once the work was completed, management was notified so that they would not be surprised when the audit was issued. To maintain anonymity and accurately assess the citizen experience, community volunteers and a summer intern were used to perform the tests rather than audit staff. A cell phone and e-mail accounts were used that could not be traced to the auditor's office. Items that were requested were straightforward and realistic. In effect, the county was afforded the best opportunity to succeed.

Overall, the county's response was lower than required by law. A total of 49 requests were made for information. Requests were made over the phone, on-site, and through e-mail. Using the telephone directory, the primary means available to the citizen, the right location to request

a record was found 57 percent of the time. Volunteers were successful receiving 67 percent of the requests. Further, automated telephone messages created barriers and many times employees did not respond to messages. In one case, 12 days passed without communication. It took up to 18 days and multiple contacts to obtain relatively simple records.

AUGMENTING TRADITIONAL APPROACHES

Evidence designed to measure effectiveness from the citizen's perspective can also be used to add emphasis or augment other more traditional audit methodologies. Some examples used in the Multnomah County Auditor's Office are listed below.

* Personal property tax audit--Businesses were contacted to determine if electronic filing was feasible. The audit found that many taxpayers were far ahead of the county technologically and were able to prepare and store electronic returns.

* Citizen survey--An annual survey conducted by the auditor's office found that only 25 percent of county residents were aware of an information and referral line. Residents reporting that it was difficult to find a service increased seven percentage points over the previous year.

* Citizen involvement audit--Auditors reviewed the county's program for engaging citizen participation and found that avenues for involvement were unclear and the program was poorly organized.

* Housing services audit (conducted jointly with the City of Portland Auditor's Office)--Volunteers posed as homeless clients asking for referral services via the telephone. Usually more than one call was required before volunteers talked with an individual who could offer help. One service provider did not answer calls during business hours. Another used an answering machine to notify callers that funds were used up, and asked them to leave a name and number to be called in the next month. When information was provided, its quality was poor in most cases. If you were homeless with only enough money for one call, you can easily imagine the frustration.

* Adult foster care homes--Auditors developed criteria for measuring the quality of adult foster care homes from the resident's perspective. Surprise visits were conducted and homes were scored based upon the criteria. Auditors found conditions in eight homes that they were required to report to the program immediately.

POSITIVE ALTERNATIVE APPROACH

For audits to result in the recommendations being implemented, they must present clear evidence of the need for change. Proof is required to overcome resistance to change and complacency Auditors constantly struggle with reporting audit results in a way that inspires action. Putting a dollar amount to poor performance is one way of forcing change but it can have negative side-effects. It can result in an affirmation of the public's poor opinion of government and possible humiliation of government workers.

Audits that collect information from the citizen's point of view are an alternative and positive means for producing change. This type of evidence can clearly demonstrate how a program may not be accessible or have the intended outcomes--and can do so with a humanistic approach. Government workers, rather than being threatened, can understand and identify with the public's frustration. Further, embedded within this type of audit are suggestions for the future design of programs. Auditors should look for opportunities to put citizen point of view audits into practice.

SUZANNE FLYNN is the elected auditor for Metro, a regional government that includes three counties and 25 cities in the Portland, Oregon, metropolitan area. Previously, she was the elected auditor for Multnomah County, Oregon, for eight years. She is a Certified Internal Auditor and has a master's degree in urban planning.