How successful internal audit departments are evaluated.

How successful internal audit departments are evaluated

Because of such recent unveilings as the Treadway Commission Study and the Foreign Corrupt Practices Act, internal auditors have become more important to businesses. Most corporations have increased both the size and visibility of their internal auditors. Bank of America, for example, now has 380 internal auditors, and General Electric has a staff of 120, both companies nearly doubling in the last 10 years the number of auditors they have on staff.

With this increased attention, a practical issue many companies are facing is how to evaluate the effectiveness of their internal auditors. Line management can be evaluated on the basis of profitability or success in meeting a budget, but such measures may not be appropriate for auditors.

In many companies, there is a tendency to evaluate auditors on the basis of quantifiable costs and benefits, such as whether or not they follow their audit plan or produce cost savings. However, an internal audit department may not be effective even when the plan is followed or when actual costs are less than budgeted costs.

To determine how companies with outstanding internal audit groups evaluate their auditors, we conducted personal interviews with the CFO, CEO, and a member of the audit committee of 13 companies reputed to have excellent internal audit departments. These companies were: 3M Company, Aluminum Company of America (ALCOA), American Can Company, Florida Power Corporation, Ford Motor Company, General Electric Company, IBM Corporation, ITT Corporation, Marine Midland Bank, PepsiCo, Security Pacific National Bank, Southern California Edison, and Southern New England Telephone Company. Top management and audit committees in all 13 companies perceived their auditors to be very effective.

How do these companies evaluate their successful internal auditors? The criteria used by top management and the audit committees were of three types: quantitative, qualitative, and feedback.

Quantitative criteria are those measures of effectiveness that are measurable and documentable. They include such factors as the dollars saved versus the cost of operating the internal auditing department; the number of reports and findings; the percent of time spent auditing; the number of auditors per dollar of assets or revenues; the actual costs versus budgeted costs; and the completion of the audit plan.

Qualitative criteria are not strictly measurable and usually are not documentable. They include such gauges as meaningful audit comments, the professionalism of the department, the absence of surprises, and the development of people.

Feedback measures include comments about the effectiveness of the audit by auditees, external auditors, other management people, and the audit committee.

The figure on page 40 identifies the criteria used to evaluate auditor effectiveness in the 13 companies. It shows that CEOs believe "feedback from auditees about the internal auditors" is the most important criterion to evaluate effectiveness. Audit committee members, on the other hand, ranked "providing meaningful comments" and the "professionalism of the internal auditors" as the evaluation factors most important. CFOs' evaluations were similar, with "meaningful recommendations" and "feedback from auditees" deemed most important.

The figure also shows a high degree of similarity between the factors felt to be important by CEOs, CFOs, and audit committee members. For all three, qualitative measures and feedback are significantly more important than are quantitative measures of performance. This result is interesting because most auditors interviewed reported quantitative measures to audit committees and believed they were being evaluated on those reported measures.

From the interviews, it was not clear how CEOs, CFOs, or audit committee members determine that auditors are providing reasonable comments, being professional, or doing an effective job of developing people. Indeed, executives and directors could not articulate quantitative measures of these factors. The impression was that feelings about these qualitative criteria are developed from personal contacts with the internal auditors, especially in the case of audit directors, and from feedback from others. Therefore, even though feedback measures important as qualitative factors, feedback probably significantly influences the perceptions about the qualitative factors.

One by one, what do

the responses mean?

"Meaningful findings and recommendations," which was ranked first by two of the three response groups, is certainly qualitative in nature. The degree of meaningfulness is a matter of judgment and deals with the perceived helpfulness of the audit. Executives want recommendations that are substantial and well founded, and they are highly intolerant of "nit-picking" results that they believe auditors sometimes include simply to have a greater number of findings and recommendations to offer.

The feedback from an auditee is similar to the meaningful recommendations in that both factors consider the value added to the company by the audit department. An auditee's evaluation of auditors is primarily subjective and is influenced by the auditors' professionalism, the helpfulness of their recommendations, and the general feeling that the auditors are "team players." CEOs, CFOs, and audit directors value highly the impressions and judgments of the company's operating managers. Likewise, operating managers look to the judgments of external auditors, CFOs, and others when determining their assessment of internal-auditor effectiveness.

Here are two typical responses from top management about the value of auditee feedback:

From a CFO: "I base my evaluation of the auditors on the recommendations that they make and the degree of acceptance of those recommendations by the auditees. If we got a lot of grumbling from the organization about cheap shots or shoddy workmanship, we would question what is going on."

From a CEO: "I don't have a good quantitative answer about how I evaluate the auditors. The grapevine feedback of auditees is important. I ask questions about the auditors--are they respected?

The professionalism of the internal audit department was considered overall the third most important characteristic of a good internal auditing staff. It takes into account such elements as the independence of the audit department, the integrity of the director and the staff, peer recognition of the audit director, the department's physical presence and bearing, its presentations and ability to respond to inquiry, the director's ability to deal with and gain the respect of top management, and the levels of certification and professional involvement of the auditors. Based on the survey responses, professionalism is determined primarily through interaction with the audit director and the top audit assistants, with such reactions as the following from respondents:

From an audit committee member: "I look at the quality of the personnel, at the independence, organizationally, of the internal audit people. Are they able to set their own schedules...move into areas without problems thrown up in front of them? I look for completeness of their procedures."

From an audit committee member: "I look at the relationship between the head of auditing and the audit committee--the efficiency of reporting to the committee."

From a CEO: "I evaluate them at the audit committee meetings. I see their work output and responses to critical questions, when [they bring in] top assistants, their treatment of problems they have experienced, and how their presentations are made."

The external auditor's evaluation of the internal audit department, surveyed as the fourth most important criterion, was especially critical to CEOs and audit committee members. What is interesting about this evaluation factor is that interviews with external auditors revealed that they base their opinions about internal auditors primarily on the quality of the audit work papers, a factor that top management doesn't really care about.

At the same time, executives and directors expect the internal audit department to provide early warnings of any problems in operations or controls. That's why an absence of surprises was evaluated as important. If problems developed in areas that had recently been audited without the auditor identifying the potential problem, the auditors were judged to be less effective. In other words, executives want to be protected by the auditors. Their typical mindset is: "They [the internal auditors] are effective if their findings are early rather than late. When a problem occurs, we want to know if it is something we have been forewarned of by the audit department or is it totally new."

As far as quantitative methods of auditor evaluation go, the highest-ranking quality was adherence to the audit plan, which was sixth overall. This means evaluating the relative risk of the business and ensuring that riskier segments receive higher audit priority in scheduling audits. All 13 audit departments prepare a formal, long-range audit plan. In addition, most use a quantitative evaluation model to assess the riskiness of various company segments. Each audit department makes a formal presentation of its long-range plan to the audit committee.

Developing people was the seventh most important audit-department characteristic overall. The executive management in all 13 companies views internal auditing as a pool of talent from which the company can fill vacant management positions. A probable future management position is often held out as a "carrot" to attract talented people into internal auditing. One company rotates its top people through the internal audit department to gain a corporate perspective and to develop evaluative skills. That department is very proud of its internal audit alumni, and it points out that most senior executives are former internal auditors.

Three of the five remaining criteria are quantitative measures. Of these, the number of times the internal auditors' work is requested is considered the eighth most important way to evaluate auditors. In short, management believes that the more effective the auditors are, the greater the number of requests they will have to perform audit work.

Also important is the cost effectiveness of the internal auditing department, which was ranked ninth. Most auditors document the cost savings for which they're responsible whenever possible, although they do not always communicate these figures to top management. The low ranking of this measure is interesting because auditing literature suggests strongly that cost savings is one of the most important measures of auditor effectiveness.

Tenth in importance is the audit director's report. It is comprised of communications from the auditing director to the audit committee and top management. This combination of reports contains information on the completion of the audit plan, any significant findings, utilization of audit resources, and general disclosures of the overall "health" of systems, programs, departments, etc. In some cases, the reports are issued jointly with the external auditor and include information relevant to the role of the external auditor.

Finally, the audit committee's and top management's evaluations of the internal audit department were ranked eleventh and twelfth, respectively, in importance. Obviously, these were only considered by one group because top management relies on feedback from the audit committee and vice versa.

A profile of successful

internal auditing

Because the data collected from the interviews identifies the criteria used by executive management and audit directors to evaluate internal auditors in companies with successful audit departments, its importance to other companies is obvious. In general, the interviews show that these individuals base their evaluations of auditors much more on subjective criteria than on quantitative measures traditionally thought to be important by auditors. As one CEO said, "We don't care how they do their job as long as they keep us out of trouble and get their job done."

On the basis of these results, auditors would do well to make sure their relationships with management, auditees, and external auditors are good and to worry less about quantifying their success in various reports.

In all 13 companies, there is a very high level of management support for internal auditing. Because these companies believe in their auditors, they allow them to hire talented people and then provide them with excellent career advancement opportunities. They often place strong leaders as heads of internal auditing, who in many cases are not audit technicians, but general managers. In addition, they have expressed throughout the organization their strong support of the internal auditing function and staff and have allowed the auditors to have high visibility and respect within their companies.

Within this supportive environment, each of the audit departments has made strong commitments to training, certification, building strong relationships with its counterpart external auditors, and being team players.

Table : Internal auditor evaluations: a ranking of techniques

PHOTO : Tureen and stand of silver-gilt, Germany, c. 1735 Made for Augustus III, who was elected King of Poland in 1733.