End result Auditing.

Audits that are limited to compliance tests on existing controls may also be limited in value. Innovative audit approaches--like John Hancock's ERA--focus on the business objectives and end results that the controls are designed to produce.

AT JOHN HANCOCK FINANCIAL SERVICES, end-result auditing (ERA) helps us to reassure our primary customers--the audit committee and senior management--that effective controls are in place and underscores the value of internal auditing. ERA, an audit approach we developed, combines an end-result focused, control-review process with a testing process that identifies the level at which a business unit is currently achieving its business objectives. In other words, end-result control reviews plus end-result testing equals end-result auditing.

CONTROL REVIEWS

End-result control reviews are a natural outgrowth of Internal Control--Integrated Framework, a 1992 report issued by the Committee of Sponsoring Organizations (COSO). The COSO review process broadened the concept of internal control beyond the realm of finance to include all business controls-operating controls, financial reporting controls, and regulatory compliance controls. COSO also established a process for evaluating controls that starts with the identification of a business unit's objectives and the risks that could prevent achievement of those objectives.

The control review phase of our ERA process is based on COSO and starts with an identification of the business unit's objectives. We then identify risks that may prevent achievement of the objectives and identify the control procedures used to reach each goal. After confirming with management our understanding of the business objectives, risks, and control procedures, we perform a preliminary assessment of the control process.

Control reviews can also be performed jointly with management using control self-assessment (CSA). In either situation, operating area management is closely involved with the control review phase.

TESTING

Our testing processes are directed at reaching two distinctly different conclusions. Compliance testing verifies that control procedures are actually in use, whereas end-result testing determines the precise level of achievement for each of the unit's objectives.

Testing each significant control procedure verifies compliance and further enhances our confidence in the control system. But even when the control system "looks" effective and compliance with procedures appears to be occurring, we are still dealing with a theoretical design. Our next step is to verify that the function's objectives, or end results, are actually being achieved. We can't be certain that subtle cracks don't exist in a control system until we examine the end results the control system is supposed to produce.

In addition, when the preliminary assessment indicates a weak control system, we do not abort the testing. End-result testing is critical at this stage to determine whether the weak control system has resulted in failure to achieve the business unit's objectives, as well as to quantify negative results. We believe audit committees and senior management want and need to know the impact of any weak controls. The results of the testing will also help to demonstrate the value of the recommended control improvements.

Having completed an analysis of the control system, compliance testing of control procedures, and end-result testing, the auditor is in a position to conclude on the reliability of the control system with a high degree of confidence. End-result testing will frequently identify business shortfalls or problems of which management is unaware, an important added value derived from the testing.

THE ERA ADVANTAGE

In essence, the ERA process answers two fundamental questions that are at the core of every effective audit: whether or not the business area is meeting its current objectives and whether or not effective controls are in place to achieve future objectives. In addition, ERA enables auditors to provide significant improvements in operations, broaden the scope of audit testing, and enhance the CSA process.

OPERATIONAL IMPROVEMENTS ERA provides value that goes beyond control evaluation. Because ERA is directed at a business unit's objectives, the process frequently leads the auditor to identify operational improvement opportunities. Further, including selective, operational tests with the end-result testing increases the chances that the auditor will identify operational improvements. At John Hancock, we expect our auditors to be alert to possible operational improvements throughout the audit and to add selective testing to specify the monetary value of improvements identified. Operational improvements include both expense savings and revenue enhancements.

Reporting hard-dollar savings from control and operational improvements allows the auditor to demonstrate at least the measurable value produced by the audit function. One cash flow recommendation at John Hancock produced enough in annual investment income to cover one-half of internal auditing's costs for the foreseeable future--but that was just the beginning! In two key audits, we identified significant profit-improvement opportunities. One of those

operational improvements was developed using desktop database software to perform in-depth analysis.

Internal auditing is not an overhead function. Every dollar spent on the internal audit process can add far more than a dollar to the bottom line. We don't identify operational improvements in every audit; in fact, we may only identify one significant improvement in every five or 10 audits. Often you can't estimate the specific dollar value of an improvement, and it's important never to exaggerate savings or make wild estimates. But when you do hit those triples and home runs, they clearly demonstrate the tremendous value of internal auditing.

EXPANDED DATA TESTING In today's high-tech world, the auditor's capability to perform end-result testing, operational testing, and fraud identification testing is extended considerably. Such testing was impossible or cost prohibitive a few years ago, but the amount of planning required to access and test the data has been significantly reduced in recent years. Audit software can now be used to test databases for a variety of conditions; and auditors should be performing more, not less, data testing. The nonstrategic "transaction testing" of the past should not be eliminated, but it should be replaced by strategic, conclusion-oriented testing. The maxim that "the devil is in the details" holds merit.

The value of most control improvements is often not quantified until a serious loss comes to light. At that point, it's easy to measure the loss precisely and determine the value of the missing control. Unfortunately, by then it's too late. ERA doesn't provide a magic formula to compute the value of each control; however, end-result testing will frequently allow the auditor to identify and place a monetary value on the current impact of a weak control structure and help to sell the control improvement recommendations.

CSA OPPORTUNITIES Ready coordination with CSA is another significant ERA asset. The framework used in many CSA operations is the COSO model, which is the same framework used for the control review phase of ERA. ERA just extends the process to verify compliance with controls confirm that end-result objectives are being achieved, and identify profit-improvement opportunities. Customers can also participate on ERA teams, just as they do with CSA initiatives.

SYNCHRONIZING GOALS

ERA can help an audit function to achieve its highest potential and demonstrate its value to management. In focusing on end results, the goals of internal auditing are clearly synchronized with those of the audit committee and senior management.

Today's internal auditor must be both a consultant and an auditor. Auditors must assume the role of a friendly management consultant while simultaneously examining every operation with the ownership perspective and intensity of a CEO--in other words, "acting like a consultant and thinking like a CEO." We've found that ERA makes that process much more straightforward.

DONALD ROBITAILLE, CIA, CPA, CISA, CDP, CFSA, MBA, is Vice President and Corporate Auditor at John Hancock Financial Services, Inc. in Boston.