Reporting on systems reliability.

EXECUTIVE SUMMARY

* THE AICPA AND THE CICA HAVE JOINTLY INTRODUCED an assurance service, SysTrust, in which practitioners report on the reliability of an entity's systems. To earn an unqualified SysTrust report, a system must meet all of the 4 principles and 58 criteria.

* A SYSTEM IS AN INFRASTRUCTURE of hardware, software, people, procedures and data that--together in a business context--produces information. A reliable system operates without material error, fault or failure during a specified time in a specified environment.

* THE FOUR ESSENTIAL PRINCIPLES UNDERLYING reliable systems are availability, security, integrity and maintainability. For each there is a set of criteria that enables a practitioner to assess whether a system has achieved that particular principle.

* IN THE UNITED STATES, a SysTrust engagement is performed under AICPA Statement on Standards for Attestation Engagements no. 1, Attestation Standards. In Canada, the engagement is performed using standards found in the CICA Handbook.

* AN UNQUALIFIED SYSTRUST REPORT PROVIDES system users with assurance about system reliability. Management can gain confidence in its own internal systems. A report can also increase the confidence business partners have in each other's systems.

Introducing SysTrust, a new assurance service.

In today's increasingly interconnected economy, one company's glitch on Monday can be another's bad headline on Tuesday. It's not just a company's own systems that need to be reliable; the systems of suppliers, business partners and customers must also be dependable. In the drive to find new markets, reduce costs and provide better customer service, companies rely on each other's systems through outsourcing, partnerships and joint ventures. In response to concerns about unreliable systems, the AICPA and the Canadian Institute of Chartered Accountants jointly developed a new assurance service SysTrust[SM]--to provide assurance that a system is, in fact, reliable.

In a SysTrust engagement, accountants report on the availability, security, integrity and maintainability of a system. A SysTrust engagement includes a system description that delineates the boundaries of the system covered by the engagement, management's assertion about the system's underlying controls and an attestation report by a CPA that evaluates the system against specific criteria. To earn an unqualified opinion, a system must meet all of the SysTrust principles and criteria. (See exhibit 1, pages 76-79, for more details.)

Exhibit 1: SysTrust Principles and Criteria

Availability: The system is available for operation and use at times set forth in service-level statements or agreements.

A1) The entity has defined and communicated performance objectives, policies and standards or system availability.

A1.1 The system availability requirements of authorized users--and system availability objectives, policies and standards--are identified and documented.

A1.2 The documented system availability objectives, policies and standards have been communicated to authorized users.

A1.3 The documented system availability objectives, policies and standards are consistent with the system availability requirements specified in contractual, legal and other service-level agreements and applicable laws and regulations.

A1.4 Responsibility and accountability for system availability have been assigned.

A1.5 Documented system availability objectives, policies and standards are communicated to entity personnel responsible for implementing them.

A2) The entity utilizes procedures, people, software, data and infrastructure to achieve system availability objectives in accordance with established policies and standards.

A2.1 Acquisition, implementation, configuration and management of system components related to system availability are consistent with documented system availability objectives, policies and standards.

A2.2 There are procedures to protect the system against potential risks that might disrupt system operations and impair system availability.

A2.3 Continuity provisions address minor processing errors, minor destruction of records and major disruptions of system processing that might impair system availability.

A2.4 There are procedures to ensure that personnel responsible for the design, development, implementation and operation of system availability features are qualified to fulfill their responsibilities.

A3) The entity monitors the system and takes action to achieve compliance with system availability objectives, policies and standards.

A3.1 System availability is periodically reviewed and compared with documented system availability objectives, policies and standards.

A3.2 There is a process to identify potential impairments to the system's ongoing ability to address the documented system availability objectives, policies and standards and to take appropriate action.

A3.3 Environmental and technological changes are monitored and their impact on system availability is assessed on a timely basis.

Security: The system is protected against unauthorized physical and logical access.

S1.1 The system security requirements of authorized users and the system security objectives, policies and standards are identified and documented.

S1.2 The documented system security objectives, policies and standards have been communicated to authorized users.

S1.3 Documented system security objectives, policies and standards are consistent with system security requirements defined in contractual, legal and other service-level agreements and applicable laws and regulations.

S1.4 Responsibility and accountability for system security have been assigned.

S1.5 Documented system security objectives, policies and standards are communicated to entity personnel responsible for implementing them.

S2) The entity utilizes procedures, people, software, data and infrastructure to achieve system security objectives in accordance with established policies and standards.

S2.1 Acquisition, implementation, configuration and management of system components related to system security are consistent with documented system security objectives, policies and standards.

S2.2 There are procedures to identify and authenticate all users authorized to access the system.

S2.3 There are procedures to grant system access privileges to users in accordance with the policies and standards for granting such privileges.

S2.4 There are procedures to restrict access to computer processing output to authorized users.

S2.5 There are procedures to restrict access to files on off-line storage media to authorized users.

S2.6 There are procedures to protect external access points against unauthorized logical access.

S2.7 There are procedures to protect the system against infection by computer viruses, malicious codes and unauthorized software.

S2.8 Threats of sabotage, terrorism, vandalism and other physical attacks have been considered when locating the system.

S2.9 There are procedures to segregate incompatible functions within the system through security authorizations.

S2.10 There are procedures to protect the system against unauthorized physical access.

S2.11 There are procedures to ensure that personnel responsible for the design, development, implementation and operation of system security are qualified to fulfill their responsibilities.

S3) The entity monitors the system and takes actions to achieve compliance with system security objectives, policies and standards.

S3.1 System security performance is periodically reviewed and compared with documented system security requirements of authorized users and contractual, legal and other service-level agreements.

S3.2 There is a process to identify potential impairments to the system's ongoing ability to address the documented security objectives, policies and standards and to take appropriate action.

S3.3 Environmental and technological changes are monitored and their impact on system security is periodically assessed on a timely basis.

Integrity: System processing is complete, accurate, timely and authorized.

I1) The entity has defined and communicated performance objectives, policies and standards for system processing integrity.

I1.1 The system processing integrity requirements of authorized users and the system processing integrity objectives, policies and standards are identified and documented.

I1.2 Documented system processing integrity objectives, policies and standards have been communicated to authorized users.

I1.3 Documented system processing integrity objectives, policies and standards are consistent with system processing integrity requirements defined in contractual, legal and other service-level agreements and applicable laws and regulations.

I1.4 Responsibility and accountability for system processing integrity have been assigned.

I1.5 Documented system processing integrity objectives, policies and standards are communicated to entity personnel responsible for implementing them.

I2) The entity utilizes procedures, people, software, data and infrastructure to achieve system processing integrity objectives in accordance with established policies and standards.

I2.1 Acquisition, implementation, configuration and management of system components related to system processing integrity are consistent with documented system processing integrity objectives, policies and standards.

I2.2 The information processing integrity procedures related to information inputs are consistent with the documented system processing integrity requirements.

I2.3 There are procedures to ensure that system processing is complete, accurate, timely and authorized.

I2.4 The information processing integrity procedures related to information outputs are consistent with the documented system processing integrity requirements.

I2.5 There are procedures to ensure that personnel responsible for the design, development, implementation and operation of the system are qualified to fulfill their responsibilities.

I2.6 There are procedures to enable tracing of information inputs from their source to their final disposition and vice versa.

I3) The entity monitors the system and takes action to achieve compliance with system processing integrity objectives, policies and standards.

I3.1 System processing integrity performance is periodically reviewed and compared to the documented system processing integrity requirements of authorized users and contractual, legal and other service-level agreements.

I3.2 There is a process to identify potential impairments to the system's ongoing ability to address the documented processing integrity objectives, policies and standards and take appropriate action.

I3.3 Environmental and technological changes are monitored and their impact on system processing integrity is periodically assessed on a timely basis.

Maintainability: The system can be updated when required in a manner that continues to provide for system availability, security and integrity.

M1) The entity has defined and communicated performance objectives, policies and standards for system maintainability.

M1.1 Documented system maintainability objectives, policies and standards address all areas affected by system

M1.2 Documented system maintainability objectives, policies and standards are communicated to authorized users.

M1.3 Documented system maintainability objectives, policies and standards are consistent with the requirements defined in contractual, legal and other service-level agreement and applicable laws and regulations.

M1.4 Responsibility and a accountability for system maintainability have been assigned.

M1.5 Documented system maintainability performance objectives, policies and standards are communicated to entity personnel responsible for implementing them.

M2) The entity utilizes procedures, people, software, data and infrastructure to achieve system maintainability objectives in accordance with established policies and standards.

M2.1 Resources available to maintain the system are consistent with the documented requirements of authorized users and documented objectives, policies and standards.

M2.2 There are procedures to manage, schedule and document all planned changes to the system are applied to modifications of system components to maintain documented system availability, security and integrity consistent with documented objectives, policies and standards.

M2.3 There are procedures to ensure that only authorized, tested and documented changes are made to the system and related data.

M2.4 There are procedures to communicate planned and completed system changes to information systems management and to authorized users.

M2.5 There are procedures to allow for and to control emergency changes.

M3) The entity monitors the system and takes action to achieve compliance with maintainability objectives, policies and standards.

M3.1 System maintainability performance is periodically reviewed and compared with the documented system maintainability requirements of authorized users and contractual, legal and other service-level agreements.

M3.3 Environmental and technological changes are monitored and their impact on system maintainability is periodically assessed on a timely basis.

A "system" is an infrastructure of hardware, software, people, procedures and data that--together in a business context--produces information. See exhibit 2, at left, for clarification of these terms. A system may be as simple as a personal computer-based payroll application with a single user. Or it may be as complex as a multiapplication, multicomputer banking system accessed by virtually an unlimited number of users inside and outside the entity.

Exhibit 2: System Components

* Infrastructure. The physical and hardware components of a system, including facilities, mainframes, servers and related components and networks.

* Software. The programs and operating software of a system, including operating systems, utilities and business applications software such as enterprise resource planning (ERP) and financial systems.

* Personnel. The people involved in operating and using a system, including information technology (IT) personnel such as programmers and operators, system users and management.

* Procedures. The programmed and manual procedures involved in operating a system, including IT procedures such as backup and maintenance, and user-based procedures, such as input.

* Data. The information used and supported by a system, including transaction streams, files, databases and tables.

THE RELIABILITY FRAMEWORK

To describe the framework, the systems-reliability task force, a joint venture of the AICPA assurance services executive committee and the CICA assurance services development board, compiled a set of principles and definitions that accountants will use as the basis for the service.

Unreliable systems will display some common symptoms:

* Frequent failures and crashes that deny internal and external users access to essential system services.

* Unauthorized access, making the system vulnerable to viruses, hackers and loss of data confidentiality.

* Loss of data integrity, including corrupted, incomplete and fictitious data.

* Serious maintenance problems resulting in unintended negative side effects from system changes, such as loss of access to system services, loss of data confidentiality or loss of integrity.

A reliable system is one that operates without material error, fault or failure during a specified time in a specified environment. The four essential principles underlying such systems are

1. Availability. The system is available for operation and use at times set forth in service agreements.

2. Security. The system is protected against unauthorized physical and logical access. (Logical access is the ability to read or manipulate data through remote access.)

3. Integrity. System processing is complete, accurate, timely and in accordance with the entity's transaction approval and output distribution policy.

4. Maintainability. The system can be updated in a manner that provides continuous availability, security and integrity.

For each principle, criteria enable a practitioner to determine if an entity's system met it. The criteria are organized into three categories:

1. Communications. The entity has defined and communicated performance objectives, policies and standards for system availability, security, integrity and maintainability.

2. Procedures. The entity uses procedures, people, software, data and infrastructure to achieve system availability, security, integrity and maintainability objectives in accordance with established policies and standards.

3. Monitoring. The entity monitors the system and takes action to achieve compliance with system availability, security, integrity and maintainability objectives, policies and standards.

A system must satisfy all of the SysTrust criteria to be deemed reliable. To obtain evidence that criteria have been met, a practitioner examines the controls related to the criteria. The SysTrust guidance materials provide practitioners with several illustrative controls related to each criterion.

RULES TO FOLLOW

In the United States a SysTrust engagement is performed under AICPA Statement on Standards for Attestation Engagements no. 1, Attestation Standards. In Canada the engagement is performed under CICA standards for assurance engagements, found in the CICA Handbook. AICPA and CICA professional standards specify that an independent, objective, knowledgeable practitioner will perform tests of management's assertion or of the subject matter to which the assertion relates. A practitioner will gather evidence about the assertion's conformity with the criteria in the same way he or she would in other examination-level engagements: by inspection, observation, inquiry, confirmation, computation and analysis to verify that the criteria have been met. The practitioner then expresses an opinion on management's assertion or on the subject matter to which it relates.

How does a SysTrust engagement differ from existing services, such as a service auditor's engagement performed under SAS no. 70, Service Organizations (in the United States), and S 5900, "Opinions on Control Procedures at Service Organizations" (in Canada)? SAS no. 70 applies when an auditor audits the financial statements of an entity that obtains services from another organization (a service organization). It is designed to provide information and assurance to the auditor of the user organization about controls at the service organization that may affect the user organization's financial statements. A SysTrust engagement is designed to provide report-users with assurance about whether the entity has maintained effective controls over the reliability of a system. In a SysTrust engagement, users will not receive a detailed description of the system, the procedures the practitioner performs and the results of those procedures--as they would in a service auditor's engagement.

Readers also may wonder about the differences between the SysTrust service and two other assurance services--WebTrust and ISPTrust (a new assurance service being developed by the electronic commerce task force that will evaluate Internet service providers). There are differences in both the nature of the systems being addressed and the nature of the assurance being provided. Both WebTrust[SM] and ISPTrust[SM] focus only on Internet-based systems; SysTrust applies to numerous types of systems. And while WebTrust and ISPTrust focus primarily on controls over Internet-based transactions, SysTrust focuses specifically on the reliability of systems themselves. Although it is possible to have a qualified SysTrust report, this possibility does not exist for a WebTrust report.

THE NEED FOR STANDARDIZED ASSURANCE

How can an unqualified SysTrust report (see exhibit 3, page 82) benefit today's competitive business world? An unqualified report can provide many parties with confidence about the reliability of systems they use in e-commerce or for which they pay user fees. Management and the board of directors can gain more confidence in their own internal systems by making sure they are subject to appropriate controls. This enables an entity to differentiate itself from competitors who cannot provide the same assurance. Internal auditors and system owners can use the framework to guide them in developing and implementing a reliable system within an entity. These services can lower costs, help avert systems-development rework and prevent loss of reputation or market share due to unreliable systems.

Exhibit 3: Sample Unqualified Report on the Assertion on AICPA Standards

Independent Accountant's Report(*)

We have examined the accompanying assertion by the management of ABC Corp. that it maintained effective controls over the Financial Services System to provide reasonable assurance that--

* The system was available for operation and use at times set forth in service-level statements or agreements. (Availability)

* The system was protected against unauthorized physical and logical access. (Security)

* The system processing was complete, accurate, timely and authorized. (Integrity)

* The system could be updated when required in a manner that continues to provide for system availability, security and integrity (Maintainability)

during the period Month X, 200X, to Month XX, 200X, based on the SysTrust principles and criteria established by the American Institute of CPAs and the Canadian Institute of Chartered Accountants. This assertion is the responsibility of the management of ABC Corp. Our responsibility is to express an opinion on the aforementioned assertion based on our examination.

Additional information about the AICPA/CICA SysTrust principles and criteria may be obtained from the AICPA Web site, www.aicpa.org. Management's summarized description of the aspects of the financial services system covered by this report is presented in the accompanying description of ABC Corp.'s financial services system.

Our examination was conducted in accordance with attestation standards established by the American Institute of CPAs and, accordingly, included examining on a test basis evidence supporting management's assertion and performing such other procedures as we considered necessary in the circumstances. We believe that our examination provides a reasonable basis for our opinion.

Because of the inherent limitations of controls, errors or fraud may occur and not be detected. Furthermore, the projection of any conclusions based on our findings to future periods is subject to the risk that changes made to the system or controls, changes in processing requirements, or the failure to make changes to the system when required may alter the validity of such conclusions.

In our opinion, management's assertion that it maintained effective controls over the financial services system to provide reasonable assurance that--

* The system was available for operation and use at times set forth in service-level statements or agreements. (Availability)

* The system was protected against unauthorized physical and logical access. (Security)

* The system processing was complete, accurate, timely and authorized. (Integrity)

* The system can be updated when required in a manner that continues to provide for system availability, security and integrity. (Maintainability)

during the period Month X, 200X, to Month XX, 200X, based on the AICPA/CICA SysTrust principles and criteria, is fairly stated in all material respects.

[Signature]

[Date]

(*) Draft report. Actual wording may change.

System integrators, vendors and those who do outsourcing can engage a practitioner to provide assurance about the reliability of the systems and services they provide to their customers. In turn, system builders and consultants can use the framework to design reliable systems. Finally, a SysTrust report on system reliability can increase business partners' confidence in each other's systems.

Consider the following scenarios.

Scenario 1. Acme Co. is competing to win business as a supplier to Fisbees Department Store, a major retailer which has a just-in-time inventory system that depends on its suppliers. Acme can differentiate itself from its competitors with a SysTrust report on its systems. Fisbees also can require all its major suppliers to provide periodic SysTrust reports.

Scenario 2. Acme decides to outsource its employee care systems (human resources, payroll, benefits). As part of its request for proposal, it specifies that the successful bidder must maintain an unqualified SysTrust report on its outsourced systems.

Scenario 3. With the heightened awareness that post Y2K systems may be subject to various reliability issues, companies with dependable systems need to differentiate themselves in the marketplace to preserve shareholder value. Both Fisbees and Acme commission SysTrust reports to assure the reliability of their systems.

Scenario 4. An insurer is asked to provide Fisbees with business interruption coverage. Before writing the coverage,the insurer asks Fisbees to provide a SysTrust report on its inventory management system.

Scenario 5. Fisbees Department Store publishes sales information on its Web site. External stakeholders voice concern about the reliability of the information being disseminated. A regulator requires a periodic report on the system that furnishes financial information to the entity's Web site.

Scenario 6. Fisbees is divesting itself of a subsidiary. To increase buyer interest, ensure top price and reduce buyers' due diligence procedures, it commissions a SysTrust report on the subsidiary's systems.

These and other scenarios suggest ways a SysTrust report can benefit both internal and external stakeholders of entities engaged in commercial activity that relies on key information systems.

To support effective and consistent use of SysTrust reporting, the systems-reliability task force is developing several training courses (see exhibit 4, below). In addition, it is putting together a competency model illustrating the skills needed to perform a SysTrust engagement, as well as practice aids including model workplans, engagement letters and checklists of controls.

Exhibit 4: SysTrust Products and CPE Training Courses

1. AICPA/CICA SysTrust Principles and Criteria for Systems Reliability

This publication contains authoritative guidance that explains SysTrust. Included is background on the service; key definitions of a system and system reliability; the principles and criteria against which systems will be evaluated; illustrative controls corresponding to each criterion that supports system reliability; examples of management's assertion; system description and report examples. It equips practitioners to perform SysTrust engagements. (Available December 15, 1999)

Level: Basic
Product Number--Print: 060465JA
Product Number--CD-Rom: 060466JA

Format: Print and CD-Rom
AICPA members: $11.50; nonmembers: $14.50
AICPA members: $11.50; nonmembers: $14.50

2. SysTrust Service: An Overview to the New Assurance Service on Systems Reliability

This self-study course introduces practitioners to the new SysTrust service. It will help practitioners decide whether to offer SysTrust and what resources they need to develop the service. (Available December 15, 1999)

Level: Basic
Recommended CPE Credit: 8 hours
AICPA members: $119

Format: Text
Product Number: 730027JA
Nonmembers: $149

3. How to Perform a SysTrust Engagement

This practical course trains practitioners to issue an attestation report on a system's reliability based on the SysTrust service's four key principles and criteria. (Available December 15, 1999.)

Level: Basic
Recommended CPE Credit: 8 hours
AICPA members: $119

Format: Text
Product Number: 730026JA
Nonmembers: $149

All materials are available by calling the AICPA order department at 1-888-777-7077.

REQUIRED COMPETENCIES

To perform a SysTrust engagement, practitioners should have a number of competencies, including information technology (IT)-related skills. However, the degree of IT sophistication will depend on the nature of the system the CPA is examining. Many practitioners already have most of the essential skills needed to conduct an effective evaluation of internal control. With modest additional training, practitioners can enhance these skills to enable those with internal control evaluation skills to provide valuable SysTrust services to their clients.

Some aspects of a SysTrust engagement may require more specialized IT skills. Those skills can be brought to bear on an examination as needed--they are not required for the entire engagement. Thus, with effective teamwork and skills management, practitioners can combine their talents with those of colleagues who are IT specialists to provide SysTrust services.

THE NEXT STEPS

For the immediate future, the systems-reliability task force will work on building awareness and acceptance of this new assurance service among practitioners and the business community--including management, boards of directors, system developers, outsourcers and internal auditors. It will seek to demonstrate the value of SysTrust to both industry and practice. For practitioners, SysTrust represents potentially significant engagements they can leverage into opportunities to provide other services such as security profiling and design, application controls consulting and privacy consulting.

Will a SysTrust report prevent the situations headlined at the start of this article? By itself, no. What SysTrust will do is reduce the risk that such situations will occur and provide a common level of assurance that management has taken prudent steps to address reliability and to implement a balanced set of controls that operate effectively. The SysTrust principles and criteria are a rigorous test of system reliability from which business partners, customers and regulators can take comfort.

EFRIM BORITZ, PhD, FCA, CISA, is Ernst & Young Professor of Accounting and director of the Center for Information System Assurance at the University of Waterloo, Toronto. He is a member of the systems-reliability task force. His e-mail address is [e-mail]. ERIN MACKLER, CPA, is a technical manager in the AICPA assurance services division. She is the staff liaison to the systems-reliability task force. Her e-mail address is [e-mail]. DOUG McPHIE, CA, CISA, is a partner with Ernst & Young in Toronto. He chairs the systems-reliability task force. His e-mail address is [e-mail].3