It's 2003: do you know where your data is? The government is enforcing strict new guidelines on archived data. Is your company complying?

In an era of diminishing trust and public skepticism, regulations are being enforced with greater vigor to enable regulatory authorities to accurately reconstruct past processes and events from electronic records. These sweeping initiatives are being targeted throughout the economy, with some regulations focusing on securities, broker-dealers, pharmaceutical companies, healthcare organizations, major manufacturers and public corporations with more than $75 million in market capitalization.

Legislation now requires many of these U.S. organizations to retain certain records in a way that prevents them from being erased or modified for substantial time periods, sometimes 30 years or more. Steps must also be taken to prevent records from being accessed by those without authorization. Some of these regulations, like those based on the Sarbanes-Oxley law, are new. Others, such as SEC regulations applying to broker-dealers, have existed in one form or another since the 1930s. However, the common themes are broader regulatory purview over more types of records and substantially strengthened enforcement. The main thrust behind strengthened records-retention regulations is the government's desire to maintain an exact record of past activities in order to improve corporate governance, protect investors, enhance national security, ensure the safety of new drugs or medical devices, and modernize medical care, while protecting patient privacy.

Securities Trading 

As a result of some widely publicized Wall Street scandals, the SEC is now enforcing its Rule 17a--originally written in the 1930s--much more aggressively. The use of electronic communication in business has exploded, and brokerage houses are now heavily reliant on e-mail, instant messaging and electronic forms (tickets, statements, approvals, etc.) than before. The SEC now requires the retention of all electronic client communications and many other brokerage records on non-erasable, non-rewritable media (also known as write-once/read-many or "WORM" media). Additionally, the SEC is demanding increasingly rapid responses to more frequent and broader requests for information. A recent enforcement example of the new regulatory reality for broker/dealers is the $8.25 million fine levied on five Wall Street firms in December 2002 for failure to retain regulated e-mails for the proper amount of time. Other firms have been fined or forced to pay large fees because they were unable to find and recover archived e-mails from tapes in a timely fashion.

Healthcare

The Health Insurance Portability and Accountability Act of 1996 (HIPAA), whose privacy rules came into effect April 14, 2003 (with more rules to follow), is designed to improve the efficiency of healthcare through improved access to patient records while simultaneously protecting patient privacy. The government's position is that the quality of medical care can be improved through rapid online access to patient records, but that strong protections must be in place to guard against malfeasance and misuse of confidential patient data. While HIPAA does not mandate how data is stored, the requirements are effectively forcing healthcare payer/providers (hospitals, insurance companies and HMOs) to manage all patient records electronically using secure systems and secure media.

Life Sciences and Pharmaceutical Industries

Federal regulation 21 CFR Part 11 is designed to streamline the process that brings drugs to market, a major interest of the Bush administration. The goal is a well-designed, well-managed flow of information about drug development, testing and batch manufacturing so that the accelerated path from discovery to market will be simultaneously fast and safe--not to mention well-documented. Rigorous records retention is essential to both verify each drug has been thoroughly tested before approval and to ensure a proper investigation should something go wrong with a drug. Since most pharmaceutical companies also maintain clinical trials data, they must also carefully guard the privacy of such data for HIPAA compliance. Enforcement of 21CFR Part 11 has been temporarily suspended because drug companies objected that it was applied too broadly and that compliance was too unwieldy. The government is now recasting the regulation to make it more specific, but it's clear that the regulation will still require careful retention and safeguarding of records for many years.

Corporate Financial Statements

Congress passed the Sarbanes-Oxley Act in response to a series of major corporate financial scandals where C-level executives claimed that they were not accountable for--or even not aware of--faulty financial statements. The new legislation, which affects U.S. public companies with more than $75 million in equity market capitalization and quarterly reporting requirements to the SEC, specifies that CEOs and CFOs must personally certify financial statements as accurate, under penalty of jail time. Additionally, "all audit or review workpapers" must be retained for five years from the end of the corresponding fiscal period. Sarbanes-Oxley specifies significant criminal penalties for "whoever knowingly alters, destroy, mutilates, conceals, covers up, falsifies, or makes a false entry in any record, document, or tangible object." Although open to interpretation, such wording implies great care as to how records associated with the production of financial statements are archived. Many organizations will no doubt conclude that implementing secure WORM storage for such key business records is much simpler and less expensive than risking a financial scandal or losing executives to prison terms.

The U.S. Department of Defense

The new DOD standard, known as DOD 5015.2-STD, specifies design criteria for electronic records management applications (RMA) software, which include data retention capability. The deadline for compliance with 5015-2 is June 2004. This is a standard for specific software applications, but its implementation implies using hardware storage systems capable of rigorous records retention, proper security settings and, when appropriate, thorough destruction of classified information. Government organizations outside the DOD are expressing interest in using the standard as well as a guideline for good records management. The Joint Interoperability Test Command (JITC) conducts certification tests of RMA software for certification, and storage may be included in a certification test as an option.

Self-Regulated Businesses and Industries

Many organizations have established their own data retention policies to maintain industry standards, preserve their knowledge base, safeguard vital business records, provide protection against legal actions, or comply with accepted best practices. In all cases, they need reliable, cost-effective storage that protects data against modification, deletion, and inappropriate access. For example, many software and manufacturing companies must keep the final source code or design files for each shipping product for several years to satisfy the expectations of different European or Asian countries.

Putting the Right Infrastructure in Place

Compliance with electronic records retention regulations, whatever their source, involves three distinct issues:

Storage medium. Most regulations do not specify storage technology, but in all cases security, data integrity, cost of ownership, performance, accessibility, and searchability must all be taken into account. The choices for electronic records are tape, optical disk and magnetic disk.

Applications. The application you choose to manage your data must comply with the relevant regulations or enable such compliance, providing features such as records protection and online indexing, categorization, search, and audit capabilities.

Policies and procedures. Your company must clearly define how data is to be moved and stored, how and when authorized IT and other personnel can access and modify the data, and then if and when data should be destroyed after a certain retention period. Policies must ensure that unauthorized employees do not have the ability to inappropriately access, alter, or delete records.

Bringing all three of these areas into compliance will produce an overall solution that satisfies the relevant regulations and minimizes liability exposure for the organization. An additional benefit is that a properly designed system will also provide disaster recovery capability, ensuring that nothing short of a major disaster could alter the records, permit an inappropriate security breach, or cause loss of critical data.

Establishing a Data Retention and Protection Policy: Tape, Magnetic Disk, or Optical Media?

SEC Rule 17a-4 specifies that data must be stored on "non-rewriteable, not-erasable" media. For other regulations, WORM storage is not explicitly required but may be a de-facto requirement. (It's like saying that firewalls and anti-virus software are not required by any law but no IT organization lives without them.) Organizations must examine their options and choose the best data-retention storage medium for their needs. In most cases, magnetic disk storage will be the medium of choice for several key reasons:

Fast access to data: The intention of most of these regulations is not only to retain records but also to enable investigators and company lawyers to find records quickly. SEC Rule 17a-4 specifies that companies must respond to requests for data in days, not weeks. When a customer service agent is on the phone refinancing a mortgage or processing an insurance claim, customers expect an answer within seconds. In investigations of the side effects of a drug, authorities may need clinical trial data urgently. HIPAA compliance depends on fast and secure access to patient records. These needs all preclude the use of offline or offsite storage on tape or optical disks.

Searchability: Magnetic disk storage enables Past searches across large amounts of data without the need for locating mad mounting tapes or optical media. Use of robotic libraries often entails determining the right cartridge, loading it, and then waiting to retrieve the data. This can make indexing or searching across many files very slow and expensive.

Cost effectiveness: The cost of ATA magnetic disk storage continues to drop, making it more and more competitive to optical and tape media. Tape media alone is still quite inexpensive, but does not include the cost of the libraries, drives, maintenance, slow retrieval performance, and media management. When total cost of ownership (TCO) is considered, the fact that all data stored on a magnetic disk storage system is quickly accessible all the time means searches are faster and less costly. This makes magnetic disk storage the clear choice.

In the past, optical and tape media offered WORM storage capability. Magnetic disk storage systems now exist that enable users to write records that cannot be modified under any circumstances. The Network Appliance regulatory compliance solution includes integrated hardware and specialized software (Data ONTAP, SnapLock) that runs on all its primary and nearline storage systems. SnapLock is not just an add-on piece of software but an inherent part of the storage operating system that can be activated for compliance.

Choosing a Magnetic Disk Technology

ATA-based disk drive systems have improved in reliability and capacity while dropping in price, which makes them highly competitive. However hardware alone does not provide a solution. An optimized storage microkernal and file system that provides the performance needed for indexing, search, fast backup, and remote mirroring are critical as well. Here are key characteristics to look for in an ATA-based storage system:

Speed: Slow system drives can tie up servers and users unnecessarily when archiving email and other records to compliant volumes. Look for a high performance storage system that doesn't sacrifice speed to achieve compliance. Storage should write data to compliant volumes at the same speed as other volumes.

A stable, solid OS: A field-proven operating system with years of continuous debugging and enhancement provides a solid, reliable compliance solution. Beware of solutions built on new, unproven platforms.

A common OS: Using the same OS and management interface consistently across all product lines enables a vendor to give users great flexibility in architecting compliant solutions while maintaining a low TCO.

Open protocols: The storage system should provide open protocol access to read files from the directory structure in compliant volumes via CIFS and NFS. Use of proprietary APIs should be avoided, as these may make it more difficult to access or migrate data over the years that it must be retained.

Sophisticated, yet simple-to-manage WORM implementation: It should be simple to enable WORM and to modify applications to work with the data-retention solution. The simplest magnetic WORM storage system only requires two lines of code: one to set the expiration date, the other to WORM the file.

Flexibility and rich data management tools: The system should safely retain regulated data, yet also be usable for other applications such as data backup, disaster recovery, general reference information storage, etc. It should also offer options such as tape backup, Snapshots, flexible access controls, and robust replication.

A secure clock: The system should not allow expiration of retention dates simply by setting the clock forward to future, as this would permit inappropriate modification or deletion of records before their retention period has passed.

Architecting Compliance: Two Case Studies

Case Study 1: A securities corporation with U.S. headquarters and regional offices in Europe, the Pacific Rim, and the Americas needs to set up SEC 17a-4-compliant storage systems in each region. Each office needs less than a terabyte of WORM-volume storage. Many of these offices already use storage appliances.

The vendor offers a line of storage products that vary widely in capacity but share a common operating system. A storage appliance is installed in each regional office, and in each case the data-retention component of the OS is activated to create a compliance WORM volume on each appliance for storage of securities trading records.

All the compliance volumes at these regional offices are mirrored back to similar compliance volumes on one large storage system in one of the corporation's data centers. The architecture enables a WORM-to-WORM, many-toone mirroring architecture that produces 17a-4 compliant, remote copies of records from several offices on a single shared storage system. The compliant volumes at the data centers can also be mirrored to a single large storage system at corporate headquarters, or to backup storage systems--all while retaining 17a-4 compliance.

Case Study 2: A medium-sized Boston bank needs data storage compliant with SEC 17a-4. The company's future needs are unclear at this point; the solution must be flexible and grow with the amount of regulated data. The company runs several homegrown applications that each must be able to work with the compliant storage system.

The vendor offers compliant storage systems in a variety of sizes and performance levels. The bank can begin by buying small-capacity systems that scale up easily and open the way to substantial growth.

The company easily adapts the compliant storage systems to its homegrown software by adding a single line of code to establish the expiration date, and another to WORM the file. The storage can be expanded as needed or extra capacity of the new storage system can be used for other non-compliance purposes, improving the ROI of the technology investment.

The Bottom Line: Magnetic Disk Storage for Compliance

An examination of compliance storage options leads to the conclusion that for the great majority of customers, magnetic disk storage solutions available today offer the best combination of reliability, flexible architecture, simple implementation, cost-effectiveness, data access, and data protection for organizations that need to be in compliance with records-retention regulations.

www.networkappliance.com

Krishnan Padmanabhan is director of Regulatotory Compliance Marketing at Network Appliance, Inc. (Sunnyvale, CA)