Printer Friendly

Taking the disaster out of recovery.

Taking the Disaster Out of Recovery

DECEMBER 1986: A BUILDING FIRE short-circuits the data processing operations of Putnam Investor Services Inc. in Boston. * August 1987: A submerged power substation cuts off power to First Chicago Bank as that city is hit by the heaviest rainfall in its history. * May 1988: A three-alarm fire breaks out in the computer center of Philadelphia's Provident National Bank, jeopardizing all critical applications. * September 1989: Hurricane Hugo sweeps over Robert Bosch Corporation in Charleston, SC, collapsing a roof and flooding its data center. * October 1989: The San Francisco earthquake damages the power lines to Charles Schwab Corporation, causing a complete power outage.

These were all disasters, but the companies recovered, with some help. Just think about those that didn't. Consider these statistics:

* The average company loses 2-3 percent of its gross sales within eight days of a sustained computer outage.

* The average company that experiences a computer outage lasting longer than 10 days never fully recovers. Fifty percent go out of business within five years.

* The chances of experiencing a disaster affecting the corporate data processing center are one in 100.(1)

Disaster recovery. Those are two vital words in today's security managers' and directors' vocabulary. Nowadays these people can't be without a disaster recovery plan or they may soon be without a job.

"It is generally known that the directors of corporations are personally at risk for the actions or inactions of the corporations they direct," explains Randall C. Miller, executive vice president, general counsel, and chief operating officer of CompuSource. "It is not as well known that senior managers, agents, and sometimes employees are also at risk. When there is a catastrophic loss because of a data processing disaster, it is likely that the vice president or manager of MIS can be held personally liable for the loss if there are actions he should have taken to avoid the loss but did not."(2)

"Numerous court cases in which disgruntled shareholders sued officers, directors, and agents for alleged wrong-doing developed the common law standard called the |Prudent Man Rule,'" continues Miller. "It requires officers, directors, and agents to discharge their duties with the diligence and care that ordinary, prudent men would exercise under similar circumstances."(3)

Aside from the legal ramifications of neglecting to safeguard vital data, disaster recovery planning is a business necessity. Simply put, business relies on computers more than ever before and will continue to do so.

"It doesn't matter what you call it--automated data processing or management information services--the life of a business or organization is at risk without disaster management," stresses Robert J. Russo, CPP, chairman of the ASIS Standing Committee on Disaster Management. "Government has already recognized the need for disaster recovery planning. But," he adds, "private industry has been slower despite some notable successes in disaster recovery."

Russo stresses the need for disaster recovery services in the full scope of continuous business operations. "Some well-developed plans are bound by turf--where, say, the MIS department is responsible for information up to a certain point, such as backups. The next level up, perhaps those in charge of data transmission, doesn't include data recovery as its responsibility. A continuous flow of recovery needs to be established."

Disaster recovery experts recognize this need due to sheer data base dependence. "More and more companies rely on data centers to do business," explains Judith Eckles, director of corporate communications for SunGard Recovery Services, located in Wayne, PA. "With this increasing reliance on technology and pressure from the financial sector that mandates disaster recovery procedures or threatens possible lawsuits, you can't conduct business without disaster recovery in place."

Eckles is referring to the Interagency Policy on Contingency Planning for Financial Institutions, which expands contingency planning requirements for all segments of federal depository institutions. The policy is published by the Federal Financial Institutions Examination Council, a coalition of representatives of the Office of the Comptroller of the Currency, the Federal Deposit Insurance Corporation, the Federal Home Loan Bank Board, the Federal Reserve Board, and the National Credit Union Administration.

Another factor that encourages senior management to do more than just ponder the benefits of disaster recovery planning is the Foreign Corrupt Practices Act (FCPA) of 1977. "The |need' for the FCPA arose out of the Watergate crisis," explains Miller. "An equally important objective of the law was management accountability through record keeping. Section 102 requires management to provide shareholders with reasonable assurances that accurate books and records are properly maintained and that the business is adequately controlled."(4) It is at this juncture that disaster recovery becomes a do or die option.

These laws and regulations, along with a number of other factors, provided the seeds for the growth of a disaster recovery industry in the 1980s. John Ratliff, senior vice president of international marketing for Comdisco Disaster Recovery Services (CDRS) in Rosemont, IL, explains that regulations in the financial industry played only one part in the growth.

"The demand for disaster recovery services had several influences," says Ratliff. "Besides the financial industry's regulations, CEOs, CFOs, and CIOs have become better educated about the critical need for sustained data transmission. The time windows to survive and get systems up to reestablish business have been shrinking due to the growing reliance on computers and the heightened awareness provided through the press that there are success stories where data can be recovered."

Insurance is another incentive for companies to arrange disaster recovery planning. Elizabeth A. Milligan, CPP, a member the ASIS Standing Committee on Disaster Management, knows. Milligan is an account engineer for Arkwright Mutual Insurance Company in Lexington, MA, which is part of the Factory Mutual System of companies. Arkwright insures $790 billion worth of property in the United States alone. Factory Mutual insures more than 40 percent of North American industry and a sizeable chunk of worldwide industry.

"A property conservation program plays a vital role in recovering from man-made and natural disasters. There are numerous cases in which the inability to recover physical assets, customers, market share, and reputation have forced [companies] out of business altogether," Milligan explains.

Factory Mutual's desirable customer profile includes a company that believes in a property conservation program, which includes physically protecting assets as well as planning for the unforeseen. The insurance industry, in general, supports varying degrees of disaster management preparedness.

Milligan points out that the difference between having an action plan in place and not having one can cost a company. "There is a definite rate differential between a highly protected risk and a less protected risk. And the term |highly protected' encompasses a broad range of items including physical protection and personnel action."

DESPITE THE GROWING DEMAND FOR disaster recovery services, the number of companies in this industry is still relatively small, due largely to the unique technical support required to recover data.

"In the late 1980s there was a lot of instability in the industry," notes John Butch, division manager of HOTSITE in Niles, OH. "Many companies started up between 1987 and 1988 but failed, causing uncertainty and turmoil within the industry. The last three years have been rather stable as far as vendors are concerned. Not many new players are entering the market."

Disaster recovery is not an easy business to get into, a fact that many of the short-lived companies of the late 1980s discovered. It's a highly capital-intensive business and requires significant technical expertise. Consider a few of the basic requirements for data recovery:

* hardware such as CPUs, front-end processors, tape drives, disk drives, printers, and network equipment;

* storage facilities for manuals, plans, and magnetic media;

* equipment to provide electronic vaulting and real-time recovery;

* communications facilities to handle critical voice and data transmissions; and

* reliable, competent support personnel such as telecommunications specialists, systems programmers, and recovery operations specialists.

Disaster recovery by its very nature is an expensive operation for both the provider and the user. For the provider, there are high start-up costs to support the technology, and it takes a lot of subscribers--companies that need the services--to absorb those costs. And as Butch notes, "It's risky business."

For these reasons many providers in the disaster recovery business are subsidiaries of larger companies already active in computer equipment leasing and other data processing services. For example, CDRS is a subsidiary of Comdisco Inc. Comdisco is a remarketer and lessor of new and used IBM equipment.

Other well-established players in the industry include SunGard, a subsidiary of SunGard Data Systems Inc., a computer services-related business, and HOTSITE, a division of CompuSource.

One of the latest players in recovery service today is IBM Business Recovery Services. "IBM's entrance into disaster recovery solidifies the market," notes Datapro Reports, a leading source of analytic information on the computer, communications, and office systems industries. "While the disaster recovery market was maturing on its own, IBM quickened the pace. IBM is capable of offering recovery services for huge systems that its competitors cannot. Another advantage for IBM is its outstanding reputation for service."(5)

Many players in the disaster recovery business are niche players, companies that offer varying degrees of services from cold sites (computer-ready facility shells) to consulting services to unique platforms or services in specific geographic areas.

All these companies, despite size, yearly revenue, and cost to subscriber, have one mission in common: to get their customers back to business without any loss of data...or reputation.

WHEN DECIDING WHAT TYPE OF DISASter recovery service is best for a company, a number of factors should be considered, such as money, time, personnel that can be dedicated strictly for disaster recovery, and the kind of recovery facility that best suits the company's needs. Several of the factors intertwine to influence each other.

To begin, a company must evaluate which of its systems are critical to the ongoing operation of business and what the critical systems' needs are. For example, does the company need instantaneous recovery, thus requiring electronic vaulting (where transactions are immediately recorded as they are entered) or does it just need simple backup of data processing?

Once these considerations are defined, a company can budget money for disaster recovery, set aside time to prepare and test recovery plans, and assign personnel to focus on disaster recovery. For example, companies with highly technical data requirements, such as financial institutions, may need to contract with a commercial vendor, test recovery services several days per year, and dedicate a full-time team of individuals specifically to data recovery.

In general, businesses have the following methods to choose from:

* a service bureau

* a shared contingency agreement

* a commercial vendor

* building a company site

Service bureaus. According to Datapro Reports, service bureaus are becoming a less desirable option. "Although service bureaus provide immediate access to timesharing services at a cost that is usually less than other commercial backup options, service is usually available for short-term use only, with limited data base security," a recent issue notes. "In addition, the service bureau's capability may change significantly, depending on the amount of computing capacity available on its computer system."(6)

In short, companies may compete with other subscribers for space at a time when a business's livelihood is on the line. Datapro recommends this arrangement for companies without critical data communications needs.

Shared contingency agreements. These are basically reciprocal arrangements with other companies that use like equipment and applications. "The agreements are usually unenforceable," cautions Datapro, "and often involve programming changes in order to run the recovering site's applications on different equipment configurations."(7)

Similar to the problems of service bureaus, testing time is limited to availability. "In addition, if the two firms are located in the same geographical area, there is the potential the same local disaster could render both data centers unusable."(8)

Commercial vendors. Most companies decide to contract with commercial vendors. Over the years, commercial vendors have expanded their services to include anything from developing a contingency plan for companies to providing hot sites (fully operational processing centers) or cold sites to consulting.

The trend over the last several years, according to Jon W. Toigo, author of Disaster Recovery Planning, is to use hot sites as the primary mainframe backup. A hot site, with backup systems and communications network already in place, can usually be available within 24 hours' notice of a disaster. In general, hot sites have hardware, software, and support personnel in place and ready upon disaster declaration.

Cold sites, the other option, are computer-ready facility shells with heating, ventilating, and air conditioning systems in place and ready for subscriber-provided hardware, software, and personnel.

For many companies, however, hot sites are the only viable option, especially small businesses and companies in the financial services industry. "Hot sites remove many of the unknowns inherent in strategies that involve the acquisition of replacement hardware in a disaster situation," explains Toigo. "Increasing diversity of installed software products and a concurrent increase in the utilization of hardware capacities have rendered service bureaus and mutual backup agreements anachronistic as competent mainframe backup strategies. And, today, as in the past, few companies can afford to build and outfit duplicate data centers."(9)

Companies that choose commercial services subscribe to a vendor for services. A subscription varies from a few hundred dollars a month to as much as $50,000, depending on the services required for that company.

Subscription rates and services required are determined up front in a contract. "Most contracts are for the long term," explains Eckles of SunGard. "No one plans for a one-year disaster recovery contract. A subscriber needs time to plan, to test, and to get used to the facility."

"Disaster recovery is a highly technical sale. It's not an emotional buy. You have to meet the unique needs of the customer," adds Butch of HOTSITE. "Our sales cycle is very long. Prospects may talk over a period of 10 years before signing a contract--at first discussing their needs, our services, and then negotiating a contract."

Price in the disaster recovery arena is negotiable. Too many variables and needs unique to each subscriber prevent a vendor from offering a flat fee to all prospects. "Each company has different requirements," continues Butch. "Every data processing system has some idiosyncrasies in how they design their operations that dictate what will be recovered and how."

Built into the negotiated price, or subscription, are some standard elements, such as testing time and support personnel, that the disaster recovery company provides. Both factors are, again, negotiable, but on average companies contract for 48 hours of testing time at a hot site facility per year and are assigned a technical support team to assist them.

Separate from the subscription fee is a declaration fee. This fee, which is used by most disaster recovery companies, is charged to subscribers that actually declare a disaster. The fee deters companies from randomly calling something a disaster and protects other subscribers at that facility from losing testing time needlessly.

Declaration fees are by necessity hefty, and many companies refuse to disclose the exact amount charged. "We don't make money off of declaration fees," explains Eckles of SunGard. "What constitutes a disaster is left up to our subscribers. But we don't want them to declare them prematurely. When we foresee the possibility of a disaster, such as a forecasted hurricane or tornado, we put subscribers on alert to get them prepared. We go into a readiness mode by having them ship us their tapes. This allays fears and helps avoid having them make a declaration."

"We just cannot afford not to have a declaration fee," adds Butch of HOTSITE, "because when a disaster is declared it inconveniences the subscribers scheduled to test. They have to vacate a facility to make way for a company or companies with a disaster. With the declaration fee, subscribers make sure that they have to use the facility."

Ratliff of CDRS notes that declaration fees encourage an approval process: "Declarations can only be authorized by individuals high enough in an organization so the facility is used for what it was intended."

Building a company site. Limited funds may be the main reason companies choose not to build their own backup facilities, as doing so demands considerable investment up front. But limited time is the main reason companies do take this route.

"Many organizations consider their dependence on computer systems to be so great that they cannot afford the time to reestablish service at a commercial hot site vendor," explains Datapro. "By maintaining its own alternate processing site, an organization can prepare for its own immediate backup needs, while a commercial hot site may need to reconfigure its systems and peripherals to meet the subscriber's needs."(10)

Companies that can afford to build their own backup facilities usually use them to the fullest extent possible when free of the threat of disaster. For example, many companies use them as research and development, training, and work overflow facilities, adds Datapro.

AS MENTIONED EARLIER, MOST COMPAnies choose to contract with a commercial vendor because it is more cost-efficient. Two major factors to consider include the location of the hot site facility and the number of subscribers it serves.

Any real estate agent will tell a prospective buyer that location is everything, and the same holds true for a disaster recovery site, especially if a company's remote communications capabilities are limited. The last problem a company needs is for its backup site to become a disaster site too. So experts suggest choosing a recovery site that is a safe distance from the company's facilities.

Datapro suggests users consider a vendor's exposure to the same regional disasters they may be involved in, such as hurricanes, earthquakes, and telephone and power outages. "The generally accepted range is approximately 25 to 100 miles from the subscribing company, although to avoid a situation where the same disaster strikes both the subscriber and the hot site, many subscribers opt for hot sites further away."(11)

Most larger vendors can allay any fears about the dangers of dual exposure to disaster because most have disaster recovery centers located throughout the nation.

For example, CDRS has 27 hot sites and 19 cold sites in North America, plus six hot sites in Asia, serving its 2,700 subscribers worldwide. SunGard has 35 separate hot sites within its six megacenters serving more than 1,000 subscribers. And HOTSITE, a smaller, niche disaster recovery provider, serves its more than 250 subscribers through four US recovery centers.

The subscriber-to-site ratio is another major factor to consider when choosing a disaster recovery vendor. As Datapro points out, the higher the subscriber-to-site ratio, the more likely it is that two subscribers will simultaneously need the site's facilities, especially in the event of an area-wide disaster.(12)

Such considerations, however, usually don't concern larger vendors. "We've had five subscribers declare a disaster at one time," explains Butch of HOTSITE. "To avoid cramping of space we matched the needs of the subscriber to our various facilities without a problem. We don't operate on a first-come, first-served basis.

SECURITY, OF COURSE, IS THE HEART OF the issue. Securing data from being lost during the disaster, guarding information in transit from the disaster site to the recovery site, and protecting information at the recovery site from internal theft as well as theft by other companies at the recovery center are some of the security challenges that arise.

"The use of an alternate facility does not negate normal security needs," agrees Datapro. "Security controls at the recovery site should be similar to or stricter than those in place in the firm's normal facilities. Minimum controls should include physical security such as controlled entrances and data security such as encryption for any classified information.

"In addition, the vendor should not release client information, especially during the disaster when the disclosure could cause panic among the client's customers and stockholders."(13)

Physical security measures at the hot or cold sites should include the basics: multilevel card access; CCTV in hallways, lobbies, and parking areas; a 24-hour security force; and fire suppression equipment and safety procedures. "Our hot site areas are separately controlled and distinct from each other so that the integrity of the information is protected," notes SunGard's Eckles.

Protection from fire and its hazards is a major concern for recovery services. As Milligan of Arkwright notes, the national average time between the discovery of a fire and the time the fire department is contacted is 19 1/2 minutes.

To counter this threat, SunGard, for example, has a dry-charged sprinkler system with a system malfunction notification for entire buildings; automatically discharged halon fire suppressant under raised flooring; ionization smoke detectors above and below raised flooring; and two-hour fire-rated walls throughout the facilities. Such measures are typical for most sites.

Another aspect of disaster recovery sites to consider is the discretion they use to protect their clients from unnecessary, or unwanted, publicity. On one hand, company shareholders find it a sound business practice that a plan and procedures are in place in case of a disaster. However, when a disaster does strike, recovery personnel need to work on recovery, not public relations. For that reason many disaster recovery facilities are only discreetly identified to the outside world. As Eckles indicates: "Not identifying SunGard prominently is another aspect of our security."

The risk of internal theft does not escape disaster recovery services, either. To counter this threat, preemployment screening of recovery service employees includes background checks and drug testing, and most employees sign nondisclosure agreements to confirm their understanding of the confidentiality of client accounts.

CDRS's Ratliff notes that on some accounts the subscribers' personnel are the only individuals authorized to be in the specified facility during testing and recovery episodes: "They use our personnel on an as-needed basis. Our personnel undergo the normal precautions for client confidentiality, such as psychological testing, background checks, etc."

SO WHAT HAPPENS WHEN THE DISASTER recovery sites are themselves the victims of a disaster? These companies cannot afford to be without their own disaster management and recovery plans.

Ask vendors and they'll explain that their facilities were built with disaster in mind. "We take advantage of the same systems subscribers have," adds Eckles of SunGard. "A disaster is less likely to happen to us than the other guys." Nevertheless, a good disaster recovery company carries business interruption insurance.

The best insurance to have when looking for a disaster recovery company is quality, according to Tom Goralski, CPP, manager of security and disaster preparedness at Xerox Corporation's El Segundo, CA, site. That means quality equipment, quality computers, quality bonding, quality insurance, and quality legal coverage.

"When you're entrusting another entity with your company's information," explains Goralski, "the quality of the company you invest in is vital. Tremendous liability is involved if they go in and access your information. You should always check the credentials of a company if it's going to touch your most personal records."

In today's high-tech environment, more and more companies are discovering they cannot be without some type of disaster recovery plan. Too much is at stake--money, time, and reputation. If company representatives don't act to protect assets from disaster, they'll pay for it one way or another.

(1)Jon William Toigo, Disaster Recovery Planning (Englewood Cliffs, NJ: Yourdon Press, 1989), p. xvi. (2)Randall C. Miller, "Your Legal Liability in a Corporate Disaster," Contingency Journal, January-March 1990, p. 1. (3)Miller, p. 1. (4)Miller, p. 2. (5)"Disaster Recovery Sites: Market Overview," Datapro Reports on Information Security, March 1990, IS39-001-102. (6)"Disaster Recovery Sites: Technology Overview," Datapro Reports on Information Security, March 1990, IS39-001-123. (7)"Disaster Recovery Sites: Technology Overview," IS39-001-123. (8)"Disaster Recovery Sites: Technology Overview," IS39-001-123. (9)Jon William Toigo, p. 115. (10)"Disaster Recovery Sites: Technology Overview," IS39-001-122. (11)"Disaster Recovery Sites: Technology Overview," IS39-001-125. (12)"Disaster Recovery Sites: Technology Overview," IS39-001-125. (13)"Disaster Recovery Sites: Technology Overview," IS39-001-126.

Joan H. Murphy is associate editor of Security Management.
COPYRIGHT 1991 American Society for Industrial Security
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1991 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:disaster recovery services
Author:Murphy, Joan H.
Publication:Security Management
Date:Aug 1, 1991
Previous Article:Security in the park.
Next Article:A model of disaster management.

Related Articles
Disaster recovery in the new decade: retrofit answers will not make it in the '90s.
Six steps to disaster recovery.
Protecting million dollar memories.
Vaulting provides disaster relief.
Disaster Recovery Yellow Pages, 8th Edition, 1999/2000.
Reasons to be wary.
Peace of mind: disaster recovery plans can keep your business alive.

Terms of use | Copyright © 2017 Farlex, Inc. | Feedback | For webmasters