Printer Friendly

Taking the risk out of disaster recovery services.

In years past, disaster recovery (DR) planning was exclusively associated with the corporate data center. Companies believed their resources and future were secure as long as a plan was in place to protect the corporate mainframe computer system and ensure the organization's ability to continually process information. However, in the aftermath of a series of nationwide disasters that have occurred over the past five years with greater frequency than ever before, this type of DR scenario is now akin to servicing an automobile's engine while neglecting to care for its tires. As a result, an increasing number of companies have come to understand that disaster recovery is no longer simply the concern of large banks and securities firms.

Simply put, in today's highly competitive business environment, few companies can afford to be without a strategic plan that protects all of the computer-based operations necessary for the company's day-to-day survival. After all, if a disaster knocks out a company's ability to produce information, what's left for the data center to process? "Today, it is no longer acceptable to focus solely on the recovery of the data center," says Edward J. Evans, president of Comdisco Consulting Services, a division of Comdisco Disaster Recovery Services (CDRS). "In the 1990s, it is important to stress the steps of analysis, prevention and recovery that are needed in order to provide back up for the organization's priority business functions."

Indeed, disaster recovery's traditional focus on the data center has now progressed to the desks of all employees, says William J. Kelly, senior vice president at New York-based J.P. Morgan. Mr. Kelly acknowledges that some areas are easier to plan for than others. For example, a function that simply accesses a mainframe is far easier to address than one that relies extensively on local area networks.

While the particulars of each plan will vary depending on each company's size and needs, the individual elements of any plan must include a data center audit, business impact analysis, backup network strategy, disaster declaration procedures, business and data processing restoration, departmental reconciliations, as well as ongoing planning and maintenance. Although protecting company operations spanning from the data center to individual PCs is a daunting task, a finely tuned corporate-wide DR strategy can accomplish this goal, says Anne M. McCarthy, manager of marketing communications for CDRS. "The risk manager is one of the key people in making this happen. They are the owners of this process. They look daily at risks and intangibles as well as insuring against any potential damages."

However, disaster planning requires developing a careful strategy. Among some firms, DR has traditionally met resistance from upper management because most business decisions are viewed in terms of a return-on-investment basis. One reason for this is that the price of planning can average anywhere from $80,000 to $150,000, depending on an organization's size and recovery needs, Ms. McCarthy says. Additionally, a plan can take up to a year to complete. However, if this sounds like a lot of time and money, think of how much the average company would lose by missing just one eight-hour work day due to a disaster.

But even with this in mind, some firms still look at DR as a bad return on investment. In essence, a DR plan requires upper management to suspend its accepted business standards and approve and pay for a planning process that does not immediately enhance company profitability. Yet without a comprehensive DR plan, an increasing number of executives understand their companies may not survive the effects of a disaster; a survey published by the Gartner Group Inc., a Stamford, Connecticut-based market research and consulting firm, reported that 59 percent of the DR executives surveyed said that a seven-day outage would put their business into "financial jeopardy" or cause "financial ruin."

RECOVERY SOLUTIONS

In regard to DR, the optimal approach is to develop a recovery strategy that includes the use of an alternate processing site. And just as disasters have become more frequent over the past five years, so have the alternate site options. Many companies first implement a DR plan in order to qualify for insurance coverage. One traditional option has been to make arrangements with another company with similar facilities to use that firm's data processing operations after hours in the event of a disaster. Although this arrangement is far from ideal, difficult to implement and feasible only if the disaster is extremely localized, it is nevertheless a simple way for a company to gain access to alternate facilities.

Most companies, however, hire a DR company to provide either hot or cold site facilities. Hot sites are ready-to-go computer environments that provide backup for the data center, whereas cold sites are open rooms with adequate power and air conditioning where a company can install its backup hardware and equipment, says Claude Brazeil, U.S. program manager of disaster recovery services for Hewlett-Packard. However, due to the rising number of service businesses that rely on low-cost toll-free numbers as well as the increase in the use of telemarketing, many corporations now derive as much revenue from voice communications as they do from the data side of the house. As a result, DR providers such as CDRS and SunGard now offer PBXs (typically AT&T System 75/85) with call rerouting at their hot sites so a company can continue to conduct its business in the aftermath of a declared disaster.

Additionally, since many companies need to recover business functions such as on-line telephone order entry and "just-in-time" manufacturing capabilities, DR providers have begun to offer recovery solutions for these operations. For example, CDRS is now offering an innovation that it calls "Workarea Recovery Centers," which, unlike the traditional hot site, allow companies to relocate key business functions and personnel to any one of nine CDRS Workarea Recovery Centers. These centers provide a hot site as well as a wide range of other services, including a local area network (LAN), data terminal (IBM 3270 and AS/400 DEC and Tandem), PC (Token Ring or Ethernet LAN connections), workstation, and voice and business staff recovery capabilities. Each workarea consists of a prewired workstation with basic telephone service, a print station with basic telephone service, and fax and copier support. Phone services include 800 access, automatic call distribution and recorded announcements and a variety of special services through a Meridian 1 PBX, such as an automated attendant function and managing incoming and outgoing calls through automatic call distribution management reports.

SunGard Recovery Services Inc. also offers similar recovery centers that provide support and data connectivity for telephones, terminals and LAN workstations. These facilities, known as MetroCenters, can also provide access to SunGard's Centralized Voice Recovery System, which provides remote connection services for the client's voice-intensive operations such as order processing, customer service and telemarketing. In addition, SunGard also offers an extensive range of Quick Ship products. According to James MacMicking, regional consulting manager for SunGard Planning Solutions, the wholly owned consulting subsidiary of SunGard Recovery Services Inc., SunGard can ship its clients inventoried PCs within a 24-hour period, as well as LAN servers and other equipment.

At Hewlett-Packard, a service known as HP Backup provides clients with a number of features such as multiple recovery centers, around-the-clock access to systems and personnel, and access to HP 3000 and HP 9000 Series 800 systems and peripherals. Hewlett-Packard also offers telecommunications capabilities and disaster site restoration assistance.

DR providers have had a number of success stories. For example, one of CDRS's recent disaster cases involved a subscriber that discovered a corruption in its mainframe. However, with a Continuous Availability Services option, backup was in place within hours and business restoration became a non-issue, Ms. McCarthy says. Another recent disaster, which occurred in a subscriber's computer room, was resolved through the use of CDRS's mobile recovery capability, which includes a trailer with a prefabricated computer environment inside; in this instance, the mobile unit was driven to the site and business was resumed as quickly as the power was hooked up.

CDRS's focus is on "the protection of information, right down to the Rolodex and individual PCs," Ms. McCarthy says. This, of course, has become a more difficult task over the past few years, because an increasing amount of information has left the centralized hub of the mainframe and been spread among a growing number of PCs.

Although CDRS, SunGard Recovery Services and Hewlett-Packard are major players in the disaster recovery industry, a number of other companies have recently entered the market. One of the most important among these firms is IBM, which introduced its Business Recovery Services unit in 1989. Other providers of DR services include AT&T, with its Accunet Reserved series of on-demand digital services, members of the "Big Six" consulting firms as well as the Bell Operating companies and their respective telcos.

Subscribers of DR services typically sign three-to-five year contracts. The monthly payments vary significantly according to the amount of equipment requiring backup and the contract length. Monthly fees run from $1,000 to $50,000, with the average at $6,000, Ms. McCarthy says. Besides this recurring cost, a subscriber must pay a fee of between $5,000 and $50,000 when officially declaring a disaster. This declaration provides the company with immediate access to recovery space.

Besides being guaranteed a hot site or other space in the event of a disaster, signing up with a DR provider gives companies the opportunity to stage periodic network tests. Comdisco, SunGard and Hewlett-Packard encourage frequent testing of their subscribers' DR plans. However, scheduling tests for active centers can be difficult, says Mr. Kelly. He suggests setting aside time on weekends or performing partial tests that involve particular business units.

WHAT'S IN A PLAN?

Before utilizing DR services, however, a company must develop a workable disaster plan. Typically, assembling a plan for a client takes about three months, Mr. MacMicking says. "We introduce the plan at the top to upper management and then plan from the bottom with line managers," he explains.

The approval of DR planning by upper management can determine the difference between a plan's success and failure. "The DR planning effort must be a top-down approach, with upper management determining objectives, assigning tasks and holding people accountable for them," says Ms. McCarthy, adding that "without this involvement, the plan will not work." Mr. MacMicking concurs, saying management's approval and involvement are "absolutely vital. In fact, we won't create a plan for a company without top management's approval because if the plan doesn't work, our reputation is on the line." Looking at DR from a dollars-lost-per-hour perspective is one of the best ways to gain the support that is needed from top management.

Putting together a disaster recovery plan can be an exhaustive process, but most DR providers offer consulting services on a fee basis. The plans typically start with a systematic analysis of the various departments that comprise a company's business. "With a good DR plan, recovery can be instantaneous," Mr. MacMicking says. "But this requires the plan to identify and prioritize the departments and functions that are most critical to the business's survival." For instance, while a firm may need to resume its trading activities immediately after a disaster in order to maintain its revenue stream, the firm's once-a-month billing procedures may be able to withstand an outage of several days.

Typically, the DR planning process covers three major areas. The first, the business planning phase, determines which aspects of the company's business are the most critical to its survival; this step creates the justification for the overall plan. Mapping out a plan requires examining every department in the corporation. This in itself can be a delicate process, because a DR plan requires determining the true value of each area of the business. This step will involve identifying the potential risks or exposures that threaten each area of the organization. "Throughout this process you will want to determine the strategies and resources you need to keep your key departments running," Ms. McCarthy notes, adding that this analysis should also help answer other questions, such as who will make decisions during the period of the business disruption and how these decisions will be communicated to the rest of the organization.

Each department should be made responsible for carrying out objectives for its area, says Belinda Wilson, a disaster recovery planning consultant with Hewlett-Packard. "Although the plan is introduced by upper management, a person or representative from each department should be given responsibility for meeting the plan's requirements," she says.

Determining the feasibility of the DR plan from a technical standpoint constitutes the second phase of the planning process. "After deciding which departments and employees to put at an off-premises site, it is essential to ensure that they'll have all the equipment and technical support to accomplish their jobs," says Mr. MacMicking, emphasizing that high-priority departments may rely on other, less important departments. "In these cases, make sure that these other departments are also covered by the plan," he adds.

The third step in the planning process is to ensure that company personnel will be able and willing to implement the plan. For example, if the business resumption requires employees to work off-hours, some may not be able to do so because of family or other responsibilities. The plan should also take personnel changes into account, Ms. Wilson adds. "Avoid the situation where one person is the only one who knows about the equipment needs of the department," she says. Finally, when determining the necessity for a DR plan, company size should not be a factor, since business resumption is as critical to a start-up company as it is to any Fortune 500 firm.

Once the DR plan has been created, it will need to be tested and updated on a regular basis. Besides conducting full-blown tests or rehearsals, risk managers should look at their plans on a quarterly basis to ensure that all its features are still relevant, taking into account any technological changes that the company has introduced, such as computer system upgrades, or the addition of new disks or boards, Ms. Wilson says.

In fact, DR providers said testing significantly improves their subscribers' networks. In an era where many corporate networks are reconfigured every three years, the alterations and improvements can now be accomplished with improved DR in mind. These improvements typically involve reducing single points of failure, dual paths for alternate routing and the availability of an access line to the recovery network prior to termination at the local central office.

PLANNING RESULTS

Companies that have a sound DR plan in place will reap the rewards of their foresight. For example, when a fire struck a Consolidated Edison substation in New York City's Wall Street financial district on Friday, August 13, 1990, the disaster left a 36-square block area, including the 110-story World Trade Center, American Stock Exchange, New York Mercantile Exchange and New York Commodities Exchange, without power. According to Contingency Planning Research, a Jericho, New York-based consulting firm that maintains disaster recovery statistics, the outage affected 320 data centers and forced the New York Federal Reserve bank to use its backup data center for the first time.

While service was restored to most of the Wall Street area within eight hours, many remained without power for up to six days. Although J.P. Morgan's corporate headquarters was in the path of 1990's Wall Street outage, Mr. Kelly says that his company's backup electrical generators kept the building operational throughout the six-day outage. "The Wail Street disaster further highlights the importance of having a disaster recovery plan," he says.

Other organizations also take a proactive approach to protecting themselves from disasters. "We stress preplanning," states Susan Meltzer, associate director of risk and insurance at Bell Canada in Toronto. By studying the effects of past disasters, Ms. Meltzer says her organization can better prepare itself for the future. Ms. Meltzer also reports that her company has developed contingency plans in conjunction with its sister company, Northern Telecom Ltd. Although both firms service remote areas, they had a plan ready when a central office switch near St. James Bay was damaged by a flood. Bell Canada acted swiftly and had a replacement switch flown in by helicopter - even though the switch had to come all the way from North Carolina.

Disaster recovery planning has matured significantly since its earliest days, when protection efforts were focused exclusively on the company's data center. Today, however, DR has become a corporate-wide risk management approach aimed at safeguarding all key company operations. The fact that DR consultants are increasingly invited - and feeling welcomed - in the executive offices indicates that many top managers regard DR planning as an urgent matter. "Five years ago, our major contact would be, say, the vice president of information services," Mr. MacMicking says. "Today, however, it's the company chairman or president."
COPYRIGHT 1993 Risk Management Society Publishing, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1993 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Meade, Peter
Publication:Risk Management
Date:Feb 1, 1993
Words:2816
Previous Article:A balanced approach to insurance regulation reform.
Next Article:New environmental challenges for the risk manager.
Topics:

Terms of use | Copyright © 2016 Farlex, Inc. | Feedback | For webmasters