Web Warps Techniques And Technology For Continuous Off-site Data Protection.

Historically, users of corporate information were co-located with most of the data they needed, which was necessary to get the needed level of performance at an acceptable cost. A major disaster hitting that data generally also impacted the users, or the facilities and infrastructure with which they accessed the data. As a result, disaster planning focused on a complete recovery solution for facilities, infrastructure, and personnel, while data recovery focused on the extent of data loss, known as recovery point objective (RPO).

Under this traditional scenario, it could take hours or even days to restore to a server, sifting through the process of recovering data from a stale nightly backup tape made many hours before the disaster. Often it also required manually recovering lost transactions that occurred after the backup. Typical solutions that were applied to minimize data loss involved continuous, online backup of data to a secondary system at a distant location, and a variety of products emerged in the marketplace specifically to address these needs.

Of these systems, which generally relied on proprietary hardware, many merely tried to extend local redundancy techniques such as disk mirroring. But trying to adapt to distributed-data redundancy techniques that were designed for local protection meant that too many of these solutions required huge amounts of dedicated bandwidth. They also imposed significant performance penalties on the production application just so the remote system could keep up.

To develop innovative solutions specifically designed for distributed-data replication, alternative technologies soon emerged that took a fresh look at the problem.

New Problem, New Solution

Ubiquitous access is the name of the game today. The ready availability of inexpensive wide-area bandwidth and global connectivity via the Internet has altered the landscape so that users now can access data from virtually anywhere and at anytime. Consequently, any network downtime for whatever reason can impact users located around the globe. For disaster planning, this evolution has entirely changed both the problem and the potential solutions.

For example, a local site disaster, such as fire or flood, or a regional disaster, such as an earthquake or hurricane, can destroy or prevent access to a critical server. With such widespread access capability and dependency, the damage to the server can leave hundreds or thousands of geographically distributed users without access to critical data and applications. Meantime, the sheer numbers of users for any given application has multiplied with the advent of internal intranets linking far-flung enterprise sites and extranets available to partners and customers, making system downtime even more detrimental.

The data protection focus, therefore, has shifted from relatively simple RPO to total recovery time with the ideal scenario often being nearly immediate resumption of the application with up-to-the-minute data to an alternate location. That's the big advantage of today's networking technology and "intelligent" applications that can easily reroute requests from one location to another. Of equal importance, such new technology can make available to most any organization continuous off-site data vaulting, even with fast failover capabilities, at affordable levels.

As costs drop for both wide-area bandwidth and storage, most companies looking to protect critical data can justify a standby server hosted at another company location or co-located at an ISP, VAR, or other service provider. It even could be a spare server at an employee's home, linked by a DSL connection for continuous updates. Or, for example, a major East Coast bank uses standard commercial servers, software, and fractional T1 lines to replicate data from 24 branch locations at a secure, disaster-hardened data center. Remember that all of these solutions can always scale up.

Choose A Solution

To determine the appropriate solution, a company first must define its specific requirements, which may vary between applications, and ask some questions. How many users access the application and from what locations? What is the cost of downtime per-minute or per-hour? How recreatable is the data? If the last backup finished at 6 a.m. and there was a disaster at 5 p.m., could the data entered or modified during that period be recreated? How long would it take and would it cost? What is the cost of that lost data?

Ideally, there would be no data lost under any circumstance--not a single in-process transaction--and zero impact on the user. That, however, is difficult, requires expensive, specialized solutions, and reduces overall application performance by imposing still more overhead.

In fact, to maintain "zero loss" it requires a two-phase commit approach where every transaction is written and acknowledged in multiple locations before the next task can proceed. This approach has a significant performance penalty which becomes proportionately greater as the distance between systems increases. This, of course, is because communication speed is limited by the speed of light, in the best case, and much less in the real world when network protocol and routing latency is factored in.

Using synchronous disk mirroring technologies across a distance presents a similar problem. Here, each write to a disk block needs to be written to the target drive and acknowledged, written to the source drive, then committed to both before any subsequent read or write I/O can be processed by the disk subsystem. The alternative is to buffer then transmit the changes as fast as available bandwidth allows. As long as the available bandwidth is equal or greater to the rate of data change, data will be transmitted and applied almost instantaneously providing "near zero" data loss without the round-trip delay overhead associated with synchronous mirroring or two-phase commit.

With the buffering alternative, if the rate of data change temporarily exceeds available bandwidth, seconds or even minutes of changes could be queued, waiting to be transmitted. Because they're still onsite, these changes could be lost in the event of a catastrophic failure. That is impossible with the synchronous system because the transactions would never occur since everything would have been slowed down to the rate of data transmission to allow the mirroring to keep pace.

Technology Alternatives

There are other technology differences to consider, including where the changes are captured, what kinds of changes are visible and what level of object filtering is available.

A disk subsystem-level mirroring system, for instance, has no knowledge of the application or the operating system (OS). It sees only disk-drive commands and disk blocks, so the minimum amount of data that can be sent is a full disk block--perhaps between 4K to 64K for each change. Filtering and selection usually is at the partition or disk level. Meantime, application-level replication within a database using "triggers" can capture individual transactions or SQL-level commands, but uses application resources (cup, memory, etc.) to execute. This means filtering can be done down to individual database tables.

Still other products operate between the application and the disk. This enables filtering down to the file level, but it still sees precisely what the application is writing to the file system before it is converted into I/O data blocks on the drive. In this case, if a database transaction writes 2 bytes to the transaction logs and then 4 bytes to the database, the same step sequence can be captured and transmitted exactly to the recovery server off-site.

Data Integrity vs. Currency

These differences raise another critical consideration of data integrity vs. currency. Whether the protection model is based on continuous replication or periodic backups, it is vital to insure that the backup image is valid at any time and represents a true snapshot of the original system at a given point in time. With either approach, there must be access to files even while in use.

Using a continuous replication solution, all data on the target system must always represent a specific point in time from the source. This point-in-time consistency can be jeopardized by either skipping transactions or applying any out of sequence information.

This can happen when using an adaptive-replication process instead of a time-sequenced queue. If a change cannot be transmitted immediately, rather than storing changed data in a sequenced first-in, first-out queue, only a pointer to the changed block is stored. As a result, the data is re-read from disk when it is ready to be transmitted. If data is changed multiple times on disk or later deleted so that it cannot be read from disk when needed for transmission, there is a combination of data on the target that never existed on the source. This can cause data corruption, particularly with databases.

Looking Forward

Lastly, each company must ask questions about its IT future. How will a protection solution adapt to increasing loads, future network technologies, or OS and hardware technology changes? Does it need the same server or storage hardware on both the source and target? Can they be mixed and matched? Does the replication connection natively take advantage of open network standards such as TCP/IP, or oblige either protocol conversions or dedicated links?

The fundamental question is whether the solution leverages emerging hardware and network technologies or confines to what is available today. Compared with proprietary hardware-based solutions, software-based solutions generally are more flexible, easily upgraded, and expanded. Some even offer consistent functionality on multiple-server OS platforms, which trims training and support costs, and lets a single administrator centrally manage and monitor multiple systems.

In fact, the same technologies that enable distributed access to data can be extended easily to allow distributed data redundancy and protection, which is particularly timely for the growing number of industries falling under legal mandates requiring such protection. In any case, with its tools and techniques maturing and available at reasonable costs, there is little reason for any company not to embrace appropriate data availability and protection.

Don Beeler is president and CEO of NSI Software (Hoboken, NJ).