Backup storage appliances come of age.

Data is one of the corporation's most important assets, but it's also one of the corporation's biggest headaches. Data needs to be fully protected in the face of growing data volumes, the demand for shorter restore windows, and reduced storage expenditures. Believing that "two out of three ain't bad," many companies have sacrificed short restore windows for tape's large capacity and economical price point. Most know that backup to disk can enable much faster restores with higher reliability and easier administration, but in the past the cost has been too high.

Traditional backups have relied on tape because it is cost effective for fast backups and getting data offsite, but it is slow and unreliable for restores--the main reason backups happen in the first place. Other types of data protection do rely heavily on disk (replication, mirroring and snapshots are prime examples) but IT reserves these more expensive methods for critical data with high availability and rock-solid recovery requirements. Disk is also ideal for data warehouses and repositories, but having to house multiple backup versions makes heavy demands on disk-based backup. On top of cost issues, getting an integrated backup function to work across multiple applications, hosts and storage targets is extremely challenging.

More and more companies have been looking at backup appliances to let them use disk-based backup at a reduced price point, but these systems have lacked advanced data protection features. The challenge for backup product makers is to provide a disk-based long-term storage system that supports multiple backup applications, multiple versions of backups, allows restore to use disk's random access capability, has a comparable price point to tape, and ensures reliable and verifiable recoveries.

Backup and Recovery Requirements

Many disk manufacturers have triumphantly announced the death of tape, but tape sales remain healthy. It's easy to see why: tape technology advances have included better robotics, increasing capacity and performance on drives and media, and the ability to store multiple copies or versions at a low cost and on removable media. However, manual tape handling is awkward and prone to error, tape needs careful tuning to optimize streaming, and backup quality can be hard to determine until you go to recover--at which point you really hope the quality is there.

One of the main technical problems with tape drives is that they must be tuned to avoid what is called the shoeshine effect--starting, stopping and repositioning the tape. To avoid this, it's much more efficient to stream tape drives by sending several concurrent backups. This process, called multiplexing, does help backup performance by minimizing the shoeshine effect. However, it takes extra time to read images and handle multiple incoming backup sources.

Disk storage counters these disadvantages by avoiding the shoeshine effect, handling incremental backups better, making off-site backup easier and more efficient, and providing more reliable recovery:

* Avoiding the shoeshine effect. Disk arrays do not need a steady stream of data, so there is no shoeshine effect even for small incremental backups.

* Simplifying and accelerating the backup process. Storage administrators like incremental backups because they help shrink backup windows, but tape restores are simpler and more effective with full backups. Disk-based backup systems allow administrators to schedule more incremental backups without worrying about performance penalties or risking restores.

* Making off-site backup copies easier and more efficient. Since it isn't multiplexing backup data from multiple backup clients on one tape, disk-to-tape copying efficiently organizes backup data by client, which speeds up the recovery process. Disk-to-tape copying also provides more flexibility than tape: tape-to-tape copying does not allow other backup or restore operations, but disk allows simultaneous access.

* Superior recovery. Disk-specific technologies like RAID make disk a more reliable medium than tape. One bad tape can cause an entire restore operation to fail, but RAID protection allows a restore to complete successfully even with a failed disk.

* Efficient single-file recoveries. According to analyst firm Strategic Research, 87% of all restores are single-file recoveries, not full-system recoveries. Disk is a random access medium, and is ideal for single-file recovery.

In spite of these advantages, people are still backing up to tape. So what's the problem? Disk has lagged behind tape as a long-term storage medium because of cost. Consider the price of storing four weeks' worth of full and daily incremental backups. Here's the math: If you assume that incremental backup sizes are about 5% of the original data set, over a month's time the backups will consume five times the original data size. Given this capacity, using primary disk subsystems for backup storage is cost prohibitive. Less expensive ATA-based disk arrays are cheaper, but at $10-$20/GB they're still much more expensive than $1-$5/GB for tape-based backups.

A disk-based storage system must avoid the disadvantages of tape and disk while keeping their advantages: disk's effectiveness, reliability and speed, with tape's capacity and price point. This means that is must provide high performance for both backup and restore operations, be cost-effective, and enable reliable, verifiable storage.

Advanced Disk-Based Backup Systems

Backup system best practices include compression, accelerated backups and restores, cost advantages versus tape, verifiable recoverability and highly resilient storage, and simple and seamless integration with backup software.

Compression for Disk Price Parity

A key technology for fast and economical disk-based backup is compression. Compression creates a smaller and more cost-effective system by pooling redundancies and storing only unique data. Backup images contain a lot of redundant data, especially weekly full backups. Even incremental backups, which capture changed files only, usually capture unchanged data blocks as well. Global compression pools backup image redundancies and stores only unique data. By slicing data into sequences of variable lengths, global compression detects duplicate files, redundant patterns within and across files, and repeated patterns within blocks. By pooling and storing only unique data, the technology produces a highly efficient and simpler-to-manage backup environment. While most compression technologies offer around 2X compression, some more global approaches can reduce repetition of large segments across the history of storage, with order-of-magnitude smaller results.

High Performance for Backup and Restore

Performance should be comparable to high-end tape drives or network attached storage (NAS) systems, without tape's multiplexing operations. With tape drives, multiplexing has been used to compensate for slow clients, multiple slow networks, and small incremental backups. Backup appliance best practices yield multiple, concurrent backup streams sent at varying speeds, and must be easily scalable.

Administrators typically make a full backup on the weekend, along with daily incremental backups. If the company requires a full system restore later in the week, tape-based backup must chug through several days of incremental backups in order to update files modified since the previous full backup. Using random access disk, this hours-long process takes a tiny fraction of the time that's required when using tapes.

Highly Reliable Storage and Verifiable Recoverability

The backup system should be highly reliable and allow IT administrators to continuously confirm the integrity of the recovery copy. Through extensive data and metadata verification, the system can validate the reliability of the recovery copy. This ability is an improvement on RAID: RAID storage systems do not provide file system verification. It's quite possible to overwrite good data on a disk subsystem, and to return bad information. Unless IT carefully plans for software faults and disk errors or have automated file system verification in place, backup data stored on RAID may or may not be good when it comes time to restore.

A reliable backup system will provide advanced RAID features like mirrored configuration (RAID1+0), hot spares and automatic restoration after (perish the thought!) a disk failure. It should also be able to verify the entire backup image at backup time, have a highly tolerant append-only file system, and offer solid write protection after a power interruption.

* Verifying the integrity of the backup image. The simplest way to verify backup quality is to restore and check the entire backup image. However, this requires extra storage and processing power, and the initial check may not be enough--IT should periodically check the backup image to make sure it remains recoverable. Since this procedure takes considerable time and effort, it's unlikely that IT will regularly check the entire backup. And they shouldn't have to; the backup system should do it by running a continuous validation process in the background.

* Append-only file system. Conventional disk storage systems can make very fast random block changes, which nicely supports transactional data operations. However, backup files are large and sequential, which means that restores are much more demanding than other types of file requests. An append-only file system has greater fault tolerance for large-scale restore movements because it allocates new blocks for writing data in only one direction. The system will not overwrite these blocks until it encounters a formal clean command.

* Verifying writes after power loss. If a drive suddenly loses power, it's often impossible to tell if it finished writing to disk--writes may look complete in the write back cache on the drive, but the data may not make it onto the actual platter. NVRAM technology allows the system to validate full writes. Another use for NVRAM's verified writes is to catch bad blocks. No matter how reliable the disk, bad blocks happen--errors in cache, firmware, hardware and bugs are all causes. When drives are mirrored, eventually the bad blocks on one drive will yield an inconsistent mirror. The backup system should be able to detect a mismatch, identify the bad blocks and resynchronize the mirrors.

For example, Data Domain's Restorer is an on-site disk-based restore repository that replaces ATA RAID systems or NAS, and complements tape by relieving the pressure of being the primary backup and restore target. It offers high capacity with 20X Global Compression, accelerates backups and restores with cost advantages similar to tape, verifies recoverability, and operates with standard backup applications.

Backup storage appliances like the Restorer offer fast and extremely reliable recovery. By making the backup and recovery process faster, simpler, verifiable and more cost effective, sophisticated storage systems extend the benefits of backup software and vastly improve the data protection infrastructure.

Brian Biles is vice president of marketing at Data Domain Inc. (Palo Alto, CA)

www.datadomain.com