Data replication over the WAN.

The need to perform WAN data replication is growing, as the demand for geographical resiliency increases. Advance-ments in networking technologies in past few years have aligned to support this important Business Continuance/Disaster Recovery (BC/DR) requirement. Specifically, optical networks with their support for storage networking protocols, inclusion of flow control mechanisms, and efficient transport capabilities have become a key enabler for cost-effective data replication networking outside the data center. In addition, the increasing availability of Managed Storage Services is helping enterprises protect their valuable business data.

Data Replication Concepts

Data replication technologies duplicate or "mirror" mission-critical business data from one highly available storage disk array to another. The movement of this data within the data center occurs in "real-time," characteristically over a high-bandwidth, dedicated network (see Figure 1). Compared to tape backup, this data protection scheme is generally more expensive due to the need for multiple disk arrays, which are usually part of an intelligent storage system. This expense is of offset by the ability to quickly (within minutes) resume operations in the event of a disaster or system failure. Data replication can occur in a synchronous, asynchronous, or semi-synchronous mode of operation.

[FIGURE 1 OMITTED]

In synchronous mode, block level data transactions or Input/Output (I/O) requests are committed to one or more remote storage systems prior to sending a write acknowledgement back to the primary unit.

The process flow is as follows:

* Data is written to the primary storage system

* Data is then written to the remote system(s)

* The write operation is completed and host notified only after the data has been written to all storage systems.

The host application is not allowed to proceed until data is successfully committed to all storage systems. For this reason, synchronous mode is the most sensitive to network latency and bandwidth. Synchronous data replication ensures that the data in all the storage systems is identical at nearly every point in time. Due to the complexity and timing issues, typically no more than two storage nodes are involved simultaneously in synchronous data replication.

In an asynchronous mode of operation, I/Os are written to the primary storage system and then sent to one or more remote storage systems at some later point in time. The host application functions as usual, unaware whether or not the data has been written to the remote storage systems. The network interconnecting the storage systems does not impact this mode of operation as significantly. However, the data on the remote systems is not an exact mirror of the host due to the time lag. Asynchronous data replication is also called adaptive copy or point-in-time copy. This mode is ideal for disk-to-disk backup or taking a snapshot of data for off-line processes such as testing or business planning. The time lag enables data to be replicated over lower bandwidth networks but does not provide the same level of protection as the other modes.

As the name implies, semi-synchronous is a hybrid approach whereby I/Os are written on the primary storage system, the application is allowed to proceed, meanwhile the data is synchronized with the remote storage system(s). This approach is similar to asynchronous with the exception that attempts to synchronize data on remote storage systems occur immediately (minimal time lag). Therefore, the network is once again an important consideration.

Determining which mode of operation to utilize for each application environment generally requires support from disaster recovery planning experts. It is common to utilize more than one mode of data replication within the same application infrastructure.

Networking Data Replication Over the WAN

A good disaster recovery plan includes geographically dispersed storage systems. Characteristics of an inter-data center storage network infrastructure to support data replication include:

* Low Latency 

* Minimal Packet Loss

* Flow Control Mechanisms

* Network Resiliency

* Scalable Bandwidth

With these sensitivities in mind, Coarse/DenseWavelength Division Multiplexing (C/DWDM) and Synchronous Optical NETwork /Synchronous Digital Hierarchy (SONET/SDH) both provide excellent MAN- and WAN-based options for data replication networking.

C/DWDM is a technology that maps data from different sources and protocols together on an optical fiber with each signal carried on its own separate and private light wavelength. This is commonly referred to as a lambda or [lambda]. Using DWDM technology, up to 80 and (theoretically, more) separate wavelengths carrying data can be multiplexed into a light stream transmitted on a single optical fiber. C/DWDM can be used to interconnect data centers via a variety of storage protocols such as Fibre Channel, FICON, and ESCON. C/DWDM is well suited for natively (no protocol conversion) extending Storage Area Networks (SANs) in cases when multiple communication links or channels are required. It allows for efficient use of fiber resources. It has been verified to support data replication over distances up to several hundred kilometers. C/DWDM provides bandwidth from one to several hundreds of gigabits/second (Gbps). In true business continuity (BC) scenarios, data replication over C/DWDM can provide the foundation for active-active geographically dispersed clusters, such as Sun Microsystems-Nortel Networks Enterprise Continuity solution (see Figure 2).

SONET/SDH technology is based on Time Division Multiplexing (TDM) where traffic from multiple subscribers is multiplexed together and sent out onto the network as an optical signal. SONET/SDH services are widely available within metropolitan areas around the world. SONET is the North American standard while SDH is utilized internationally. SONET is well established, with more than 130,000 SONET elements deployed over tens of thousands of fiber miles in North America alone. New technologies such as Generic Framing Procedure (GFP) and Virtual Concatenation (VCAT) enable multiple protocols (including storage protocols) to run over SONET/SDH links. GFP and VCAT map SAN traffic into a virtually concatenated SONET/SDH pipe. The combination of SONET/SDH, GFP and VCAT enable optical connectivity that can scale from 50Mbps to 10Gbps, meeting the needs and budgets of a wide range of IT environments. With this technology, enterprise data centers can be interconnected over thousands of kilometers for data replication and other storage-networking needs. Storage over SONET/SDH is a reliable, readily available networking option within the metropolitan area and ideal for interconnection between one or more metropolitan areas.

Data replication, particularly in synchronous mode, requires in-order packet delivery, latency tolerances in the handful of millisecond range, data rates per communication link near one or two Gbps, and network path redundancy. Deterministic optical networks can provide all of these most efffectively. Optical networking options (like C/DWDM & SONET/SDH) provide the lowest latency, highest bandwidth, and most reliable transport mechanism. An optical network is capable of delivering data 200 km in one millisecond or from New York City to Los Angeles in 20 milli-seconds. Typically, five microseconds/kilometer is used to calculate the one-way latency. Optical networks support various types of network topologies such as point-to-point, ring, and meshed. Restoration times in the event of a fiber cut on a transport path are under 50 milliseconds.

Unlike an IP-based network, SONET/SDH and C/DWDM are designed for loss-less data transmission, which is an important factor in a data replication environment. Optical networking equipment now provides buffer credits and caching mechanisms for flow control in SAN environments to enable efficient utilization of the network link. Optical signals from multiple enterprises carried across the same service provider link remain physically isolated from each other. In contrast, IP does not maintain the separation between the control and data planes and as such, is more vulnerable to attack. Security of data is a major concern for storage network administrators given the nature of the information being protected.

To obtain storage network connectivity between data centers, customers can choose between private network build-outs (owning or leasing equipment and transport mechanisms) or obtaining a managed service from a service provider. With a managed service, an enterprise relies on the carrier to ensure reliable network connectivity and can acquire additional bandwidth as required. Industry leading service providers are currently offering both C/DWDM and SONET/SDH as a managed service to support data replication. Storage vendors actively work with optical networking companies and service providers to verify interoperability, create new solution offerings, and ensure ongoing customer support. The decision on which connectivity scenario is best usually depends on availability, operational requirements, and cost. SONET/SDH connectivity is readily available to enterprises worldwide and well suited to support data replication for low to moderate bandwidth requirements both within metropolitan areas and beyond. C/DWDM is mostly available within metropolitan areas and capable of supporting moderate to high bandwidth requirements.

Summary

Any data replication plan should include geographically dispersed storage systems. Optical networks are ideal for supporting data replication. SONET/SDH is widely available, scalable, reliable and affordable. C/DWDM is well suited for business continuance and disaster recovery infrastructures with moderate to high inter-data center bandwidth requirements. Careful analysis of flow control, tolerances to packet loss & latency, bandwidth requirements, and network resiliency are needed to successfully implement a sound data replication strategy. Optical connectivity for networked storage is available as a private build-out and more commonly through a managed service. Optical storage bandwidth is available from 50Mbps to hundreds of Gbps. Distances between data centers ranges from approximately 10 kilometers to several thousand. The stringent network performance requirements of data replication are a primary driver for selecting optical transport.

www.nortelnetworks.com

Rene Dufrene is a senior manager for Nortel Networks within the Global Alliance organization (Morristown, NJ)