High availability WAN Clusters.

Within an enterprise data center, servers hosting applications and data-bases are clustered to improve performance, increase availability, and/or provide scalability. Building a cluster typically involves linking the server nodes over an Ethernet-based LAN (see Figure 1). Clustering software, residing on the servers, monitors the health status or the "heart beat" of the nodes in the cluster. In the event of a nodal failure, the clustered node automatically takes over the functions of the lost server. This requires the server nodes to be engineered to take on this additional workload in the event of a fail-over. Server clusters improve availability; however, both processing capability and data reside on the same system thereby increasing risk.

Clusters can be set up in multiple modes of operation such as active/active or active/passive. In the active/active scenario, fail-overs occur automatically with little or no intervention required by the system administrator. Recovery times may be measured in seconds or minutes. In an active/passive scenario, one or more server nodes will be in a standby state. When a fail-over is required, there will be a time delay associated with recreating the user environment. Manual processes may be involved in bringing the fail-over node into service, increasing the recovery time to hours.

Coupling a Storage Area Network (SAN) to clustered servers can provide even higher availability. Specialized data management functions, such as moving information from one system to another, can be handled by specialized data replication software and intelligent storage systems (see Figure 2).

[FIGURE 1 OMITTED]

In this environment, the replicated data is accessible from either storage system, which minimizes the risk of server failure. A SAN typically requires a high bandwidth, reliable network infrastructure using Fibre Channel as the communication mechanism. It is important to separate LAN and SAN traffic to allow bulk data transfer to occur on the SAN without any affect from bursty, less predictable LAN traffic.

Enhancements to clustering and database software, data replication technology, and networking equipment have enabled these systems to be separated over large (up to thousands of kilometers) geographical distances (see Figure 3).

As the distance increases, it is more likely that the operational mode of the cluster will shift from active/active to active/passive. There are many other important factors besides geographical distance that contribute to the options for cluster mode of operation. These factors include:

* Host application(s) tolerance to latency

* Choice of clustering, database, and data replication technology

* Performance capabilities of servers, storage, LAN and SAN switches

* System administration choices such as "Heart Beat Timeout" and "How Often to Send Heart Beats"

* Data replication mode of operation (Input/Output Transaction Acknowledgement Requirements) and SAN flow control mechanism

* WAN technology

Business Continuance & Disaster Recovery (BC/DR) experts can analyze an enterprise environment to determine the best possible solution. Typically, an enterprise is interested in utilizing geographically resilient clusters for the most mission-critical business applications. In these cases, application downtime will result in financial losses amounting to hundreds of thousands or millions of dollars per day. From an overall BC/DR, clusters reside at the top of the food chain by protecting both application and data. Replication mechanisms (which also support clusters) would be ideal if mainly data protection is required. Finally, tape and disk backup technologies would serve as a valuable data protection mechanism but with a larger recovery time compared to replication (see Figure 4). It is very common to utilize multiple application and data protection mechanisms within the same enterprise.

Networking Requirements for Clustering Over the WAN

A sound business continuance plan includes geographically dispersed server and storage systems, some of which may be clustered. The desirable characteristics of an inter-data center networking infrastructure to support WAN Clusters include low latency, minimal packet loss, network resiliency, and scalable bandwidth. Bearing these requirements in mind, Coarse/Dense Wavelength Division Multi-plexing (C/DWDM) and Synchronous Optical NETwork /Synchronous Digital Hierarchy (SONET/SDH) both provide excellent MAN and WAN solutions for geographically resilient clusters.

[FIGURE 2 OMITTED]

C/DWDM is a technology that transparently maps data from different sources and protocols together on an optical fiber. Each signal is carried on its own separate and private light wavelength. This is commonly referred to as a lambda or [lambda]. Using DWDM technology, up to 80 and, theoretically, more separate wavelengths carrying data can be multiplexed into a light stream transmitted on a single optical fiber. CWDM is an economical alternative to DWDM. It offers fewer wavelengths but otherwise has the same technological benefits. C/DWDM can be used to interconnect data centers via a variety of LAN and SAN protocols. C/DWDM is well suited for natively (no protocol conversion) extending LANs and SANs when multiple communication links are required. It ensures maximum and efficient usage of fiber resources. It has now been verified to support clusters over distances up to several hundred kilometers. C/DWDM solutions provide bandwidth from one to several hundreds of gigabits per second (Gbps). C/DWDM is an excellent foundation for active/active geographically dispersed clusters (see Figure 5) requiring high bandwidth.

SONET/SDH technology is based on Time Division Multiplexing (TDM) and transports traffic from multiple subscribers on the network as an optical signal. SONET/SDH services are widely available within metropolitan areas around the world. SONET is the North American standard while SDH is utilized internationally. SONET is well established, with more than 130,000 SONET elements deployed over tens of thousands of fiber miles in North America alone. New technologies such as Generic Framing Procedure (GFP) and Virtual Concatenation (VCAT) allow multiple protocols (including storage protocols) to efficiently run over SONET/SDH links. GFP and VCAT efficiently map data traffic into a virtually concatenated SONET/SDH pipe. The combination of SONET/SDH, GFP, and VCAT enable optical connectivity that can scale from 50 Mbps to 10 Gbps, meeting the needs and budgets of a wide range of IT environments. With this technology, enterprise data centers can be interconnected over thousands of kilometers for clustering and other storage-networking needs. SONET/SDH is a reliable, readily available networking option within the metropolitan area and ideal for interconnection between one or more metropolitan areas.

As discussed, WAN Clusters involve having data replicated at remote site(s). Data replication, particularly in support of an active/active cluster, requires in-order packet delivery, latency tolerances in the low millisecond range, data rates per communication link near one or two Gbps, and network path redundancy. Deterministic optical networks can provide all of these most effectively. Optical networking options (like C/DWDM and SONET/SDH) provide the lowest latency, highest bandwidth, and most reliable transport mechanism. An optical network is capable of delivering data 200 km in one millisecond or from New York City to Los Angeles in 20 milliseconds. Typically, five microseconds/kilometer is used to calculate the one-way latency. Optical networks support various types of network topologies such as point-to-point, ring, and meshed. Restoration times in the event of a fiber cut on a transport path are under 50 milliseconds.

Unlike an IP-based network, SONET/SDH and C/DWDM are designed for loss-less data transmission. This is an important factor for sound operations of a geographically resilient cluster. Optical networking equipment now provides buffer credits and caching mechanisms for flow control in SAN environments. This enables efficient utilization of the network link and can carry storage traffic over longer distances. Optical signals from multiple enterprises carried across the same service provider link remain physically isolated from each other. In contrast, IP does not maintain the separation between the control and data planes--as such, it is much more vulnerable to attack.

[FIGURE 3 OMITTED]

Industry leading service providers are currently offering both C/DWDM and SONET/SDH as a managed service to support business continuance and disaster recovery. Cluster and storage vendors actively work with optical networking companies and service providers to verify interoperability, create new solution offerings and ensure ongoing customer support. The decision on which the connectivity scenario is most suitable for an enterprise depends on availability, operational requirements and cost. SONET/SDH connectivity is readily available to enterprises worldwide and well-suited to support clustering and data replication for low-to-moderate bandwidth requirements both within metropolitan areas and beyond. C/DWDM is mostly available within metropolitan areas and capable of supporting moderate-to-high bandwidth requirements.

[FIGURE 4 OMITTED]

[FIGURE 5 OMITTED]

Summary

WAN Clusters provide a true business continuance solution. Chicago Tribune Company, a subsidiary of Tribune Company, announced in March 2004 that it was deploying such an infrastructure over an optical network to help ensure continued, on-time delivery for millions of its readers and advertisers. Optical networks are ideal for supporting high availability requirements. Not only are they reliable, but they also scale very easily to support other enterprise functions such as data backup and linking voice and data systems. SONET/SDH is widely available, scalable, reliable, and affordable. C/DWDM is well suited for business continuance and disaster recovery infrastructures with moderate-to-high inter-data center bandwidth requirements. Optical connectivity is available as a private build-out and more commonly through a managed service. Optical bandwidth is available from 50 Mbps to hundreds of Gbps. Distances between data centers ranges from approximately 10 kilometers to several thousand. The stringent network performance requirements of WAN Clusters are a primary driver for selecting optical transport.

www.nortelnetworks.com

Rene Dufrene is senior manager, Global Partners--Nortel Networks (Richardson, Texas)