Enabling cost effective global enterprise Storage Area Networks using 10 Gigabit Ethernet.

The demands on Storage Area Networks (SANs) go far beyond where they were only a few years ago. It is no longer good enough to just provide a large pool of scalable storage with good performance. Corporate users require easy and rapid access to data that may be globally distributed. IT managers must deliver on these requirements; ensure rapid disaster recovery capabilities, and accomplish it without breaking the bank. The five key factors defining today's metrics for a successful, global, enterprise SAN deployment are cost: bandwidth, reach, latency, and case-of-access.

While Fibre Channel (FC) has satisfied the early demands for SANs, the evolution of the Internet Small Computer Systems Interface (iSCSI) and 10 Gigabit Ethernet (GbE) from Neterion have enabled a compelling alternative for building Global SANs.

10 GbE iSCSI Full Error Recovery And Distance Advantages

The completion of the entire iSCSI protocol, as defined by IETF RFC 3720, provides the full error recovery features as required by storage managers. These enhancements ensure robust and highly-available system operation across geographically diverse enterprise networks. The maturity and robustness of iSCSI with Error Recovery Level 2 (ERL 2) support, combined with the throughput, distance, and economy of scale advantages of Neterion's Xframe 10 GbE solutions, present a complete framework for deploying globally dispersed, ubiquitous, enterprise SANs. The key points are:

* Ethernet offers seamless integration with the LAN, MAN, and WAN, enabling native sharing of data anywhere in the world.

* Ethernet is based on well-proven protocols, management tools, with a wealth of trained experts worldwide.

* 10 GbE continues to benefit from emerging Ethernet advancements, such as hardware-based iWARP support (RDMA over TCP/IP) that further optimizes performance and reduce CPU utilization.

[FIGURE 1 OMITTED]

* True QoS functionality enables traffic to be consistently classified, prioritized, and queued at line rate speeds

* 10 GbE coexists natively with currently deployed 1 GbE infrastructures.

Furthermore, 10 GbE iSCSI SANs are not subject to distance limitations like FC-based SANs. Fibre Channel was designed as a campus-wide network, not to exceed 100 miles, and because of its sensitivity to delay; there are many issues with supporting it over WANs. These limitations negatively impact network costs making it very difficult to cost-effectively provide global access to a FC SAN.

Total Cost of Ownership Advantages

Calculating TCO for a SAN requires evaluation of more than just initial hardware costs. All costs including initial capital expenditures, maintenance fees, systems integration and management, ongoing support, physical space, port density, power, cooling, upgrades along with soft costs (e.g., slow network performance and downtime due to overloading) must be considered. There are always costs with deploying SANs, but the real differences are the costs of staffing, training, and managing a FC SAN versus iSCSI. Because Ethernet is already widespread in the data center, the incremental management costs with 10 GbE iSCSI are negligible in comparison. Additionally, the simplification stemming from a converged Ethernet network results in savings on other hidden costs like downtime, problem tracking, and resource drains. Although these costs can be difficult to quantify, they are very real and can drive up the TCO. In fact, annual recurring management costs can range from 30% to 50% of the initial hardware investments.

Optimized Efficiency and Consolidation Advantages

Since iSCSI can be implemented at the software layer, existing servers can be upgraded with off-the-shelf iSCSI software with ERL 2 support allowing them to become a member of the SAN. Upgraded servers with 10 GbE Neterion Xframe adapters enables a single, high performance, easily managed IP network for data and storage. The virtualization capabilities on an Xframe adapter also ensure that IT managers have the most power to manage bandwidth for best performance.

Another key advantage of 10 GbE iSCSI over FC is the management of bandwidth. Both iSCSI and FC are block level protocols (as opposed to file-level protocols like NFS/CIFS), but unlike FC, where unused bandwidth in a dedicated channel is wasted, complete iSCSI implementations with ERL 2 can optimize bandwidth usage by automatically allocating bandwidth from unused connections.

The potential for consolidation savings are also significant. Instead of an under-utilized five-chassis data center that can have an initial hardware cost between $750,000 and $1,200,000 and annual management costs that can run between $252,000 and $372,000, consolidating to a single chassis configured with a Neterion 10 GbE adapter drops the hardware costs to between $150,000 and $250,000 with management costs to less than $56,500. For global SANs, the cost savings of this consolidation with 10 GbE iSCSI versus FC can be tremendous.

Furthermore, IT managers can still leverage their existing FC investments by bridging from their 10 GbE iSCSI SAN to the FC networks.

Sample Configuration Demonstrating Real-world Performance

The following is an example of a high performance, highly available, scalable iSCSI SAN built with readily available components (Fig. 1). The products used are: Neterion Xframe 10 GbE adapters, a Force 10 TeraScale E-Series switch/router, and the iSCSI Enterprise Software from SBE, Inc., (formerly PyX Technologies) with ERL2. The SAN architecture ensures no single point of failure end-to-end with guaranteed quality of transport supported by multi-path, session continuation, and dual loop back functionality.

This 10 GbE iSCSI SAN illustrates ERL 2 iSCSI performance at 10 Gbits/s (Fig. 2). In this configuration, three ERL 2 iSCSI initiators access one RAM-disk storage target (note, processing more than one target unit running over PCI-X introduces negligible overhead). The results were measured using DiskTest, a standard profiling tool available from the Linux Foundation.

The benchmark transported large files to keep session initiation overhead down to accurately characterize throughput. Transferring sequential 32-kbit-size blocks yielded a throughput of 797 Mbytes/s on reads and 667 Mbytes/s when writing. Compare this to a standard 2 Gbit/s FC SAN that typically transports up to 200 Mbytes/s with a 32-kbit block size. Hence, 10 GbE iSCSI with ERL2 represents to a 4X performance increase over FC.

Summary

In summary, five primary reasons to consider 10 GbE iSCSI with ERL2 GbE for building SANs are:

* Security, reliability, quality of service (QoS) of iSCSI ERL2

* Performance of 10 GbE versus 2 Gb/s FC

* Cost of Ethernet verses Fibre Channel (initial costs and TCO)

* Proven capabilities of Ethernet, such as unlimited distance, diskless server booting capability, etc.

* iSCSI with ERL2 enables robust disaster recovery, backup, and replication to remote sites over existing IP WANs.

With the immediate availability of 10 GbE adapters from the industry leader, Neterion, full featured iSCSI software from SBE, Inc., and the variety of switches equipped with 10 GbE ports, IT managers can easily deploy or extend better and faster SANs.

Leonid Grossman is the vice-president of software engineering at Neterion. He holds an MSEE degree from Moscow Petroleum University, an MS in Computer Science from Moscow University of Physics, and an MBA from San Jose State University.

[e-mail]

Nicholas Bellinger is a chief architect at SBE, Inc. He's a cofounder of PyX Technologies (acquired by SBE, Inc. in 2005) and is the principle inventor of the PyX Storage Engine and iSCSI software.

[e-mail]

Sponsored by Neterion

www.neterion.com

www.pyxtechnologies.com

Leonid Grossman, Neterion, Inc. and Nicholas Bellinger, SBE, Inc.

            Megabyte per second
Block Size  Read   Write

4k          1,777  1,156
16k         4,581  2,920
32k         6,374  5,333
64k         6,460  7,370
256k        6,900  7,500

Figure 2. The chart shows the total traffic, including the headers.

Note: Table made from bar graph.