Is The Network Broken?

Businesses find fibre channel SANs too complex, expensive, and immature

The requirement for Internet storage has exploded over the past four years to many thousands of terabytes. Businesses demand ubiquitous network storage that supports multiple platforms--and many of the largest businesses are getting it through fibre channel storage area networks (FCSANs). FC-SANs have traditionally been cited as ideal for companies with a large investment in data that want to make sure raw data blocks are readily available where necessary. They offer high performance and the ability to back up data offline. Proponents of FC-SANs say that nothing is wrong: FC-SANs provide comprehensive access to block data storage at speeds that are superior to anything else currently available. But even a superficial look at the market suggests that something is seriously wrong with FC-SANs.

Surveys Find Problems With FC-SANs

An August 7, 2000 Computerworld survey of businesses using FC-SANs found 38 percent complained of high implementation costs, 36 percent of lack of staff and resources with expertise in fibre channel, and 30 percent of the immaturity of the technology. Lack of standards, difficulty in building a business case in support of FC-SANs, lack of product interoperability, and inexperienced integrators were also cited as problems by a significant percentage of respondents.

Implementing an FC-SAN is very expensive. Even a quick look at basic network components bears this out: an FC Host Adapter/NIC costs $629-$969, compared to a GbE version, which retails for $117-$307. A 50-foot cable for FC costs $138; for GbE, it costs $11. And network components are a relatively minor part of the total expense. Staffing is more significant, and the tremendous difficulty of finding staff with expertise in fibre channel makes them more expensive. A Piper Jaffray report from the April/July 2000 issue of eWeek/Information Week finds that only 11 percent of companies have deployed FC-SANs. On the other hand, Ethernet networks are everywhere, resulting in a much greater pool of IT professionals who have had experience of such networks.

Compared with Ethernet, FC-SAN technology is immature and vendors have been slow to set and adopt true standards. As a result, once companies have chosen an FC-SAN vendor, they are essentially stuck with their choice. To switch to another vendor would mean writing off virtually all of their existing: investment in the FC-SAN, because each FC-SAN vendor has had to invent its own way of solving the technological challenges of networking.

E-SANs Offer Standards-Based Options

While most Fortune 500 businesses have some FC-SANs, smaller businesses, newer companies, and startups do not have the deep pockets or the desire to invest in FC-SANs. They are deterred by the expense, complexity, and the difficulty of finding IT staff who have expertise in FC. Businesses do have other options: Network attached storage (NAS) or gigabit Ethernet (GbE) E-SANs.

These Ethernet-based storage systems run on TCP/IP, the Internet standard, which has already solved the challenges of networking in a standards-based way. Some IT professionals believe that FC will never catch up to Ethernet-based solutions because its nonstandard networking creates a huge technological gap that no committee can close. Even as FC networking standards solidify and improve, Ethernet standards will continue to be refined, maintaining Ethernet's technological superiority.

For most businesses, it is much easier to build on their existing Ethernet-based storage solution than begin from scratch with a proprietary and expensive system. They benefit from the fact that IT professionals already know how to work with Ethernet and that virtually all businesses and many homes are already wired for Ethernet. They also benefit from the built-in advantages of TCP/IP: it automatically retries when it fails to make a connection; it correctly orders the blocks of information it receives; and it provides automatic failover to other pathways.

The Appeal Of The Incremental Approach

Every business already has an Ethernet-based LAN, and the idea of incrementally enhancing these existing resources has universal appeal. GbE provides new levels of bandwidth without the problems that have plagued FCSANs. IT staff are experienced in Ethernet, current systems interoperate with it, and with the advent of gigabit Ethernet (GbE) there is no longer a significant performance difference between Ethernet and FC. In fact, with the pending arrival of 10 CbE, slated for 2001, Ethernet should be a substantially faster option than fibre.

Even more important is the virtually limitless scalability of GbE. Companies simply add switches and storage nodes to boost performance and capacity dramatically, without changing anything else. This makes GbE a perfect replacement for direct attached storage (DAS) systems as well as for FC-SANs.

Choosing Between Ethernet-Based Solutions

NAS--a remote file server rather than a SAN--is another Ethernet-based way for companies to allow people to share information. However, the system is slower than a direct block storage connection, and only transmits files, not raw data blocks. This creates a problem for most databases and complex applications.

GbE-SANs do not have this limitation. They leverage the performance of GbE to offer the same advantages as fibre channel, but since they run on Ethernet they don't require new infrastructure or changes to applications. GbE-SANs are easy to install and scale, and provide redundancy for reliability. E-SANs appeal to customers who want a SAN without the cost. With external SCSI limited to a 50-meter range, storage used to be restricted to use within the same buildings, and usually the same room, as the server. E-SANs enable companies to package storage commands for transport over TCP/IP, eliminating the restriction on distance between storage and client computers--and helping to create the potential for a surge in GbE implementation.

How GbE Works

GbE is Ethernet that provides speeds of one billion bits per second. It preserves all the features of 10/100 Mbit Ethernet (Fast Ethernet), so all applications run unchanged, but faster. An important consideration is that GbE is switched Ethernet. Previous generations of Ethernet were vulnerable to data stream collisions that could reduce performance by as much as 30 percent. Switched Ethernet switches traffic automatically so that there are no collisions.

GbE is easy to install: IT staff can literally plug a storage box in, and grant access to all the storage on the system. Scalability is easy, too: the network can be upgraded simply by connecting switches to switches because TCP/IP will automatically load-share if there are multiple connections to the box.

A system could use all three storage methods, because fibre and GbE can interface through a "bridge," and a NAS application server can use any external storage, including either FC-SAN or GbE-SAN.

The Future Of GbE

Industry-wide, there's a priority on maximizing the efficiency of network storage over Ethernet. Proprietary GbE storage protocols are available now that encapsulate storage commands for transmission over Ethernet, enabling storage sharing at near-line speeds so that businesses can use Ethernet as a storage network infrastructure. These solutions provide applications the same view of storage as when data is stored on a PC's local disk, except the data and storage commands are put in TCP packets, sent across the network, and then sent to the storage controller (See Figure).

However, while proprietary solutions have some technological advantages, the success of TCP/IP and the slow adoption of FC dramatically underscores the fact that industries as a whole benefit from standards. Within the Internet engineering task force (IETF), a working group is in the process of defining and implementing iSCSI, with the goal of creating working implementations within six months. Groups are also working on a direct attached file system (DAFS) to address the limitations of NAS, but any practical solution is still some distance in the future.

A fully implemented iSCSI protocol will facilitate interoperability and remove philosophical objections to transferring storage onto GbE. Even without a final iSCSI standard however, GbE offers a cost-effective, implementation-friendly solution. It may soon be even more attractive with the upcoming availability of 10 GbE, which will deliver a tenfold increase in speed, starting with high-speed trunks between switches.

Dr. Robert Horst is the vice president of research at 3ware, Inc. (Mountain View, GA).