A new breed: IP SANs show great promise for networked storage. (Storage Networking).
There is mounting evidence that IP-based storage area networks (SANs) have what it takes to become a viable alternative, as well as a complement, to Fibre Channel-based SANs.
For one thing, it's hard to discount the value of only having to support one networking technology for both the LAN and the SAN, rather than the additional, specialized IT headcount required for supporting Fibre Channel for the SAN and Ethernet for the LAN. So, while the business case is clear, some important technical issues are still under discussion.
For example, among the various approaches for IP storage that have been discussed by the IETF are FCIP (Fibre Channel over Internet protocol) which entails tunneling Fibre Channel traffic over TCP/IP networks, as well as iSCSI, which involves encapsulating SCSI over TCP/IP networks. The advantage of the iSCSI approach is that it leverages the ubiquity of mature technologies-SCSI and IP-and also creates a seamless link between an Ethernet LAN and a SAN in the same enterprise.
If IP storage is to make good on its potential, it has to meet very particular performance, flexibility, scalability, and data integrity requirements. And as the makers of storage routers, subsystems and host bus adapters scramble to design and build products to serve the burgeoning IP storage market, they must determine what firmware and silicon products are suited to meet the unique requirements of this new approach to building SANs.
A key to satisfying those unique performance, flexibility, scalability and data integrity requirements comes in the form of a new kind of processor--a storage network access processor--that is crafted precisely to satisfy those requirements.
On the performance front, an optimal IP SAN must have the ability to keep pace with network speeds for all kinds of storage traffic. At present, network speeds run at 1Gbps and will likely move beyond 10Gbps. Another performance-centric issue comes in the form of making sure to provide for maximum offload of host processors, for example a server CPU. After all, any network protocol processing that consumes CPU cycles interferes with application processing, the primary job of the CPU. In an optimal IP SAN, the proper offloading of a host processor is achieved when a storage network access processor (SNAP) autonomously handles the complete TCP/IP traffic load-including fast path and exceptions--and ULPs such as iSCSI. A SNAP also provides for minimum interrupts to the host processor and direct data placement into host buffers.
In terms of flexibility, SNAPs need to be adaptable to continuously evolving network protocols. But beyond that, the design of the SNAP should give storage networking QEMs the luxury of being able to ignore connectivity issues (because they've already been addressed) and to focus their efforts on adding value through storage services. In this context, connectivity refers to all of the issues associated with connecting devices to one another.
When it comes to scalability, the SNAP should be designed in such a way that it can keep up with line speeds of 1Gbps and can deal with an online transaction processing (OLTP) profile and the smaller block sizes that such a profile implies. Beyond that, this new kind of processor must be capable of supporting a large number of simultaneous TCP/IP connections. For example, in a typical SAN a component like a host bus adapter (HBA) may only need to support 1,000 to 2,000 connections but other parts of that same SAN may need to support tens of thousands of connections. A well-designed SNAP should not place a limit on the number of connections.
Data integrity is the final key measure of a premier IP SAN implementation. Since traditional network processors can rely on higher protocol layers to handle any data integrity problems, such as memory soft errors, they don't need to implement data-integrity mechanisms themselves. But a SNAP operates not only on the network protocol layers but also on the upper layer protocol. Therefore, it cannot depend solely on the network layer to provide data robustness. As a result, the SNAP needs to implement comprehensive data protection mechanisms.
While IP SANs have very specific requirements, it's important to note that a variety of earlier-generation processors--namely general-purpose processors, network processor units (NPUs), and TCP/IP offload engines (TOEs)--have been used for protocol processing up to this point. It's worthwhile to look at these three approaches to protocol processing.
When looking at the use of general-purpose processors for protocol processing, it's important to delineate between iSCSI being run in software on a server, and an "iSCSI HBA" that is designed with general-purpose processors. In the first case, the iSCSI in software is stealing server resources from other applications as it consumes CPU, increases peripheral bus contention, and performs expensive memory context switches to alternate between application processing and iSCSI protocol processing. All this contention for server resources generally results in high CPU consumption and poor latency when any significant amount of IP storage traffic is processsed.
In the second case the "iSCSI HBA" is an adapter that offers server CPU offload, but general-purpose processors have trouble dealing with block sizes of 4KB or smaller, which are common in OLTP traffic profiles. They lack this capability primarily because of their memory management schemes, context switching overhead and the lack of special function units that are necessary for protocol processing. In fairness, it's important to stress that general-purpose processors were designed for application-related computing. They weren't tailored for input and output (I/O) tasks.
NPUs are also used for protocol processing but they were really designed to facilitate intelligent routing decisions in switches and routers. They are unable to fulfill two critical needs of an IP SAN. Since NPUs only process packets, they are unaware of upper layer protocol data units (PDUs) that run on top of the packet stream. To be able to process storage traffic effectively and make storage-level decisions the processor must have built-in PDU awareness. And PDU awareness is also crucial in properly performing direct data placement. What's more, NPUs process packets without connection context. As a result, NPUs are unaware of all the issues associated with TCP/IP connections, e.g. dropped packets, out of order packets, protocol time-outs, etc. And most NPUs are unaware of connection set-up, tear-down, and packet acknowledgement. A processor in an IP-networked storage environment must be able to fully terminate the protocol, hence it must be aware of all these events and must be able to manage them.
The TCP/IP offload engine (TOE) comes the closest to being a pure protocol processor and has as its core purpose, the offloading of TCP/IP protocols from the host processor to the hardware on the adapter or in the system. A TOE may be embedded in a network interface card (NIC). These kinds of processors work pretty well until block sizes decrease to 2KB to 4KB (a common block size in online transaction processing). When it comes to dealing with an OLTP profile, however, performance degradation is substantial.
Certainly, general-purpose processors, NPUs, and TOEs are well-suited to perform the tasks for which they were designed, but none of them really meet the design criteria for squeezing the maximum out of an IP SAN.
Rules of the Game Have Changed
What is needed is an architecture that meets these very specific requirements.
Unlike the earlier-generation processing technologies that have been used up to this point, an efficient SNAP design is optimized for processing TCP/IP and ULPs like iSCSI, Remote Direct Memory Access (RDMA). The architecture also assists file systems like NFS and the Common Internet File System (CIFS). With complete TCP/IP offload, even for dropped and out of order packets, and Upper Layer Protocol awareness, the SNAP is able to perform tasks such as Direct Data Placement to achieve excellent CPU Offload and low latency--even for OLTP traffic profiles.
While a mapping of protocol processing to silicon is required to get great performance, it is also important to leave the TCP/IP control plane and ULPs running on standard microprocessor cores in order to have flexibility in tracking protocol changes or to add new functionality. Hardware accelerators are an excellent way to handle fixed-nature tasks such as data integrity and statistics collection, while the nano-code engines deal with functions like packet parsing and synthesis. Another key element in an ideal SNAP architecture is a memory manager that eliminates the need to move data around in system memory, which historically has been a serious performance bottleneck. Each layer of the software can see the data in memory in the most efficient way without having to copy the data.
The resulting ideal SNAP architecture is a microprocessor core cluster that is free of any interrupts, memory access bottlenecks or context switching, meaning that it can use all of its compute power.
Clearly, IP SANs have considerable potential and as more and more IP-storage-specific technologies, like iSNAP, come to fore, that potential will be unleashed.
Mike Strickland is director of product marketing at Silverback Systems (Campbell, Calif.)
|Printer friendly Cite/link Email Feedback|
|Publication:||Computer Technology Review|
|Date:||Oct 1, 2002|
|Previous Article:||DVD gut check: where does the storage industry stand? (Storage Networking).|
|Next Article:||Zone security considerations for SANs: overcoming inadvertent overwrites. (Storage Networking).|