Multiprotocol Fibre Channel: The Foundation For Server And Storage Data Networks.

This article is the first in a two-part series. The second part will appear in the October issue of CTR.

The architects of the Fibre Channel standards defined a multilayered architecture for moving data across a network. The current implementation of Fibre Channel uses SCSI protocols to move data across storage networks. Popularized by the term Storage Area Network (SAN), this approach has received wide acclaim as a solution for addressing the limitations of bus-attached storage.

Although Fibre Channel has been very successful as the foundation for SAN, the Fibre Channel standards were designed to address applications beyond the realm of storage networks alone. This article introduces the operation of two server-to-server protocols-Internet Protocol (IP) and Virtual Interface (VI)- over a Fibre Channel network.

This article also describes the practical use of SCSI, IP, and VI protocols across an integrated, switched Fibre Channel fabric in a data mining or business intelligence solution, demonstrated by Hitachi Data Systems at Comdex in November 1999. This groundbreaking solution promises to shape the future of data-centric system area networks, the unification of storage, and server area networks. It brings the promise of the always available data utility a step closer to fruition, building on the foundation of Hitachi Freedom Data Networks.

Fibre Channel Architecture

One advantage of a multilayered architecture like Fibre Channel is that new technologies can be introduced at discrete layers without changing other layers of the architecture. This makes Fibre Channel an extremely flexible architecture, able to rapidly incorporate new technologies and respond to changing business requirements without compromising the user's investment in infrastructure. As shown in the Fig, there are five layers, numbered from FC-O to FC-4.

FC-O is the physical layer. It establishes the standards for different media types, allowable distances, and signaling. It also defines the optical and copper interfaces and the cable plant. FC1 defines the standards for encoding and decoding data for shipment over the media. FC-2 defines the Framing, Flow Control, and Service Class. FC-3 defines Common Services, still being established for facilities like data encryption and compression. FC-4 is the Protocol Mapping Layer. It defines the interfaces between Fibre Channel and upper-level applications like the Serial SCSI protocol (SCSI-FCP). The driver in the Host Bus Adapter (HBA) provides the interface function of FC-4. FC-4 supports multiple protocols, including SCSI-FCP, FC-IP, and the recently defined FC-VI. FCVI is VI Architecture implemented over Fibre Channel, which allows low-latency movement of data between memory locations in separate Fibre Channel nodes.

Fibre Channel Protocols

* SCSI-FCP. SCSI-FCP is a serial SCSI protocol that maps Fibre Channel devices to logical drives that are accessible to the operating system. This protocol enables SCSI-based applications to use Fibre Channel without modification. SCSI-FCP is the primary means of communications between servers and storage subsystems. The SCSI Extended Copy command, a new ANSI T10 standard, also supports data movement directly between storage subsystems through a data mover appliance on the SAN.

The advantages of SCSI-FCP over bus-attached SCSI have been well documented for Storage Area Networks. SCSI-FCP offers higher performance (100MB/sec), longer cabling distances (up to 10km per link), and a larger addressing space (up to 16 million nodes). Instead of block transfers, SCSI-FCP uses frame transfers. Frame transfers support intermixing short transaction data transfers with larger data-streaming transfers, improving service quality. SCSI-FCP also allows network configurations that support pooling of storage for simplified management and resource savings and it supports encoding schemes that improve reliability and availability.

* FC-IP. FC-IP is a protocol that maps Fibre Channel addresses to IP addresses. FC-IP establishes addressing by broadcasting an IP address and receiving a MAC address from the target node. This IP broadcast can disrupt a SCSI device if it cannot distinguish FC-IP frames from SCSI-FCP frames. Some subsystems can distinguish between FC-IP and SCSI-FCP frames by checking the frame header. Storage subsystems that do not have this capability must use other means (like switch zoning) to prevent FC-IP frames from being broadcast to Fibre ports on storage devices.

FC-IP has several advantages when compared to IP over Ethernet. It can save costs by using the same interconnect infrastructure as SCSI-FCP storage. Transfer speeds are much faster over 100MB/sec Fibre Channel than over 10BaseT, 100BaseT, or even Gigabit Ethernet and the transfer of data over Fibre Channel can be more efficient than Ethernet.

Ethernet transfers packets with data payloads up to 1500 bytes. The packet is the basic unit of recovery in Ethernet, necessitating an interrupt that consumes CPU cycles after every frame. This overhead is usually the limiting factor in Gigabit Ethernet, preventing full bandwidth utilization. FC-IP uses Fibre Channel frames with data payloads up to 2000 bytes. The basic unit of recovery for FC-IP is a sequence, which is made up of multiple frames. A Maximum Transfer Unit (MTU) can be defined for a sequence of up to 64 frames, allowing Fibre Channel to transfer much more data between host interrupts than Ethernet. The MTU can reduce the number of CPU cycles required and make the data transfer more efficient.

FC-IP also has the advantage of using Fibre Channel networks, which are closed networks based on flow control using buffer credits. The nature of Ethernet requires the assumption of a public network without flow control. Ethernet pumps out packets until there is a collision, then backs off and resends the packets after a brief time delay, consuming additional CPU cycles. IP applications run on FC-IP without modification, enjoying Fibre Channel transfer speeds and greatly reduced processing interrupts.

Tracy Edmonds and his team from Troika Networks, along with Patrick Wong from Finisar, assisted with this article.

John Nguyen is a Data Network architect, Pierre Raymond is the director of Data Network architecture, and Hu Voshida is the vice president of Data Network Solutions at Hitachi Data Systems (Santa Clara, CA).