iSCSI over distance: how to avoid disappointment.

With the latest set of disasters happening in the world, it is quite clear that Business Continuity and Disaster Recovery (BC/DR) are the most important business drivers for deploying wide area networked (WAN) storage. Whether recovering from a total loss from a man-made or natural disaster or simply recovering from an operation mishap, companies of all sizes must plan for the unexpected in order to sustain their businesses.

Over the past few years, IP Storage has gained a significant foothold in the marketplace and plays very well with BC/DR requirements. The days of proprietary channel extension technologies are long gone. Many of these antiquated technologies have been replaced by newer open standards storage technologies that promise significantly lower costs and higher returns on IT investments. These emerging technologies have been led by the Small Computer Systems Interface over the IP network--better known as iSCSI.

iSCSI Hits the Market

One storage technology that has the potential to significantly change storage networking is iSCSI--a storage networking protocol that was developed by the Internet Engineering Task Force (IETF) and subsequently ratified in 2003. iSCSI provides server-to-storage access over IP networks as well as server-to-server connectivity for applications such as replication and mirroring. iSCSI leverages the ability to connect servers and database servers to storage arrays, tape controllers, and archive devices (WORM drives) via Ethernet. Typically, this occurs via Gigabit Ethernet interfaces to Gigabit Ethernet switches. This connectivity has spawned a new term, "The IP SAN." Figure 1 represents a typical network configuration on how iSCSI may be used for many corporations.

[FIGURE 1 OMITTED]

Over the last couple of years, the iSCSI protocol has gained traction in the storage networking marketplace. According to industry analysts, the iSCSI market is expected to expand significantly in the coming years.

There are some very compelling reasons to leverage iSCSI versus other storage networking technologies such as Fibre Channel. This article focuses on some of the technology benefits to end users and some of the inherent challenges that will be faced when extending iSCSI over distance.

iSCSI Technology

The iSCSI protocol is a target/initiator model whereby an iSCSI initiator initiates requests and an iSCSI target receives the requests and delivers the requested data. There are typically iSCSI stacks on both ends of a storage network configuration that encapsulate SCSI block commands into Ethernet packets for transmission over the IP network. This can occur whether the storage is local or remote. Figure 2 represents a typical iSCSI packet.

iSCSI Benefits

There are several distinct benefits to IT end users when using the iSCSI technology. First and foremost, iSCSI allows companies to share storage resources across servers without enduring the high cost and complexity associated with Fibre Channel SANs. Many medium-sized businesses have delayed the implementation of IP SANs because of the complexity and expense of Fibre Channel SAN technology solutions. The iSCSI protocol now lets them benefit from the use of IP SANs with much lower CAPEX and OPEX investments.

Secondly, iSCSI runs over the Ethernet media, which is widely used in almost every IT shop around the world. Specialized 'dedicated' networks and resources to manage those networks are no longer required which also provides significant cost savings.

iSCSI provides a cost-effective opportunity to deploy storage over IP networks while still leveraging their existing IP infrastructure, data management systems and other network investments. iSCSI provides significant opportunities for small to medium businesses (SMB) as well as some enterprise shops to finally incorporate IP SANs that were not possible before.

iSCSI Uses the TCP Transport

Like many storage over IP technologies, iSCSI uses TCP for moving data between application endpoints. TCP is by far the most commonly used transport technology for replication data. However many vendors and IT end users are finding out that TCP has some basic limitations that cause many storage applications to perform poorly, especially over distance. These limitations apply to iSCSI and any other transport technology that uses TCP.

TCP: Window Scaling

Window scaling is the amount of data that can be outstanding (over the network) at any given point in time by the transport software. Window scaling is often implemented differently depending upon the platform and operating system used. In some cases, window scaling is used by default and in other cases it is not. If scaling is not used by the iSCSI driver there may be large amounts of unused bandwidth that can result in poor performance for the application.

TCP: Slow Start

TCP data transfers typically start slowly and then ramp-up to their maximum transfer rate over time. As a result, applications with short sessions often have performance issues. Slow Start is used by TCP to avoid congestion due to assumptions that large numbers of transfers might be competing for the same bandwidth.

[FIGURE 2 OMITTED]

There are also other factors that can impact the performance ramp-up time for TCP. Networks with smaller Round Trip Time (RTT) calculations ramp faster than those with larger (RTT) due to RFC algorithms. Smaller RTTs grow their congestion windows more quickly because the growth rate is tied to the RTT. This can result in connections with larger RTTs not getting their fair share of the bandwidth if shared with smaller RTT connections.

TCP: Packet Loss

Packet loss describes an error condition that occurs on a network when data packets appear to be transmitted correctly at one end of a connection, but never arrive at the other side. Packet loss occurs when packets are damaged and discarded, or when the capacity of an intermediate network component is exceeded, which results in packets being discarded. Packets can be damaged as they move across a wide area network (WAN), or as they traverse network components such as routers and switches.

Packet loss can put TCP into Slow Start and congestion avoidance modes, which can severely reduce performance of the TCP connection.

Storage replication applications are mission critical, and involve sending large amounts of data over long periods of time. The chance of a replication running during a degraded network timeframe is probably quite high, since replication applications tend to use long standing connections, or by connections that reoccur at predefined time intervals. In either case, storage replication applications use TCP connections for long periods of time, either as one long-standing connection, or the sum of multiple shorter connections.

Packet loss has a more significant impact on high-bandwidth networks, even over short distances. High-bandwidth networks require large windows to fully utilize the network capacity. However, the Slow Start and congestion avoidance algorithms may prevent large windows from being achieved when packet loss occurs, due to the continuous reduction of the window size. Some storage replication methodologies attempt to mitigate this problem by maintaining multiple TCP connections. However, if there is congestion occurring somewhere in the network, driving multiple TCP data streams only adds to the congestion.

TCP: Error Recovery

During error recovery, TCP may cause the entire data stream from lost portions to be retransmitted in its entirety. High bit-error-rates or packet loss may cause large amounts of data to be resent over the WAN even though most of the data was already successfully received. This is a waste of bandwidth, especially considering the increased latency time of resending the data and the performance penalty issues

of Slow Start. iSCSI implementations use selective acknowledgement (SACK) for improving error recovery; however, this TCP enhancement may still lack efficiencies required for these data intensive storage applications.

iSCSI Reality

To maximize the performance of iSCSI and to avoid disappointment, several TCP WAN issues need to be mitigated.

HyperIP, from NetEx, is a storage networking appliance that enhances the performance of TCP transfers (including iSCSI) over high bandwidth networks. HyperIP provides a highly efficient data transport that mitigates latency, and is tolerant of packet loss and network congestion. HyperIP is the only application accelerator that is focused on improving storage network applications efficiencies in the market today.

Robert MacIntyre is vice president of Business Development & Marketing at NetEx Software (Maple Grove, MN)

www.netex.com