QUALITY OF CONNECTION FOR THE ENTERPRISE SAN.

Considering the number of Fibre Channel topologies and competing interconnect devices available in the market today, planning a Fibre Channel Storage Area Network (SAN) can be confusing and complex. However, the ultimate choice of topology, architecture, and type of backbone device has a definitive impact on the ability of a SAN to deliver expected results, especially if it is to serve as the infrastructure for major enterprise applications like e-commerce and e-business. This article discusses an approach to designing such an "enterprise SAN" based on the concept of SAN Quality of Connection (QoC) as defined by Strategic Research Corporation.

SAN QoC encompasses three metrics that characterize how well a SAN will service enterprise applications. These metrics are availability of connection, performance degradation, and bandwidth scalability. Availability of connection alone is inadequate, as it describes only the presence or absence of a service. Quality of Connection is a complete characterization that describes how a service will operate under normal and adverse failure conditions and how effectively it will scale bandwidth with connectivity.

Included in this article is a SAN Quality of Connection matrix composed of five different QoC classes. One need only define the SAN QoC class required for a particular application and this matrix, will identify the topology, architecture, and type of backbone interconnect device required to achieve the level of service desired for that application.

INTRODUCTION

The Storage Area Network is the latest breakthrough in the way Unix and NT computing and storage resources are deployed throughout a business. Using high-speed storage networking technology such as Fibre Channel, a SAN allows servers, storage, and tape devices to be interconnected in a synergistic way that was not previously possible. By interconnecting these resources with a SAN, they can be consolidated and shared, leading to improved data accessibility and protection, more efficient data management, higher quality of service, and lower cost of ownership.

Until now, open systems platforms have typically been dispersed throughout an enterprise as individual islands of operation incapable of sharing resources. As a result, potentially valuable business data has not been accessible throughout the enterprise. With a Fibre Channel SAN, businesses can now interconnect these distributed islands of operation, combining them into larger and more efficient continents of operation with the resulting advantages of expanded data accessibility economies of scale, and true scalability for the Internet economy. Expanded data accessibility can multiply the value of data many times over by making it available wherever it's needed for enhanced decision support and, ultimately, competitive advantage.

SANs enable the vital new e-business and e-commerce applications that are so crucial to the ongoing success of today's preeminent businesses. By leveraging the increased speed and connectivity offered by Fibre Channel over its SCSI predecessor, enterprise SANs deliver the computing infrastructure for the new millennium and the Internet revolution. With an enterprise SAN, data can be consolidated and shared as needed within the company and with suppliers, partners, and customers in the data value chain.

FIBRE CHANNEL TOPOLOGIES

SANs can be implemented with three different topologies: point-to-point connections, arbitrated loops, and switched fabrics. Point-to-point connections are similar to SCSI implementations, but they offer greater bandwidth over much greater distances as a result of Fibre Channel technology (see Table 1 for a comparison of SCSI and Fibre Channel technologies).

An arbitrated loop is a single, shared path that requires arbitration before data can be transferred. All connected devices share the 100MB/sec bandwidth of the loop and the loop is limited in connectivity to a total of 126 nodes. Most arbitrated loop SANs have been implemented with no more than two host servers due to the fault isolation issues inherent with arbitrated loops. The performance, distance, and scalability limitations of a loop make this topology too restrictive for all but the smallest of SANs.

Today's enterprise SANs are based on Fibre Channel switched fabrics. By using a switched fabric instead of a shared loop, an enterprise SAN scales linearly in throughput performance and connectivity as devices are added to the fabric, since each device has its own dedicated path and data transfers occur in parallel. Data is dynamically routed throughout the fabric using a switching interconnect device.

Bandwidth scalability is determined by the number of available ports that can be connected to the switching device's backplane without creating performance bottlenecks. By using inter-switch links, a switched fabric can be expanded to connect hundreds or thousands of nodes. However, inter-switch links have the potential to create bandwidth bottlenecks in the enterprise SAN backbone (more on this later).

FIBRE CHANNEL INTERCONNECT DEVICES

No interconnect device is required for a Fibre Channel point-to-point connection, which is implemented by connecting a Fibre Channel Host Bus Adapter (HBA) in a server to a Fibre Channel storage adapter using a fiber optic cable. Standard multimode and single mode cables are supported (see Table 2 for distance considerations).

Although Fibre Channel can be implemented using copper wires, copper is not practical for enterprise SANs because of severe distance restrictions. Fibre Channel adapters designed for copper connection can be easily converted to support fiber optic cables with media interface adapters.

* SAN Gateways

Most tape products do not yet offer native Fibre Channel adapters. However, they can still be connected to an enterprise SAN using a SCSI-to-Fibre Channel bridge, sometimes called a router or SAN gateway.

SAN gateways can be used to connect other SCSI devices to an enterprise SAN, as well, including RAID arrays and servers equipped with SCSI HBAs. For example, McDATA recently announced the ability to connect HPUX servers to a switched fabric enterprise SAN using a SAN gateway running in host mode. SAN gateways provide investment protection for installed SCSI devices as enterprise SANs are deployed to achieve competitive business advantage.

* Hubs and Switched Hubs

SANs based on an arbitrated loop topology are typically implemented using a Fibre Channel hub. A hub provides centralized connectivity plus an automatic shunting capability that enables devices to be added to the loop on the fly. Without a hub, the loop must be taken out of service when devices are added or removed.

If multiple loops are desired, a switched hub provides centralized connection points; however, it does not provide simultaneous any-to-any connectivity between all attached devices since data transfers are still restricted to their respective loops. The switching capability of one of these devices requires manual configuration changes.

* Fabric Switches and Directors

Dynamic switching is provided only with a fabric switch or a Director, such as the ED-5000 Enterprise Director manufactured by McDATA Corporation. These interconnect devices are designed to provide any-to-any connectivity between all attached devices plus linear throughput scalability. Each port of a switch or Director provides 100MB/sec of bandwidth and full-duplex operation.

The major differences between a fabric switch and a McDATA Director are summarized in Table 3. Notice that a Director is designed to eliminate single points of failure and maximize bandwidth scalability by offering twice as many ports per backplane as a fabric switch. The fault tolerant features and bandwidth scalability of a Director make it ideal for enterprise SAN backbones.

* Interconnecting Switches and Directors

As long as they are manufactured by the same vendor, fabric switches and Directors can be interconnected today using inter-switch links to increase the number of switched connections in a single fabric. To facilitate heterogeneous switch-to-switch and switch-to-Director interoperability, all of the major Fibre Channel switch manufacturers have recently formed the Open Standards Fabric Initiative (OSFI). Spearheaded by McDATA, the OSFI is expected to present its standards work to the ANSIT-11 committee by year-end.

In general, however, inter-switch links should be avoided in an enterprise SAN backbone since they do not scale bandwidth together with connectivity. Unrestricted scaling of both connectivity and bandwidth is only available to ports connected to the same backplane in a switch or Director, since inter-switch links restrict the bandwidth available between ports in different switching devices. Interconnecting switches or Directors is done for connectivity reasons, not bandwidth.

From an application point of view, SAN connectivity requires only that the server(s) used for the application be given access to needed storage with adequate bandwidth. Connection to other servers and storage devices not used by the application is not required. This normally limits the connectivity requirements for a specific application to a fairly small fabric (typically 32 ports or less), since an enterprise SAN is optimally configured with a single, consolidated enterprise storage pool.

Inter-switch links are useful for connecting peripheral interconnect devices to the enterprise SAN back-bone. They may also prove useful in the future for server-to-server communication required for clustering. Today, however, server-to-server communication is better handled with a LAN than a SAN, since running IP over Fibre Channel is not mainstream.

EPL Switches

Fabric switches can be made to function like switched hubs using Emulated Private Loop (EPL) code rather than switched fabric code. Hewlett-Packard's 16-port switch is an example of this type of implementation. It is important to understand that EPL functionality is not equivalent to standard fabric switching.

CONFIGURING ENTERPRISE SANS FOR HIGH AVAILABILITY OF CONNECTION

One of the fundamental requirements for an enterprise SAN is continuous operation. In the Internet economy, 24x7 availability is no longer an option--you are either online (i.e., open for business) or not when it comes to e-commerce and e-business. The data value chain for a company engaged in e-commerce extends literally around the globe, which means a significant number of transactions occur outside of local business hours. Enterprise SANs must, therefore, be consciously designed and implemented with continuous operation squarely in view as a fundamental requirement. Otherwise, it will not be achieved. Continuous operation is achieved only with high-availability components and high-availability architectures.

* SAN Points of Failure

Implementing a highly available enterprise SAN requires that potential points of failure be eliminated wherever possible and that failures be repaired quickly if availability of connection is lost. Unfortunately, failure points exist at every level in the enterprise SAN infrastructure. HBAs, storage adapters, fiber optic cables, interconnect devices, and path failover software are all potential points of failure.

Hardware points of failure are eliminated through redundancy and failover. This is accomplished by deploying hardware components in redundant pairs and configuring redundant paths.

Redundant paths should be routed through different interconnect devices for maximum availability of connection. Path failover software running in host servers is required to initiate failover from a primary path to its alternate when a path failure develops. A path failure can occur anywhere between a path's HBA and its storage adapter (inclusive).

This hardware solution does not eliminate path failover software as a potential point of failure. If path failover malfunctions when a path failure occurs, neither the primary path nor its alternate is available, which means the application fails because it can no longer perform I/O. In this situation, the user has no recourse but to repair the path failure as soon as possible in order to allow the application to be restarted. This requires either repairing or replacing the failed device.

* Path Failover Malfunction

How likely is path failover malfunction to occur? The answer depends on many things. Path failover software undergoes extensive testing by the vendor to demonstrate recovery from artificially induced failure conditions specified in a test plan. It is unlikely that testing in a lab can cover all the potential failure scenarios that could occur in a user environment. Also, vendor testing is often outsourced to different test labs that specialize in specific operating environments, so test plans may not be consistent across server platforms. Furthermore, since this type of software interfaces directly with operating systems and HBA drivers, it must be constantly updated and re-tested to stay current with new driver and operating system releases and fixes.

The availability of current support usually lags availability of new operating system and driver releases by several months because storage vendors or independent software providers, rather than operating system and HBA vendors, often provide path failover software. For users who like to stay current with operating system releases, this means testing of path failover software may unknowingly take place onsite rather than in the vendor's lab.

Keeping path failover software at current maintenance levels is a critical management challenge for IT shops. With all the software and coordination involved, the chance for mismatch of versions and fixes is high. Users must test frequently to ensure that path failover works properly before it is actually needed. Not all IT shops are prepared to enforce this level of change management policy and discipline on a consistent basis.

Path failover has been a component of high-availability storage solutions for a decade. Prior to the market introduction of switched SANs, the impact of a path failover malfunction was typically limited to a single server or a small, homogeneous cluster. Path failover requirements deserve closer scrutiny in a switched SAN environment, however, because a large number of heterogeneous servers are potentially affected by a switch failure. For example, a 16-port fabric switch experiencing a motherboard failure could trigger path failover on twelve or more servers. As the frequency of path failover invocation increases, so does the probability of a path failover malfunction.

* Guarding Against Path Failover Malfunction

The best way to guard against malfunctioning path failover is to prevent path failures in the first place. This can only be done by using equipment with fault tolerant features in the enterprise SAN such as the McDATA ED-5000 Director.

ED-5000 port cards can also be hot swapped for fast repair. This is important because the ability to quickly repair port failures may be the only recourse one has to recover from path failover malfunction.

The ED-5000 was designed for continuous operation. All critical components (except port cards) are redundant, will automatically failover, and can be repaired concurrent with switching operations by hot swapping. Failure of one of these components will not trigger path failover. Proactive replacement is initiated via automatic phone-home to the service center.

Some fabric switches such as the McDATA ES-2500 and the Brocade Silkworm 2000 series provide redundant and hot-swappable power and cooling. While these features help to reduce the frequency of triggering path failover, they cannot deliver the same level of connection availability as a Director, since the entire switch must be taken out of service to repair other failures. The window required to repair or replace a switch is substantial and results in significant application downtime in the event of path failover malfunction.

* Selecting Path Failover Software

It is critical to identify path failover software early on when designing an enterprise SAN. Path failover software is typically certified to support only the vendor's storage. VERITAS DMP is an exception that supports multiple brands of storage, but even its support is limited to a short list. Path failover software may or may not be available, depending on the type of storage used for the enterprise SAN storage pool.

If redundant paths are required for tape devices, the selection of path failover software is even more limited. Also, if multiple storage devices are used, different path failover products may be required, making change management even more complex and difficult. For all these reasons, it only makes sense to minimize one's dependency on path failover by implementing interconnect devices that have fault-tolerant features.

CONFIGURING ENTERPRISE SANS FOR HIGH PERFORMANCE

Another fundamental requirement for an enterprise SAN is continuous high performance. Internet users are fickle, willing to jump to a competitor's web site just as readily for a performance problem as for being offline. Maintaining customer loyalty is one of the biggest challenges of e-commerce; likewise, keeping partners, suppliers, and customers happy with Internet response time is a major challenge of e-business. Enterprise SAN performance can have a major impact on both.

The I/O latency, measured in microseconds, incurred by a properly configured enterprise SAN is not noticeable to Internet users or typical enterprise applications. However, enterprise SAN performance may be degraded if the number of paths available in the SAN infrastructure is substantially reduced.

This is particularly true if I/O load balancing is employed to drive I/Os. Load balancing takes advantage of multiple paths to achieve higher performance. SAN performance can degrade significantly if any paths used for load balancing are lost. For example, if a load-balanced e-commerce server is connected to a SAN with dual paths, loss of one path reduces by 50% the bandwidth performance resource of that server.

SAN paths are taken out of service when failures occur in the paths themselves or in the interconnect devices to which they are attached. While path failover software can be employed to help maintain availability of connection when this happens, path failover software cannot prevent the loss of paths from impacting enterprise SAN performance. The only way to protect against enterprise SAN performance degradation due to path loss is to utilize interconnect devices with fault tolerant features and granular port packaging. For example, a fabric switch equipped with redundant, hot-swappable power supplies will not degrade performance if a power supply fails, since it may be replaced while all the ports continue to operate. However, if a failure occurs on the motherboard, the entire switch must be removed from service for repair. All 16 ports are rendered unavailable until the switch is either replaced or repaired and re-installed. Even if the failure is isolated to a single port circuit, the entire switch must eventually be taken out of service for repair meaning all 16 ports will be offline for the duration.

A Director, on the other hand, is designed to minimize performance degradation at all times. For example, if a control processor, memory module, or message path controller fails, the Director continues to operate with no performance degradation and the failed component can be replaced while switching operations continue unimpaired. Even if a port card fails, the impact to performance is limited to the four ports housed on that card. Moreover, the card can be replaced in a minute by hot swapping. When it comes to performance degradation, there is no substitute for fault tolerant features designed into the interconnect device.

CONFIGURING ENTERPRISE SANS FOR HIGH BANDWIDTH SCALABILITY

The third fundamental requirement of an enterprise SAN is to enable near-boundless application scalability. Highly publicized examples of web sites that can't keep up with demand are now commonplace. The reality of the Internet era is that no one can accurately predict the scale of storage capacity, storage performance, and server power required for successful e-commerce and, while e-business growth may be easier to predict than e-commerce activity, rapid scaling of e-business applications is both a business goal and an infrastructure requirement. In the Internet economy, there is never enough bandwidth. As discussed previously, the bandwidth scalability of an enterprise SAN backbone is determined by the number of ports connected to the backplane of the switch or Director used for that backbone.

SAN QUALITY OF CONNECTION

Quality of service for a SAN is defined with the SAN Quality of Connection classification matrix (Table 4) introduced in a Strategic Profile entitled "SAN: Quality of Connection" by Strategic Research Corporation. The scope of SAN QoC encompasses all interconnect devices and cables in a SAN. It does not include HBAs or storage adapters connected to the cables. This concise definition allows QoC to focus exclusively on the network portion of a SAN.

Note that five different QoC classes are defined in Table 4. Each guarantees a different level of SAN availability of connection, performance degradation, bandwidth scalability, and overall service. In order to achieve a specific QoC class, the associated SAN architecture described in the matrix must be implemented.

If nothing ever failed in a SAN, one architecture might perform as well as another. Unfortunately, hardware and software failures do occur, often at critical times. These failures clearly expose the differences between SAN architectures. The genesis for SAN QoC was the need for a simple, yet concise, SAN architecture classification methodology that provides the ability to establish service level guarantees for end users. Since availability of connection by itself is inadequate to do this, the concept of quality of service was applied to define how a SAN responds during failures.

SAN QoC guarantees are based on actual test experience. They reflect the level of service that different SAN architectures can deliver under various failure conditions, including path failover malfunction.

* Explanation of QoC Metrics

SAN availability of connection is defined as the annual percentage of time that an application has access to its data through the SAN. It requires the availability of at least one functioning path between each server required for the application and its storage. If any connection is lost that prevents the application from accessing data, the SAN has lost availability of connection. A primary path failure does not necessarily cause loss of connection unless path failover software is either not implemented or malfunctions.

A path-minute of SAN performance degradation is defined as one path, equivalent to a port on an interconnect device, that is unavailable for one minute, SAN performance degradation is typically caused by interconnect device failures. For example, if a Director port card with four ports packaged together is unavailable for five minutes before it is hot swapped, 20 path-minutes (Four ports x Five minutes) of performance degradation are tallied. Likewise, if a broken 16-port fabric switch takes 65 minutes to replace and reconfigure, 1,040 path-minutes (16 ports x 65 minutes) of performance degradation are recorded. In a working SAN, when a path becomes unavailable, it may or may not actually cause degraded performance, depending on workload and whether or not it is an idle alternate path. SAN QoC performance degradation provides a useful metric for specifying upper limits to actual performance degradation.

The principle of SAN performance degradation is very important because it focuses the design of interconnect devices in the right direction--towards fault tolerance. The objective is to minimize both the number of paths impacted by a failure and the length of time required for repair. SAN performance degradation, as measured in path-minutes, establishes the proper design objectives for the Fibre Channel networking industry.

SAN bandwidth scalability is classified according to the topology of the SAN. In order to scale bandwidth simultaneously with connectivity, additional paths that transfer data in parallel are required for incremental connections. Not all SAN implementations provide this capability.

To illustrate, consider an arbitrated loop that can scale connectivity from two to 126 nodes, but each node must share 100MB/sec of bandwidth with all of the other nodes. In this topology, bandwidth doesn't scale at all with connectivity.

On the other hand, fabric switches and Directors provide additional bandwidth each time a new connection is made, so switched fabrics scale bandwidth directly with connectivity as long as inter-switch links don't limit available bandwidth. As discussed previously, inter-switch links can increase connectivity in a SAN, but they don't scale bandwidth at the same time. Consequently inter-switch links alter the balance between connectivity and bandwidth in a SAN backbone. For this reason, the number of ports connected to a backbone device's back-plane determines the SAN QoC bandwidth scalability class for switched fabrics.

* Class 1 Quality of Connection

Class 1 QoC provides the lowest level of SAN service and offers no quantitative metrics or guarantees. Class 1 is extremely sensitive to failures because no redundancy is employed in the SAN architecture. Bandwidth scalability is characterized by a single, shared 100MB/sec path. This QoC class will result when a SAN is implemented using a single hub or point-to-point connection.

* Class 2 Quality of Connection

Class 2 QoC provides a higher average level of SAN service associated with 99% availability of connection, which is not very good for an enterprise SAN. Class 2 service levels can be highly variable, since its metrics are defined only by statistical averages. This class offers a degree of failure resilience because it introduces redundancy of paths and interconnect devices to the SAN architecture, but only on a partial basis.

SAN availability of connection is specified at an annual average of 99%, which corresponds to 3.7 days of annual average downtime. Average annual performance degradation is specified at 50,000 path-minutes.

Bandwidth scalability is characterized by multiple 100MB/sec paths, which can be achieved with multiple loops or a switched fabric. This QoC class is achieved when a SAN is implemented using switched hubs, EPL switches, or fabric switches with some path redundancy (appropriate path failover software is required).

* Class 3 Quality of Connection

Class 3 QoC provides a higher level of SAN service with less statistical variation than Class 2. Its metrics are specified as statistical averages with limits. Class 3 offers a high degree of failure resilience because it requires complete redundancy of paths in the SAN architecture and either redundant interconnect devices or single interconnect devices with fault-tolerant features.

Annual SAN availability of connection must perform in the range of 99.85% to 99.95%. This means annual downtime cannot exceed 13.1 hours and averages 8.8 hours. Annual performance degradation is specified at 5,000 path-minutes with a 5% range of variation. The bandwidth scalability needed for Class 3 QoC can only be achieved with a switched fabric.

Class 3 architecture requires redundant fabric switches for the backbone, redundant peripheral interconnect devices, and complete path redundancy (appropriate path failover software is required). Class 3 QoC can also be achieved using a Director for the backbone rather than redundant fabric switches. For Class 3 only, primary and alternate paths may be routed through the same Director.

* Class 4 Quality of Connection

Class 4 QoC consistently provides, a very high level of SAN service. Its metrics are tightly defined as limits. This class offers a higher degree of failure tolerance than Class 3 because it combines full redundancy of paths and interconnect devices with fault-tolerant features in the SAN backbone. These features reduce the number of times path failover is triggered and minimize service degradation if path failover malfunctions.

Annual SAN availability of connection is specified at a minimum of 99.99%, equivalent to no more than 53 minutes of annual downtime. Annual performance degradation cannot exceed 500 path-minutes. Hot-swappable components in backbone devices, including port cards that are packaged with a small granularity of ports per card, enable these service levels to be reached.

Bandwidth scalability at the Class 4 level requires a switched fabric and is characterized by the maximum number of ports per backplane available. Glass 4 Quality of Connection requires a SAN design that balances connection scalability with bandwidth scalability. As illustrated, inter linked switches do not provide this balance. A large number of ports per backplane scales the best.

Class 4 QoC can only be achieved today by implementing a SAN with redundant Directors for the switching backbone and redundant peripheral interconnect devices. Complete path redundancy (with primary and alternate paths routed through different Directors) using appropriate path failover software is also required.

The section following entitled "Fabric Switch Considerations" discusses why switches don't qualify for this QoC class today. In the future, enhanced designs may qualify them for Class 4. The concept of SAN QoC requires careful design in order to balance availability and performance with scalability. This design calls for maximizing the number of ports per backplane, minimizing port packaging granularity, and implementing fault tolerance with redundant, hot swappable components.

* Class 5 Quality of Connection

Class 5 QoC provides the highest and most consistent level of SAN service. It offers a high degree of failure tolerance because the architecture specifies that all interconnect devices be configured with fault-tolerant features to minimize the number of times path failover is triggered and to expedite recovery if path failover malfunctions. Full redundancy of paths and interconnect devices is also required.

Annual availability of connection must achieve 99.999% or less than five minutes of annual downtime. Annual performance degradation cannot exceed 50 path-minutes. Like Class 4 QoC, bandwidth scalability requires minimizing the number of backbone back planes and maximizing the number of ports per backplane. Today, QoC Class 5 can only be achieved by implementing a SAN exclusively with redundant Directors and complete path redundancy (appropriate path failover software is required).

* Fabric Switch Considerations

Fabric switches available today cannot deliver Class 4 or Class 5 QoC because they lack the fault tolerant features required to meet Class 4 QoC guarantees. Even with redundant power and cooling, they are still exposed to single points of failure that require the entire switch to be removed from the SAN for repair or replacement.

Replacing a backbone switch in a production SAN requires uncabling up to 16 cables, setting up and powering on the new device, establishing device management communications, updating zoning configuration information, recabling, and executing path failback for all attached servers. These activities take time that eats into the availability-of-connection and performance degradation windows of Class 4 QoC.

The time required to replace a 16 port switch causes a fabric switch backbone to exceed the Class 4 QoC annual performance degradation specification by a large margin. Further, if path failover malfunctions, the Class 4 annual availability-of-connection window (less than 53 minutes of downtime) cannot always be met. Also, fabric switches available today do not meet the bandwidth scalability specification of Class 4.

A backbone composed of fabric switches requires considerably more interconnect devices than a backbone implemented with Directors. This means that a fabric switch backbone is more likely to experience hardware and code failures and, therefore, path failover malfunction. Keep in mind that SAN Quality of Connection guarantees require that QoC metrics be achieved under both hardware and software failures.

SAN BACKBONE DEVICE QUALITY OF CONNECTION

With the SAN Quality of Connection classification matrix defined, it becomes a straightforward task to classify types of interconnect devices according to their QoC potential as SAN backbone devices. Table 5 presents this classification matrix as defined by Strategic Research Corporation in the Strategic Profile referenced earlier. QoC class and architecture descriptions in this matrix match those in Table 4.

Since QoC Classes 2 and 3 rely on the redundancy of paths and interconnect devices to achieve their respective availability-of-connection targets, fault-tolerant features in backbone devices are not required for these classes. However, adding fault-tolerant features such as redundant, hot-swappable power and cooling will certainly improve quality of service by increasing connection availability and reducing performance degradation. A single Director may be used in lieu of redundant fabric switches to achieve Class 3 service levels.

In order to reach Class 4 QoC, backbone devices must be consistently upgradable and repairable or replaceable within the 53-minute availability-of-connection window guaranteed by Class 4. They must also meet the Class 4 performance degradation guarantee of 500 annual path-minutes maximum. These guarantees are nearly impossible to meet unless backbone devices are designed for continuous operation. Redundant Directors achieve this QoC class today and future fabric switches with enhanced designs may be able to achieve it as well.

Class 5 QoC backbone devices must clearly be capable of delivering continuous operation by protecting all critical components with redundancy, automatic failover, and hot replacement. Port cards must have minimal granularity and be hot-swappable to meet the performance degradation specification. In addition, code must be upgradable while the device is in full operation. Only under these conditions can a backbone device meet the stringent availability and performance criteria of Class 5 QoC. Currently, a Director is the only interconnect device offering these features.

DESIGNING A SAN

As a rule, SANs should always be designed with the application in mind. Specific service goals should be set by the application's requirements for availability, performance, and scalability, rather than dictating one service level for the entire infrastructure. If not, excessive costs will likely be incurred for applications that don't require the same class of service needed by critical applications. Conversely, undercapitalizing the SAN infrastructure will lead to poor application service. Using SAN QoC by application ensures the lowest cost and greatest probability of success.

Once the requirements of the application are determined, the QoC classification matrix shown in Table 4 or 5 specifies the SAN architecture required to achieve that quality of service. For example, if an application needs Class 5 QoC, a SAN employing redundant Directors and redundant paths with path failover is required. No other architecture meets this need. Table 6 recommends the appropriate SAN QoC Class to consider when implementing common enterprise applications.

IMPLEMENTING AN ENTERPRISE SAN

The Internet revolution is changing the way commerce and business are conducted around the planet. E-commerce and e-business enable the most significant revenue opportunities available to companies today. Implementing these applications effectively allows an organization to realize tremendous return on investment by achieving competitive advantage, process efficiency and organizational streamlining.

Implementing e-commerce and e-business applications effectively requires that business information be made more accessible, scalable, available, and manageable. An enterprise SAN provides the consolidated infrastructure and resource-sharing capability necessary to achieve this.

The applications that enable large-scale e-Commerce and e-Business operations should be designed to deliver consistent high performance and continuous, scalable operation because they must be online around the clock for a hard-to-please user set. For the enterprise SAN infrastructure that supports these applications, this means specifying Class 5 Quality of Connection.

Gary Wright is the director of market development at McDATA Corporation (Broomfield, CO).

                 Storage Connectivity Technologies Compared
       Attribute         SCSI-2 FWD  Ultra SCSI   LVD SCSI
    Rated bandwidth      20MB/second 40MB/second 80MB/second
   Addressable nodes         16          16          16
     Maximum bus or       25 meters   12 meters   12 meters
      cable length
  Switching capability       No          Yes         Yes
 Nondisruptive upgrades      No          No          No
Full duplex transmission     No          No          No
  Termination required       Yes         Yes         Yes
  Protocol independent       No          No          No
       Attribute           Fibre Channel
    Rated bandwidth        100MB/second
   Addressable nodes        16 million
     Maximum bus or      10,000 meters per
      cable length             link
  Switching capability          Yes
 Nondisruptive upgrades         Yes
Full duplex transmission        Yes
  Termination required          No
  Protocol independent          Yes
                   Fibre Channel Distance Considerations
   Type of fiber    Maximum distance
    optic cable         Per link
     9 micron        10,000 meters
    single mode
50 micron multimode    500 meters
    62.5 micron        175 meters
     multimode
                    Director / Fabric Switch Comparison
     Feature      Director Fabric Switch
  All critical      Yes         No
   components              (maybe power
  except ports              and cooling)
  are redundant
 with automatic

internal failover
  All critical      Yes         No
 components are            (maybe power
 hot replaceable            cooling, and
                              GBICs)
  Nondisruptive     Yes         No
   code loads
  Nondisruptive     Yes         No
    upgrades
 Component-level    Yes         No
 fault isolation
  Maximum ports      32         16
                         SAN Quality of Connection
 QoC   Service Level        Architecture             Availability of
Class   Description         Description                 Connection
                                                     (annual uptime)
  1     Unspecified           Failure
      (no guarantee)         sensitive
                        -no redundancy
  2      Variable          Failure resilient           99% average
       (statistical)    -partially redundant        (3,7 days average
                          paths                      annual downtime)
                        -partially redundant
                          interconnects
  3      Variable        Failure resilient      99.9% [plus or minus] .05%
      (deterministic)  -fully redundant paths       (8.8 hours average
                       -fully redundant or           annual downtime.
                         fault tolerant               cannot exceed
                         interconnects                  13. 1hrs)
  4      Constant        Failure tolerant                99.99 %
      (deterministic)                                    minimum
                       -fully redundant paths        (maximum annual
                       -fully redundant               downtime under
                         interconnects                 53 minutes)
                        backbone
                         interconnections falut
                         tolerant
  5      Constant         Failure tolerant               99.99 %
      (deterministic)  -fully redundant paths            minimum
                       -fully redundant              (maximum annual
                         interconnects                downtime under
                       -all interconnects fault         5 minutes)
                         tolerant
 QoC    Performance      Bandwidth
Class   Degradation     Scalability
       (annual path-
         minutes)
  1                    Single point-
                        to-point or
                        single loop
  2    50,000 annual     Multiple
       path-minutes     loops. EPL
          average         and/or
                         switched
                          fabric
  3     5000 annual      Switched
       path-minutes       fabric
       average. cannot
       exceed 5250
  4     500 annual       Switched
       path-minutes     fabric with
          maximum         maximum
                         ports per
                          backplane
  5     500 annual       Switched
       path-minutes     fabric with
          maximum         maximum
                         ports per
                         backplane
Source: Strategic Research Corporation
                 SAN Backbone Device Quality of Connection
QoC   Architecture Description
Class
1     Failure sensitive
      no redundancy
2     Failure resilient
      * partially redundant paths
      * partially redundant intes
3     Failure resilient
      * fully redundant paths
      * fully redundant or fault tolerant
      * interconnects
4     Failure tolerant
      * fully redundant paths
      * fully redundant interconnects
      * backbone interconnects fault tolerant
5     Failure tolerant
      * fully redundant paths
      * fully redundant interconnects
      * all interconnects fault tolerant
QoC        Minimum Device Capability           Current Device
Class                                          Implementation
1          No critical components are            Single hub
       redundant with automatic failover
2          No critical components are        Redundant switched
       redundant with automatic failover   hubs, EPL switches, or
                                              fabric switches
3        If redundant interconnects, no       Redundant fabric
        critical components are dedundant    switches or single
      with automatic failover, otherwise,         Director
      fault tolerant features are required
4         All critical components are       Redundant Directors
       redundant with automatic failover
5         All critical components are       Redundant Directors
       redundant with automatic failover
Source: Strategic Research Corporation
                            Recommended SAN QoC
                       Class for Common Applications
Application                  Recommended
                               SAN QoC
                                Class
eCommerce                      4 or 5
eBusiness                      4 or 5
Enterprise Resource Planning   4 or 5
Data Warehouse                 4 or 5
Decision Support               4 or 5
Financial                      4 or 5
Transaction Processing         4 or 5
Customer Service               4 or 5
Web serving                    3 or 4
eMail                          3 or 4
Imaging                        3 or 4
Technical                      2 or 3
Application Development        2 or 3
File Serving                   l or 2