Where Will The Intelligence Reside In Storage Area Networks?

System architects spend long hours pondering where functions belong in the big picture. After modeling designs to optimize cost, performance, and reliability metrics, recommendations are rendered and decisions are made. Yet, as the industry delivers the next inflection point in storage systems-the Storage Utility--the market will make its decisions based on what can be reliably deployed, managed, enhanced, and extended across heterogeneous systems. If networks do not evolve gracefully, GIOs are given new "career opportunities."

It looked like computing environments were on the verge of becoming homogeneous with a single OS (NT) running on a single hardware architecture (x86), but Solaris is strong in the Internet market; Linux disciples are multiplying and a new Unix system, Project Monterey, will debut this year. The demise of NetWare has long been predicted, but sales keep chugging along. Gigabit Fibre Channel, the SAN interconnect standard, will be replaced by 2Gbps fibre next year and 10Gbps is on the drawing boards, but 10 gigabit fibre may never see the light of day as a SAN Interconnect, since the industry heavyweights are backing InfiniBand with bandwidth up to 30Gbps in 2002.

Platform heterogeneity and looming infrastructure changes are major impediments to the promised Storage Utility and primary determinants of where storage "intelligence" will reside in SANs. Traditionally, storage has been server-dependent with data management implemented at system level via file systems, device drivers, tools, and utilities. Distributed, server-dependent storage limits flexibility, resource sharing, and is costly to manage.

Network Attached Storage has taken the market by storm with low cost and plug-and-play simplicity. If you need more storage, just plug-in a NAS appliance. If you need shared storage in a remote office where you can't afford a system manager, plug-in NAS. High-end NAS systems have an equally appealing feature-- self-contained data management. With RAID, tape, and software utilities, a NAS system will serve protected files all day long. NAS is effective if the network can handle the bandwidth demands and serving files satisfies application needs.

The Storage Utility envisions the same plug-and-play simplicity and self-contained data management, but it offers more flexibility, data protection, and security without chewing-up client network bandwidth. A SAN is a specialized back-end network, enabling local or geographically dispersed servers to access shared storage resources and a prerequisite to the promised Storage Utility. Yet, to fully realize the Storage Utility vision, storage management functions need to migrate from heterogeneous servers to intelligent devices, providing consistent storage services to all SAN platforms.

External RAID arrays are inherently OS independent. They map physical disks to logical volumes protected against disk failures and export their volumes to servers across shared interconnects. Today's RAID controllers have the processing power and memory to do a lot more, will evolve into SAN storage managers in the near-term, and potentially SAN data managers in the longterm. A storage manager allows access from a single system to a set of addressable storage locations, whereas a data manager enables concurrent access from multiple systems to the same set of addressable storage locations. The former deals with block translation and the latter shares data through distributed file systems.

Storage management is migrating to controllers; four functions vital to the notion of a Storage Utility have already been delivered in RAID arrays: Storage Virtualization, Volume Mapping, Remote Mirroring, and Virtual Volumes. Longer term, back-end device sharing may evolve to back-end data sharing. NAS technology at a finer granularity of access. SAN and NAS technologies will coexist and may eventually merge-the only differences being the protocols they speak and whether they plug into front-end client networks or backend storage networks.

Storage Virtualization

Decentralized, server-dependent storage is difficult to protect and expensive to manage. Hence, the trends toward re-centralizing storage that enables a pool of storage to be shared by a group of servers. Storage virtualization allows disks to be aggregated, subdivided, and parceled Out to SAN nodes. As disks are added, the RAID controller transparently increments the usable capacity of the pool. Disks with varying capacities can be mixed and matched without wasting disk space.

Drives are logically combined into RAID groups and assigned a RAID level such as striping or mirroring that provide the best optimization of cost, performance, and reliability for a specific application. The capacity of the RAID group can be allocated to a single large logical volume or subdivided into separately addressable volumes. Volumes can be expanded on the fly to meet changing application storage requirements or deleted and returned to the pool when they are no longer needed. Storage virtualization was once the domain of host-based software, but has migrated to array controllers where the disk pool can be effectively managed as a single entity.

Volume Mapping

Since RAID controllers present logical volumes to all attached nodes, SANs have to deal with disk ownership issues. Unix systems are generally well behaved and only access volumes that have been explicitly mounted. Other OSs claim ownership of every volume they can see on their I/O channels by writing an identifying signature. If multiple servers claim ownership of the same disk, the second server to claim ownership can corrupt data written by the first. Clusters solve this problem with software that enforces storage access policies. In storage networks, an access control or mapping mechanism creates virtual private networks between nodes and volumes to prevent one SAN node from stepping on another node's data. Mapping volumes to nodes also creates a security barrier by restricting access to sensitive data.

Three mapping strategies have been implemented. Filter drivers are used to restrict system access to specified volumes. The drivers use proprietary protocols, are OS-specific, and may need to be upgraded with new OS releases. Management complexity increases unless the filter drivers are integrated into a common management framework. The second approach uses "zoned" switches that map server ports to array ports. Zone or "hard mapping" is less flexible, requiring physical configuration changes to alter mapping schemes. Additionally, the volumes behind an array port cannot be assigned to servers in different zones. The third approach uses array controllers. Mapping physical disks to logical volumes is intrinsic to RAID controllers. Dynamically mapping volumes to SAN nodes is a natural extension of this technology. No overhead is imposed on the servers and I/O latency remains unchanged. Mapping tables can be replicated across dual active controllers for high availability and integrated into the array's management utility.

Mylex delivers a patented controller-based mapping feature called SANmapping. Mapping tables uniquely identify SAN nodes by their worldwide names and specify their access privileges to logical volumes.

Remote Mirroring

Studies show the hourly cost of downtime ranges from $100,000 in airline reservation systems to over $2 million for credit card sales. With electronic commerce gaining ground in every market segment, companies need to develop a continuous customer service strategy. Remote mirroring replicates data from online production servers to back-up servers at remote sites that can take over if a disaster strikes the primary site.

Remote mirroring was initially implemented in host-based software systems. For each application write, the software issues a write I/O operation to the local disk pool and a second write I/O across a communication link to the remote site. The overhead on the production server is a function of the application's read/write mix and the efficiency of the software. Up to 5% of a server's CPU cycles are burned propagating writes to the back-up site.

IBM and EMC were among the leaders in migrating remote mirroring from system software to RAID arrays and Compaq has delivered remote mirroring on arrays for x86 servers. This strategy has two obvious benefits: server overhead is eliminated and remote mirroring is host independent. An array controller can mirror writes issued from any Os, an important consideration as heterogeneous SANs are deployed (See Fig).

Virtual Volumes

Another feature driven by e-commerce and the exponential growth in databases is creating virtual volumes for read-only applications. Global electronic commerce requires around-the-clock service access. Continuous customer access conflicts with data back-up procedures that shut down a database before copying it to tape. Decision support applications also require a point-in-time copy.

An early solution was N-way mirrors. RAID software was used to create a three-way mirror set and one of the mirrors was taken offline. A fault-tolerant two-way mirror remained on-line for application access and a copy of the data was available for read-only applications. Creating a three-way mirror with system software consumes server and channel resources and requires 50% more disk space.

IBM, StorageTek, and others have designed innovative array controllers that create point-in-time virtual copies of on-line data nearly instantaneously. One approach uses a temporary storage area to save the "before state" of on-disk data. Prior to servicing write I/Os, the controller copies the "before-state" of disk blocks to a temporary storage area. A data structure directs read-only applications such as backup to the online volume for blocks that have not been updated since the virtual copy was created and to the temporary storage area for blocks that have been updated. Controller-based virtual volume technology eliminates server overhead, dramatically reduces channel and disk utilization, and is server independent.

Shared Access And Shared Data

Most SANs deployed this year will use the shared access model -- shared interconnect and controller resources with disks allocated to a single SAN node. The shared access model enables storage pooling to increase flexibility and lower costs. However, in some environments, applications on different SAN nodes require simultaneous access to the same stored files.

Shared data clusters use a lock manager to synchronize file access. Nodes acquire locks on disk blocks before accessing them. As the number of cluster nodes increase, so do the lock-related messages and the node hosting the lock manager, limiting scalability. In the future, a distributed SAN file system may be implemented in an array controller to address this. A disk I/O may include an implied lock on accessed blocks . Since each array controller only needs to manage locks pertinent to the volumes it controls, the lock manager could be distributed across array controllers in the SAN, eliminating the hot node problem.

Smarter Arrays

A case can be made for host-based storage management in homogeneous environments where a single set of software products provide consistent function and behavior across SAN nodes and easily integrate into a common management framework, but most IS shops are heterogeneous today and for the foreseeable future. In this environment, self-managing arrays will reduce complexity, lower costs, and increase purchasing flexibility, eventually becoming the enabling technology behind the Storage Utility.

Kevin Smitb is the senior marketing director RAID controllers at Mylex Corp.