Without A Revolutionary File Management Solution There Can Be No True SAN.

Numerous publications in the IT industry have run articles addressing the unprecedented growth in the amount of digital storage. The combined disk, tape, and optical storage industry will approach $50B in revenues this year with storage demand ranging from 60% to more than 100% for some businesses. The dynamics of the storage industry are creating some diverging trends affecting the established methods of managing digital data storage.

The growth of data is now exceeding our ability to manage the data and its associated storage infrastructure. The number of people that manage storage is diminishing at the same time as the requirement to manage ever-increasing amounts of storage is increasing. Data is now being created and stored everywhere, as data storage is no longer the exclusive domain of centralized computing centers. The value of data is continually escalating: just ask any financial company or e-commerce business. Managing storage effectively and implementing capacity planning, resource management, backup, restore, migration, recall, archive, and data deletion are all key elements of storage management. Unfortunately, despite countless software offerings, our collective ability to implement these elements of storage management in order to keep pace with demand is steadily diminishing. Software has not yet overcome the real and growing needs of most customers.

Our options include trying to develop and implement the ultra storage management system or just letting someone else manage storage for us. Outsourcing storage was discussed in a recent article about SSPs (Storage Service Providers), so let's look at the alternative aspect of actually managing storage. As an industry, we are nowhere near supplying fast and seamless access to geographically diverse storage resources. Future storage management software systems will make decisions about the optimal place to organize data on disk, tape, or other mass storage subsystems, regardless of how the devices are physically organized. The requirement to allow the existing, de-centralized computers and the exploding Internet platforms to truly share data from a vast menu of selections by using a new and disruptive file system approach has never been greater.

The above description describes the most pressing need for an almost revolutionary file management solution. The solution ideally is seamless and scalable, supporting continuous and unlimited growth and delivers very high availability and quality of service. By deploying file-intelligent I/O combined with distributed file system technology, we can finally provide a centralized approach to storage management across an increasingly wide storage network. The physical volume that we have grown to understand gives way to a virtualized networked storage pool that understands files and metadata about the files as the new logical unit. The core building block by which all data is accessed is now the file. This is consistent with the Internet file-structure standards and allows for universal development on a universal platform. The Internet represents a sea change of a magnitude requiring an entirely new approach to data storage and, in five years, the Internet and its associated applications are expected to account for over one-half of the world's digital storage demand.

We are now passing through the stage where storage no longer needs to be partitioned, allocated, or owned by a single server. The file system ideally shifts from being server based and becomes distributed over the storage network. In this case, the storage network can include SAN and NAS implementations. The true goal for this software is to create a single file system that can be accessed by multiple, non-similar servers and their clients and is transparently mapped across multiple storage devices or subsystems. As storage devices are added, the storage administrator(s) should no longer be burdened with manual data migration, cross-mounting, re-mapping security permissions, or assigning other volume management fencing constructs. The result is that server specific volumes are no longer required and the logical path to a file is the same no matter which server or client requests the file.

Why haven't we seen the optimal intelligent file management solution arrive yet? Automating these functions has previously been too costly and often too complex. Today's leading operating systems such as Unix, NT, and Linux still depend on owning specific blocks of storage. This is true, even with a SAN environment. Volume and true data sharing has always been very complex and logical management of the SAN is still accomplished by moving, backing-up, or replicating data. In a SAN, each server often ends up managing its own storage "zone". Resource management is frequently performed for each zone and a complete view of the entire storage pool is hard to determine. Utilization, therefore, can still be inefficient in the typical SAN, though overall storage management significantly improves due to the more centralized focus that the SAN offers.

Ideally, we should be able to map and stripe files on the SAN across the available storage devices, even if they don't have RAID capabilities. Creating a single pool view of storage improves capacity planning capabilities, as free space can be dynamically used wherever appropriate and is not limited to a specific storage zone. New storage devices can be added dynamically without manually cross mounting volumes to a specific server, thus allowing transparent scaling of storage.

This vision describes the future "brains of the SAN" and suggests that a distributed, shared SAN file system is optimal. With a shared SAN file system, each connected server can be configured to the exact same view of the total storage pool. Availability, today's overriding attribute for most data, is further enhanced by implementing an intelligent file management system in conjunction with shared-disk clusters. In case of a server failure, all files are available to the remaining servers in the cluster. A shared-nothing cluster does not permit files to be accessed from more than one cluster in case of a server failure. Files must be duplicated and must be kept synchronized in the common shared-nothing cluster. Concerns about implementing a new file management system and how long the implementation will take have persisted. The question, "Will the new system work with existing databases or applications?" is frequently asked. Will the new system work with existing backup, restore, and HSM software? Obviously, these questions are part of the reason why we see the continuance of block 110 based systems.

Implementing a true SAN has taken longer than expected, but progress is being made. The true value of a SAN will not be realized until multiple heterogeneous servers can share the same file systems. Given that SAN implementation overall is still in its infancy, why not implement a file management system for the future as part of the SAN? Much of the work will be the same. We have chosen servers and operating systems based on their specific business needs; why not choose the SAN file system that offers the greatest flexibility and best positions the organization for the storage management challenges ahead? We are approaching the point in the evolution of storage management where we will no longer account for an operating system owning the data. The separation of logical and physical data is finally dawning.