Storage Management--The Grand Vision Unfolds.

From time to time, I have often gone back to what I refer to as the Grand Vision for Storage Management as one of the best-ever examples of defining a long-term strategic vision. In the past, a typical strategic planner in the storage or IT industry would try to create a strategic plan using past trends and survey data that would look forward as much as five years. The outer two or three years of the plan were normally a best guess based primarily on past trends. Disruptive factors, dark horses, and alternative scenarios were seldom addressed in these look-ahead processes. Today in the year 2000 as much change occurs in a one-year period as happened in a five-year period in 1990. Producing a great IT vision today is much more difficult and is likely to form its basis on factors that can't be trended from past results or be extrapolated. Disruptive factors in the IT world appear with much greater frequency than ever before.

In the early 1980s, SHARE and GUIDE (now defunct), two large IBM user groups submitted requirements, often referred to as the Guide White Paper, for improving storage management functionality and setting directions for future developments. These requirements are listed as Guide Publication NO. GPP-106 dated 1983. The landmark 1983 Guide White Paper concluded with the statement: "Implementations of solutions to the problems outlined meeting the requirements stated will likely be a series of evolutionary steps or new features introduced in the one to five-year timeframe with a complete solution in the five to 10-year timeframe".

It is now seventeen years later and many of the elements of this storage management doctrine have been addressed, some more successfully implemented than others. More importantly, the requirements listed in the document remain paramount issues in the storage management industry today extending to all levels of computing. The Guide White Paper identified the eight following requirements as the top challenges for storage management for a ten-year period.

The Eight Original Storage Management Requirement

1. A revolutionary solution to be presented in an evolutionary manner.

2. Productivity of support personnel must exceed the storage growth rate.

3. Users must be aware only of data attributes, not physical attributes.

4. The subsystem must be self-adjusting to a changing environment.

5. Data must be accessible across multiple execution environments.

6. The subsystem must be capable of recovery from failure.

7. The security interface, a universal interface to a standard system security manager, must be available in the storage subsystem.

8. The subsystem must be flexible to allow the addition and removal of components without service interruptions.

The scope and requirements of the Guide White Paper have obviously expanded but it is a remarkable example of a far-reaching view into an area that we can barely see beyond the twenty-four months look-ahead window of today's plans. The applicability of these storage management requirements in the year 2000 is great and extends to virtually all levels of computing. Furthermore, many of these requirements define functionality that is or promises to be embodied in today's emerging SAN and NAS architectures that are becoming the foundations of network storage management. These two terms were unknown in 1983 and again ten years later in 1993. Did this paper underestimate the scope of building storage management architecture? Was the plan extremely insightful and ahead of its time? Could the issues facing the management of data be the same ones facing us nearly twenty years later?

Commentary On The Original Requirements

To begin with, is storage management being presented in an evolutionary manner and producing revolutionary results? The preferred storage model has evolved from a data center view in 1983 to an independent server model creating both computing and data islands in the late 1980s to the middle 1990s, to a storage network model in 2000. Storage management is clearly evolving to a storage network model. Are the results really revolutionary? The answer to this can be measured by marking progress against the remaining requirements.

Did keeping the productivity of storage support personnel above the storage growth rate actually occur? Software tools improved the ability of a storage administrator to manage more data. A study from Strategic Research Corporation indicates that a storage administrator managed an average of 50GB in 1995 and will manage an average of nearly 250GB in 2000. A five times productivity improvement in five years versus an annual digital storage growth rate of at least 60% per year indicates that the 50GB would grow to 524GB in the same five year period. Clearly, storage is growing faster than the administrator's productivity, even with the introduction of new tools. The numerous Internet-based applications are generating digital data even faster, often well over 100% per year, further indicating that productivity is not keeping up with storage growth rates. This has led many end-users to consider that their only options to deal with the storage management problems is to either outsource their storage requirements or to simply "buy more storage" and forget about managing it. NAS and SANs promise to improve storage management by providing a single management interface with common access controls.

The AS/400 system has had a single-level of storage for nearly ten years and has made notable strides in eliminating the need to be aware of physical storage or device attributes. For NT, Unix, OS/390, and Linux systems, the need persists, though help is on the way via the SAN. SANs reduce the need to understand physical data attributes and enable applications such as backup and recovery to occur based on data attributes by providing server-less backup that moves data directly from disk to the tape subsystem without direct user involvement. In general, we are still aware of the underlying storage technologies.

One of the primary benefits of the SAN promises to allow the user to view storage as one large data pool and enable movement of data outboard of any SAN-attached server. Does a SAN allow for the storage system to be self-adjusting? Cached disk subsystems (first successfully delivered in 1981) have implemented this by promoting data to the faster cache when increased performance is needed and demoting (de-staging) the data to slower and less costly levels of disk storage when performance or access requirements are minimal. HSM (Hierarchical Storage Management) takes this a step farther by migrating data from disk to automated or manual tape storage systems if the data is inactive for longer periods of time. The data is recalled from tape to disk when next referenced. Like most advanced storage management concepts, HSM is just now beginning to find its way into the Unix, Linux, and NT markets.

Making data accessible across multiple environments is now commonly referred to data sharing and again is a fundamental element of SAN development activities. Though we have had some degree of data exchange, data reformatting, file copy, and duplication capability for several years, accessing a single instance of data or true data sharing is the ideal state for heterogeneous client access and is the development goal of several hardware and software companies today.

Making a system capable of recovery from failure is a primary goal of servers, storage subsystems, and networks and is the underpinning of much of the discussion around high availability in general. Notable progress has been made since the white paper identified this major requirement. For storage systems, the widespread use of RAID (Redundant Arrays on Independent Disks) significantly enhanced the availability of online storage. RAID was an unknown when the white paper was written in 1983 and storage failures were much more common then than today.

It was not perceived that the security system itself would be part of the storage subsystem, but simply that any server will use the same security interface and constructs and that the storage subsystem will have a standard interface to that system. We can see progress here again through the SAN concept as the SAN moves to treating storage with a common set of data access interfaces.

The eighth requirement states that the subsystem must be capable of adding and removing devices without service interruption. Some disk and tape subsystems allow dynamic device in addition to increase storage capacity, install functional upgrades, and provide for repair or removal without disruption. This has been and is a fundamental storage requirement. NAS has offered this capability for several years while SANs intend to make this the norm for all SAN-attached storage systems.

The Guide White Paper correctly identified many of the long-term requirements that today shape the storage and networking industry. Nearly twenty years later, we are still trying to implement much of the intent described by the original project members for large systems on distributed platforms. The Internet, the popularity of NT and Unix, the power of personal computing, and the overwhelming dependency of a global society on information technology were not known or foreseen in 1983. The disruptive factors were in their infancy. The vision of the original project members was nonetheless exceptional and far-reaching; it was indeed a Grand Vision.