SMS for open systems: the time is now.

In April 1988, IBM announced DFSMS (Data Facility Storage Management Subsystem) for their MVS (mainframe) computers. Now commonly called SMS, this architecture consisted of a set of software products that made storage management a reality for today's enterprise-class mainframes or z/OS processors. SMS has provided the capability to centralize storage management, automate allocation and isolate users from most aspects of data storage and administration. In time, it became an effective policy engine for managing storage resources and required users to get to know their data and its value in helping businesses access the right data, at the right place, at the right time.

Much of the storage management dilemma that mainframe storage administrators faced then was directly addressed by SMS--namely, to meet performance requirements for users while preserving data integrity and minimizing costs. It had been difficult to achieve a balance among the three objectives. For example, putting data on magnetic disk and not using tape will maximize performance but also maximize costs. Similarly, backing up and replicating data to tape on a frequent basis improves data safety but may impact performance or recovery times. There are several components to SMS, but HSM was the catalyst that enabled businesses to address the storage dilemma referenced above. Ultimately, SMS successfully accomplished this feat after 15+ years of evolution, and it is now a standard component of nearly all mainframe computing systems representing possibly the most significant achievement ever in storage management.

CMG 2003 Dallas

I chaired a storage futures panel at the Worldwide CMG (Computer Measurement Group) conference in Dallas last December. Well-known storage industry luminaries on the panel were Jon Toigo of Toigo Partners International, Greg Schulz of CNT, Rob Peglar of XIOtech, and John Tyrell of EMC, who holds two patents on DFSMS. The panel concluded with considerable audience participation addressing the subject of storage management for non-mainframe systems. What most people wanted, if not demanded, was a DFSMS-like system for Open Systems--specifically for Unix, Win2K/NT and Linux. After all, these systems generate more than 85% of the world's stored digital data.

Open Systems Issues Mount

Why is storage management so much more of a problem on Open Systems than mainframes? Why does a storage administrator on a mainframe manage, on the average, well in excess of 30 terabytes of online storage while Open Systems storage administrators manage just a few terabytes on the average? Why isn't there a single, centralized solution for storage management like SMS that really works across multiple operating systems?

The storage operating system services for Unix that are present today are basically the same ones that existed 20 years ago when Sun first released NFS (Network File System) in 1984, and have seen little improvement since. There have been no enhancements to the general operating system in terms of storage services since the days when a large Unix server had about 1-gigabyte of disk storage and PCs were just standalone boxes. No one knew then that these distributed computing systems would one day be asked to do the work of a mainframe and to access as much, if not more, data than their mainframe counterparts. Sadly, storage management capabilities didn't exist then to any degree outside the mainframe.

If much foresight had existed, the storage services for these systems would have been seen as unacceptable and would have been resolved long before now. Multiple versions and variations of Unix emerged and each adopted their own volume managers, backup/recovery products and data movers. Few versions were easily interchangeable. In addition, hundreds of independent storage management software products were developed to address the exploding Open Systems storage management problem, which became apparent by the end of 1990s. There is little basis to believe that we can scale Open Systems to the level necessary to replace the mainframe-class servers without having the same type of tools available that made their predecessor so successful. Nonetheless, we have already moved much of the storage, data, and applications to computing platforms that have no SMS, and are ill-equipped to effectively manage the terabytes of storage they now have to deal with. The storage management challenge for Open Systems is often called the "The Storage Management Gap" or "The Infinite Disruption."

The Infinite Disruption

For Open Systems, managing storage has become much more than making sure enough capacity is available to meet demand. Disk and tape technologies from multiple vendors, a variety switching devices, network management, SAN, NAS, DAS, implementing acceptable backup/recovery and high availability technologies, and meeting the SLAs of key applications make the complexity of effective storage management out of reach for most all non-mainframe businesses.

Throughout the 1990s, the easiest way to manage non-mainframe storage subsystems was usually by simply adding more storage. This straightforward strategy seemed to work well for many small and medium-size businesses for many years. As the growing management gap between storage capacity growth and the number of storage administrators continued to diverge, this strategy began to collapse.

Storage capacity grows faster than the deployment of management tools and the supply of trained people to manage storage doesn't keep pace with demand. As a result, a great deal of valuable business data is not effectively managed--meaning that backup, recovery, performance and capacity are dealt with in a reactive manner--if at all--often placing a company's most valuable asset at risk. By 2007, it is expected that the average storage administrator will be able to manage about 15TB of storage, while the amount of data to be managed will exceed 60TB (remember, these are averages). It is imperative that efforts to close this gap escalate quickly in order for the disruption to not become infinite in duration.

Storage management can be simplified by adding NAS for certain file-based applications, SANs for others, adding virtualization software, unifying block and file storage systems. These take time and resources to implement. Possibly the most significant development will be in the form of storage management functions, or even an SMS, that move off the server and into the storage subsystem or network. This concept is referred to as the Intelligent Fabric or Intelligent Switch and offers significant potential to impact the gap. Development is now under way in this area. We continue to observe that a mainframe enables storage to be much more efficiently managed than all other platforms. As we observe, the typical mainframe (z/OS) storage administrator manages an average 30TB or more of disk storage while the typical storage manager on a Unix, Linux and Win2K/NT system manages between 500GB to 1.5TB of disk storage. It has finally become the goal of these systems to achieve the storage management capabilities of the mainframe.

Overcoming the Barriers: SMS for Open Systems

There are hundreds of companies providing solutions to various storage management problems for Open Systems. The sheer number of product choices makes buying decisions complicated and it is impossible for all the various products to interoperate. This suggests that a consolidation of the countless vendors will accelerate and that true Open Systems will continue to be a dream versus a reality for some time. Several HSM products exist for Open Systems but none support the three fundamental Open Systems computing platforms: Unix, Win2K/NT and Linux. Therefore, a business often needs to install multiple, semior non-compatible software products to effectively manage their storage environment. This is exactly what they don't want to do and, with increasing storage growth, it represents an unsustainable strategy.

Several fundamental components of an SMS and HSM for Open Systems exist and have been made possible by the incorporation of the Data Management Application Programming Interface. DMAPI is now available on many file systems, including the open source XFS file-system for Linux. Given the numerous software products customers have to choose from, the open source route may be the only realistic way that a heterogeneous, open SMS and HSM will be developed. The success of Linux evolving in this mode is a good example of developing and improving software through open source. The era of combining weak operating system storage management services with very large storage subsystems may soon be coming to a close. Look for at least one exciting strategic initiative now underway to attack this problem in 2004.

Regarding SMS for Open Systems: The time is now.

www.cmg.org

www.cnt.com

www.emc.com

www.ibm.com

www.toigoproductions.com

www.xiotech.com