10 challenges for the storage industry: how many do you think will be met in 2003?

It's hard to believe that we are already approaching the end of the third year of this century. Three years ago, the dotcoms dominated the industry, Y2K was driving all sorts of irrational actions, and the IT industry was exploding with new architectures and innovation was everywhere. Many thought that all the lights and electrical systems would go out at midnight Dec. 31, 1999. Three years later, we've seen the dot-coms collapse, a prolonged recession--particularly in the technology sector--painfully witnessed the weaknesses in almost all aspects of security, and have raised interest in data protection and disaster recovery technologies to unprecedented levels. It's doubtful that any visionary could have seen all these events coming and developed the appropriate responses to deal with the issues in advance. With the end of 2002 imminent, what are some of the biggest issues the storage industry will face in the year(s) ahead?

1 Storage Management Capability Is Not Keeping Pace With Storage Growth

As the number and types of storage devices continue to proliferate, the complexity of managing storage grows non-linearly. Annual storage growth ranges from 50%-70% while the improvement in a storage administrator's capability improves at 20%-30% annually. It is imperative that efforts to close this gap escalate quickly in order for the disruption not to become infinite in duration. Storage management complexity can be simplified by adding NAS for certain applications, SANs for others, adding virtualization software, unifying block and file storage systems into a singular architecture, and implementing server-less storage management functions. The traditional mainframe does enable storage to be much more efficiently managed than other platforms. The typical mainframe (z/OS) storage administrator manage 30-40 or more terabytes of disk storage in 2002 while the typical manager of Unix, Linux and Win2K storage manages about 750GB on the average. It is the goal of these systems to achieve the storage management c apabilities of the mainframe someday.

2 There Is No Real Answer in Sight for Widespread Interoperability

While everyone wants to arrive at a point in time where any product can interoperate with any other, the reality is that this goal isn't getting much closer. Standards committees meet endlessly to reach consensus while vendors quietly resist too much agreement fearing a loss of competitive edge. The real progress made in interoperability remains centered on a vendor working closely with a few of their select partners, but a true open architecture is not in sight.

3 The Shortage of Skilled Storage Workers Is Growing

The shortage of trained storage specialists is becoming more serious and no university offers a degree in storage or storage management as of yet. Nonetheless, a few certifications and certified programs are just now beginning to appear to offer specific in-depth training in all aspects of storage management. This could be an excellent re-training path for numerous workers who need to transition their skill sets in this disruptive transition period for the storage and I.T. industry. SNIA offers vendor neutral certifications on implementing Fibre Channel networks. Also see http://www.datastorageuniversity.com.

4 The Security Capability for Data Protection and Network Access Must Be Dramatically Improved

As the value of data grows, traditional IT and storage infrastructure protection is becoming a more well-defined discipline than ever before. The new enemy for data survival may no longer be from technology such as a disk crash, a tape that won't read, or a blank screen from a system or server crash. The biggest challenge is coming primarily from people and from environmental disasters. Storms, electrical failures, floods, viruses, hackers, piracy, insider threats, foreign countries, business competitors, modified accounting records, and stolen files are now causing the majority of computer downtime. The Code Red virus' total cost was $2.6 billion in 2001. In a recent MetaGroup study, the Energy sector reports an hourly revenue loss of $2.8 million per hour followed by the Telecommunications industry, which loses $2.06 million per hour of computer outage. Growing data security issues are forcing companies to implement dramatic improvements to I.T. security systems. These new systems will use biometrics, advan ced encryption, and spur the securities market to grow from $6 billion in 2000 to $21 billion by 2005.

5 Recovery and Business Resumption Strategies of Most Businesses Are Inadequate for Survival

Recovery and business resumption strategies are being upgraded facing a new wave of terrorism, natural disasters, and a variety of cyber-crimes. Existing systems have numerous exposures impacting data loss or access. Data protection, disaster recovery, and security are possibly the three most critical issues facing the IT industry today. "The Cost of Downtime Survey" published by Eagle Rock Alliance indicates that 24% of businesses surveyed believe that their business survival is at risk if the computer downtime exceeds 8 hours. The sum of TCO, the lost revenue due to downtime, plus the total cost of business resumption is the new metric called The Total Cost of Survival (TCS).

6 Disk and Tape Storage Vendors Need to Provide Scalability in Both Capacity and Performance

Disk and tape drive suppliers historically focus efforts on driving capacity to higher levels but this is no longer sufficient as systems that scale capacity and much faster than performance are becoming throughput bottlenecks. As disk drive capacity grows much faster than disk drive performance, utilization levels on larger disks will decline further to reduce contention and maintain acceptable levels of performance. Today's disk drives can contain 181GB and roadmaps indicate paths to over 500GB per disk drive in the next 3-5 years. Tape cartridge roadmaps are now targeting 1TB or more native capacity and it is imperative that as capacity increases, performance increases accordingly.

7 Non-Disruptive Changes and Upgrades Will Become Mandatory for all Levels of Computing to Drive Higher Availability Levels

The requirement to modify any piece of the IT infrastructure in a non-disruptive manner is mandatory for most businesses as downtime of any type becomes unacceptable. Vendors with disruptive upgrades and maintenance strategies will be at a major competitive disadvantage.

8 The Issue of Files and Blocks Must End Quickly, and Will

Nearly all Unix, Linux, and Win2K computer systems must deal with two distinct data formats that significantly increase management complexity. While NAS is best suited for file access or non-transaction processing and SANs are best suited for block-level access or transaction processing, the need for a single unified storage topology that handles both files and blocks is badly needed and long overdue. Though some non-transaction oriented databases are running on NAS systems, most NAS systems don't have the processing power to execute the TCP/IP instruction stack at high transaction rates. The new wave of converged block and file storage devices should be well received.

9 The Last-Mile Problem Impacts Over 50% of the World's Digital Data and Will Persist for Several Years Before a De-Facto Standard Arrives

Over 50% of the world's data sits within the last mile and is waiting for a de-facto standard to appear. Satellites (wireless), DSL, and cable are all trying to establish the primary foothold. The last-mile problem refers to the performance problems associated with transmitting information to and from individuals either at home or to those who don't have high-speed fiber broadband access. Cable modem and DSL are the two most popular choices for providing higher speed Internet access and each have an availability index of about 95%, well below standard telephone connections. Presently, cable modem represents the fastest, cheapest, and most reliable solution.

10 The Availability of Electricity Will Become Increasingly Critical

Without electricity, there is no IT industry. The assumption that there will always be a ready supply of electricity (and energy) has to be re-examined given the threat of terrorist activity and the growing demand on energy resources in general. The criticality of insuring electrical energy is available will become critical to the Total Cost of Survival. This past year has brought the arrival of a new and critical factor for the IT sector in the form of a power crisis. Rolling brownouts and scheduled blackouts have become part of the IT availability equation in some areas. Network switching gear, computers and disk drives are the largest electricity consumers. It is estimated that over 50% of all electricity consumed in the United States by the year 2010 will be generated from computing technologies and related information-based appliances. Maybe our refrigerators will order food when supplies are low? Can you imagine a 5TB refrigerator running xGigE? Data centers currently use well overl00 watts per square f oot, more than 10 times that of the average household. Only a few new technologies have been identified to address the energy consumption problem.