Data growth outruns ability to manage it: just one of the infinite disruptions impacing storage.

Given that the rules of many aspects of our lives are quickly changing, it's appropriate that we take a deeper look into the changes, discontinuities, and disruptions that are underway in the IT and storage networking industry. I had the opportunity to explore these issues with valuable insight from Rob Nieboer, Director of Advanced Marketing at StorageTek, on a recent trip to Asia. We identified eight (and there may be more) well-established discontinuities and disruptive trends that play a significant role in shaping the direction of storage. Even with the Y2K bubble having passed, the dot corn demise, a recession, and events following the Sept. 11th attacks all impacting storage demand, we remain deep into an era of dramatic information creation that is driving storage growth at unprecedented levels,

Disruption # 1--The cost of managing storage hardware ranges from two to ten times the acquisition cost of the storage hardware.

* TCO = 2-10X the acquisition cost of hardware, price per megabyte is not the most critical decision factor.

* Goal: Reduce need for human intervention to manage storage and achieve 100% availability.

* Long-term goal: completely eliminate human intervention in the management process,

* Transfer management burden from humans to software

--Requires "policy-based" management

--Requires outboard and probably artificial intelligence to enable proactive decision-making

* Primary IT expenses moving from hardware to management, facilities, people, and software. Disruption # 2--The value and criticality of data is increasing exponentially while the percentage of data that is actually managed is decreasing.

* Information is the "lifeblood" of most businesses.

* Information is the business for many businesses today, note finance, insurance.

* Productivity in storage management has not nearly kept pace with proliferation and criticality of information.

* Can no longer rely on human intervention for error-free management.

* Must aggressively turn to storage management software to close the growing management gap, just adding hardware capacity creates a bigger problem later on.

Disruption # 3--The demand of skilled IT personnel exceeds the supply of trained IT personnel.

* No college producing storage managers, on-the-job training is the curriculum.

* No human endeavor is error-free.

* Business viability is most often based on IT availability today and will define success and survival for most every business in five years.

* Stakes are too high to contemplate failure.

* Goal remains to eliminate human intervention using advanced storage management functions.

Disruption # 4--The fastest growing storage markets are least equipped to manage growth.

* The environments producing the vast majority of storage growth are the least well-equipped to manage it as the tools and capabilities are playing catch-up with standard mainframe practices

--20+ varieties of Unix--dominated by Solaris, AIX, HP/UX, and SCO 

--Windows NT and 2000

--Unix and Windows markets drive over 80% of all digital storage today

--A typical storage administrator can manage from 300-500GB (disk storage) on a distributed platform to more than 15TB on a mainframe (z/OS or S/390)

--Linux growth rate faster than any storage market hut total size remains small. Linux on the mainframe is gaining strength

Other markets--iSeries (0S/400), Netware, etc. remain steady though are not high growth segments

* The optimal storage solution is probably not coming from server vendors as in the past due to the increase for heterogeneous management requirements.

Disruption # 5--Storage device capacity is growing more than 10 times faster than device performance.

* Law of diminishing returns?

* Storage capacity grows at over 60% per year while performance improves at less than 10% per year. This trend has existed for over 10 years and is expected to continue for the foreseeable future.

* Produces "access gap" and mandates that occur in (at least two dimensions): capacity and performance.

* Access density ratio = I/Os per second per gigabyte. As access density declines, the quality of service declines. Access density also applies to tape technology and is called throughput density. Throughput density is the ratio = tape cartridge capacity (gigabytes) per data rate (megabytes per second).

Disruption # 6--The growth rate of storage exceeds the price erosion rate of storage.

* Storage is growing faster than disk is getting cheaper.

--Storage growth = 60%, 100%, in 3-4 years 600%+ ...?

--IT budgets flat or shrinking

* Storage growth along with associated management, back-up/recovery, and security costs now crashing into the IT budget.

* Remember not to ignore hardware cost even though "cost per unit purchased is approaching zero" as storage prices decline at 30-40% annually. Disruption # 7--Disk and tape cost both converge and diverge.

* Disk costs on 35% to 40% (historical) cost reduction curve based on price per megabyte.

* Tape is on significant cost reduction curve however most observers don't know it.

* But if # cartridges per tape drive reduce dramatically, do disk costs approach tape costs?

--The costs per megabyte of disk and tape storage are fairly close when comparing the cost of a tape drive with an auto-loader and 7-10 cartridges to a disk drive and converge as the ratio of cartridges per tape drive decreases. However, as the number of cartridges per tape drive ratio increases, as is the case with an automated library, the cost per megabyte of tape storage diverges quickly from disk and can be as little as 1/20th the cost per megabyte of disk. This includes the use of compression which typically yields a 2.5 to 3.0 times increase in tape cartridge capacity.

* Is removable, random-access media, like small diameter disks, the trump card for automated libraries?

* Does low bandwidth cost along with wireless access deliver mobility to disk, meaning that disks don't have to be viewed as fixed media any longer?

* Does the access density problem prevent convergence of disk and tape?

Disruption # 8--The cost of energy

* By 2010 estimates for the United States range as high as 50% of all electricity generated will be consumed by computing technologies and IT appliances.

* Is this a case for removable media?

* Data Centers consume over 100 watts/square foot while the average household uses less than 10 watts/square foot.

* Certainly a case for a storage hierarchy and removable storage for less-active data.

* Only active data should consume energy.

* Movement of data from active to inactive status must dynamically occur without human intervention.

The IT industry is firmly entrenched in an era of dramatic storage demand that is growing faster than our ability to effectively manage it. While we are storing digitized data at record-setting rates, we are well beyond our ability to effectively manage this data. At the same time, the most valuable asset of the majority of businesses is its data. Data has clearly become the DNA for the future growth and sustaining competitive advantage for nearly every business entity. Just adding storage hardware as a solution to the management problem, a popular choice since storage consumption is increasing at 60 percent or more annually while costs are falling 30-40 percent annually, actually inhibits or delays implementing effective management policies. This "buy more or manage it" paradox is headed for a collision course at some point in the very near future and could indeed become the infinite disruption for the storage industry without the benefit of some revolutionary developments. This towering challenge now looms ahead of us and will stimulate new thinking and exciting solutions requiring even more insight and, in particular, a compelling vision. Remember, vision is not seeing things as they are; rather it is seeing things as they will be.