Re-generating storage demand in 2004.

Only a few years ago, the storage industry was one of the hottest and fastest growing of all business segments. The prolonged economic slump that began in mid-2000 has slowed storage growth rates to about half of their peak and shrunk storage hardware revenues nearly 30% in just over 3 years. Analysts, businesses, vendors, investors and economists are asking what it will take to bring the excitement back to the storage industry. It is unlikely that the "irrational economic exuberance" of the dot.com era will return, but there are some signs that storage growth is poised to return. This time, however, it will be based on factors that were not a significant component of the past demand-generation phase.

ILM

Though ILM (Information Lifecycle Management) is now a popular topic and becoming a widespread activity, is not a new concept. Businesses have been attempting to manage their data throughout its lifecycle for years with varying degrees of success whether it meant backup, archive, migration, or deleting data. As data ages, it is assigned different priorities and some gets stored on less-expensive storage devices, depending on how often it's used or potentially needed. Hierarchical storage is only one piece of a successful ILM strategy, however, ILM also needs and is receiving policy-driven data classification capabilities as well as the capability of non-disruptive movement of data between storage devices when data is infrequently used or deleted.

Over the next five years, a successful ILM implementation is estimated to add from 10% to 25% annual demand for storage, depending on the application or business, Compliance adds to the ILM equation but ILM is more than compliance.

Compliance

Many new government regulations, the Sarbanes-Oxley Act, and HIPAA requirements for transmission and retention of data have made us change the way we manage data as it ages. Several major health care providers indicate they will be potentially generating and storing in excess of 500TBs of patient medical data over the next few years that will need to be managed and retained for a person's lifetime plus seven years. This time period could often exceed 100 years. SEC rule 17a-4(t) mandates several new digital archiving requirements, including what type of storage format should be used, how long data must be retained, and where and how long duplicate copies of data must be stored, as well as specifying security policies. The back-end of the data lifecycle is swelling not shrinking, as was the case previously, and retention policies are now being based on data value and legality issues, not just reference activity. This change in the storage landscape calls for new management policies based on the value of data and means that a universal, standard classification scheme for data needs to emerge. All data is not created equal and storage management vendors are working on advanced data classification schemes to create a value for specific data. Compliance is expected to generate as much as 20% annual storage demand depending on the business.

Content, Information, and ...

An updated version of the landmark Berkeley study from 1999 has been just been released and serves as the most in-depth source of information for the question of how much new information is created (versus stored) each year. (Note that storing data can entail creating multiple copies and combinations of previously existing data increasing storage requirements but not creating new data.) Content is generated by many means including telephone calls, pictures, video, television, radio, satellites, the Internet, and e-mail. The new Berkeley study estimates that as much as 5 exabytes of new information was created in 2002 with 92% being stored on magnetic media. The U.S. produced about 40% of the world's newly created information. Film was estimated to store 7% of the total, paper .01%, and optical media just .002%. Based on a global population of 6.3 billion people, nearly 800 megabytes of data is being generated per person each year. New generated information grew just 30% per year between 1999 and 2002, overlapping much of the economic downturn.

E-mail

E-mail now accounts for nearly 8% of the new digital data created, generating about 400 petabytes per year. If spam were saved, these numbers would soar by more than 60% annually. Spam is estimated to be 62% of e-mail. Managing e-mail is different than storing e-mail. Most businesses do not have a means to prevent users from modifying or deleting e-mail messages. Effective e-mail management also raises the issue of security to new levels since e-mails now contain much of the confidential data of a given business.

Security

Data security has quickly evolved to encompass more than just backup, recovery, and HSM or data migration functionality. The growing intrusion factor from hackers, viruses, unwanted third parties, and other malicious types is making encryption and authentication for networks at the perimeter and storage subsystems mandatory for nearly every business. Government regulations are also driving businesses to implement much more effective security measures. Though the amount of security-related meta-data that will be generated remains difficult to estimate, the demand for storage-related security appliances within the storage and network infrastructure will accelerate. Look for security--both security meta-data and security appliances--to become a growth component to the overall storage industry revenues in 2004 and beyond.