Putting information storage in its place: we're generating digital data at a record clip. Is there a digital closet big enough to store it all?

This article is the first in a continuing series from Deloitte & Touche.

From digital photos to e-commerce transactions, our reliance on digitized images, numbers, and words is straining the data storage and management capacity of home PCs and enterprise servers alike. In 1999 alone, we generated 1.5 exabytes (equal to 1 billion gigabytes) of information. According to University of California, Berkeley researchers, we can expect that amount to double every year for the foreseeable future.

This dramatic jump in material to be stored doesn't mean that as a society we're becoming more industrious. Rather, it represents a change in the way we record information, according to professor Peter Lyman, coauthor of a UC Berkeley study released last year that analyzed the world's digital and non-digital data volume. "A lot of that data is literally data--scientific data, for example, as opposed to text or multimedia," he says. "Many things that have been invisible are now being monitored by instruments. For example, economic transactions at the point of sale go into databases now, weather satellites collect data, and inventory changes are monitored by data trails."

While bandwidth-clogging rich media, such as photographs and video, may not represent a huge portion of today's hard disk content, Forrester Research predicts that by 2005, 57 percent of U.S. households will use some form of personal rich media once a month, and video e-mail will replace text messages as the primary means of online communications.

Once we've looked at this information, it has to be either deleted from our hard drives or archived. Much of it is stored, a fact that delivers both an opportunity and a challenge to the information storage industry.

Consolidation Ahead

In the enterprise, digitized information has traditionally been archived on hard disks that are attached to file servers. As PC storage capabilities improve, people are keeping more data on their own PCs. Although disk storage costs have been dropping steadily--today it costs approximately 30 cents per megabyte, half of what it was two years ago and twice as much as it is expected to cost in 2003--hard disk capacity is being tested.

But the real strain isn't a hardware or software problem, it is a manpower problem. According to Don Swatik, vice president, global alliances for EMC, a leading storage vendor, "If you go back five-plus years, the cost of physical storage was 80 percent of the cost of owning the storage. The cost of people to run it, floor space, the power to run it, etc., was maybe 20 percent. Today, it's the reverse. Each time you add another server you have to add more resources to manage the incremental storage. You have to train new storage administrators and get them on board quick enough to keep up with your company's growth demands."

The remedy? Swatik and other industry experts predict that within the next three years, the racks of servers at corporate data centers will be replaced by information plants--storehouses that hold not only archived data, but also data that needs to be accessed on a daily basis. "At the enterprise network level, you're going to have slowdowns in information movement. You have so many touch points with your customer base--your CRM systems, for example--that unless you're taking that information and putting it into storage areas, you won't be able to react to your customers' needs," says Jeff Balentine, a partner at Deloitte & Touche.

Indeed, the hallmark of the information storage plants will be their ability to give users quick access to data. "From the user's standpoint, it will feel like local storage. You'll have the same response time and the same capabilities," says Swatik, who adds that even home computer users will be linked to information storage plants. The information plants now on the drawing board would house storage devices employing technologies that are still being perfected by the storage industry, like data placement optimization. Using data placement optimization schemes, new data--which is typically referenced more frequently than older data--is placed on the disk drive in a more easily accessible location.

Information storage plants won't win everyone over without a hitch. Outsourcing storage brings with it concerns about privacy and security. Advanced security technologies like biometric interfaces, which verify the user's identity by reading his or her retina or fingerprint, might help storage plants enjoy widespread adoption.

Network bandwidth isn't expected to be a roadblock for large enterprises as long as plans to add optic-fiber relays stay on course. But for smaller companies and home offices, it could be a problem. "I don't believe you'll drive fiber to the home or use wireless to get to the home. It's too expensive to put fiber in the ground to the house, and wireless has reliability issues. Adequate spectrum isn't available to the carriers," says Balentine. "We'll use the existing infrastructure and see new compression technologies come into the marketplace. Eventually that will take data from point A to point B."

Early Remedies

Storage vendors and IT departments have been inching toward storage consolidation for the last several years. Today's storage service providers (SSP) could be considered forerunners of tomorrow's information plants, but on a smaller scale. SSPs provide storage capacity and management services to companies whose servers are overloaded.

Enterprises are also implementing technologies like Network Accessed Storage (NAS) and Storage Area Networks (SANs) to consolidate centralized data storage and streamline file access on their networks. With NAS in place, files that used to reside on multiple servers--and in the process, take up valuable space and processing power--an be removed from the servers and be served up by the NAS.

A SAN eases the load on a company's local area networks (LANs). Functions like data backups or data warehouse loading can be done during the workday without compromising the LAN's performance, because the data movement during backups and data warehouse loads occurs on the SAN's own backup network. According to ITCentrix, a single storage administrator can typically manage approximately 200 gigabytes of data. With a SAN solution, which simplifies the administrator's tasks, a single administrator can manage in excess of 3.2 terabytes of data, or 16 times more data.

And finally, the time-honored act of making daily incremental tape backups is in the midst of being streamlined. To ensure that a company's computer system can be rebuilt after a disaster, such as an earthquake or the recent terrorist attacks on the United States, most large enterprises supplement their daily backup. The traditional method has been to back up the entire system onto magnetic tape once a week, and then ship the backup tapes, via truck, to an offsite silo located hundreds of miles away. A major limitation of this process is that it can take several days for the tapes to be shipped back and the system put back online.

EMC and IBM have developed their own proprietary technologies to circumvent the arduous process of tape backup and shipping. EMC's Symmetrix Remote Data Facility (SRDF) and IBM's Peer-to-Peer Remote Copy (PPRC) automate the remote duplication of an enterprise system at regular intervals, on an ongoing basis. "If the main data center in Seattle goes down, the backup data center in New York will come online and act as the primary location," explains Roy Mathew, a senior consultant with Deloitte Consulting. "The end user isn't aware of it because the recovery could happen in a matter of seconds."

What's the biggest challenge in implementing these disaster recovery (DR) systems? "Convincing CIOs and CEOs that a small, unplanned outage can severely impact market position and response time, and that upgrading their DR solution should be at the top of their priority list," says Mathew.

As new technologies for managing our ever-mushrooming mass of digital data continue to be unveiled, it's likely that data storage priority lists will be under scrutiny for some time to come.

Naresh Lakhanpal is the national director of Products & Services at Deloitte & Touche, Technology & Communications Group (Dallas, TX).

www.us.deloitte.com