A surge of storage innovations: gaze into the crystal ball. (Storage Networking).

The issues surrounding storage management seem to grow every day. Storage demand has historically outpaced the capabilities of the associated storage management infrastructure and technology innovations to keep pace. Many new advances are aiming at changing landscape of storage networking and its management.

Disk Could Exceed 1TB

The dominant and most popular storage technology is magnetic disk, as it contains nearly all of the world's mission-critical data. With the first disk appearing in 1956, the disk industry now totals $25 billion annually, though down from its highest level of $31 billion worldwide market in 2001. The annual demand for disk storage is expected to range between 50-70% through 2005, while annual revenues are expected to grow at no more than 5%. Entering the third era of disk evolution, the capacity of a single disk drive has grown to nearly 200GB, 40,000 times greater than the first 24-inch-platter, 5MB drive in 1956. Low-cost RAID devices using the low-cost serial IDE interface are being implemented in a growing variety of storage appliances and blade servers addressing cost sensitive and lower performance markets. The future success of disk storage demands vastly improved storage management, widespread use of data compression, disk subsystems with embedded intelligence, and virtualization functionality just abo ut everywhere. Storage servers are emerging primarily using silicon components that enable parallel processing are positioned to overcome many of the remaining barriers of traditional architectures. These advances will become mandatory as native drive capacities should exceed 1,000GB or ITB in five years, while the raw performance of disk drives will not be able to keep pace with the capacity growth. Nonetheless, disk capacity is advancing at unprecedented rates and the end of this evolutionary trajectory is nowhere in sight.

Tape to Grow Up to 10%

The magnetic tape industry is now in its third era since the first successful tape drive appeared in 1952. After 35 years of manual tape operations and 15 years of automated robotic tape handling, the tape industry is moving into the era of intelligent tape. With a $7 billion worldwide market, including drives, media, automated loader and libraries, the tape industry has more than 10 different recording formats and nearly 15 automated tape library suppliers. Native cartridge capacities now reach 200GB and are expected to approach 2TB in five years given identified recording density improvements. The projected cartridge capacity improvements are expected to keep pace with or surpass those of the disk drive industry. Automated tape and library market revenues are expected to grow between 5-10% annually through 2005 as disaster-recovery issues, legal and archival data-retention requirements mount. Many new digital applications are beginning to take advantage of the new ultra-high tape capacities and throughput r ates. Embedded disk caching using IDE appliances, tape arrays, logical WORM (write-once-read-many) and more intelligent cartridges provide large-capacity, automated tape storage reservoirs with new functionality serving a wider range of applications. This presently has appeal as automated tape storage is currently priced from one-fourth to less than one-tenth the cost of magnetic disk storage.

Challenges of Storage Management

The current storage management gap between what is managed and what needs to be managed is huge and will become insurmountable without profound change in the capability of management tools and in storage networking operations control. Today, a "typical" distributed storage environment (Unix, Win2K and Linux) has between 2-5TB of disk storage, while the management capability of each storage administrator averages from 400-500GB. Storage demand is easily outpacing the rate of improvement in storage management tools and the result is an ever-widening gap between data that is managed and data that is not. In addition, the

larger capacity disk drives are beginning to experience lower utilization numbers in efforts to maintain acceptable performance levels as the access density problem looms. Projections out to 2005 indicate that the average administrator will be able to manage between 4-5TB of data, while the typical site capacity will have grown to over 50TB. Adding more administrators is an unlikely option for m ost businesses given the shortage of skilled personnel. As a result of the growing management gap, businesses will be forced to manage only the data that really matters unless storage-management technology makes dramatic strides. Storage area networks (SAN), network-attached storage (NAS) aggregation, virtualization, consolidation and unified storage all improve the basic management capability. Data security will be soon added to list of required storage management disciplines needed to survive the decade ahead.

Magnetic Recording Advancements

Technology advancement remains at the core of the storage industry. Since the mid-1990s, magnetic recording density has outpaced optical recording by a factor of 10 to one annually, essentially squeezing optical from the storage hierarchy. Recording head technology improvements over the past decade have brought the physical distance between the head and the disk surface below 10 nanometers, resulting in smaller bits on narrower tracks. Decreasing the size of a bit cell by a factor of two quadruples the density. Areal density, the product of track density and the number of bits per track, has increased by a factor of three million in just over 40 years. The tremendous capacity increases for storage have not yet been offset by corresponding performance improvements. The biggest future architectural shift is expected to come in the form of the movement of most storage functionality away from the server and into the storage network. Implementing or embedding software and virtualization functionality into hardware , also called "offload engines", will accelerate minimizing latency for disk drives, switches and the coming wave of hyper-routers. This will specifically take virtualization to the next level making it a widespread management solution. A more distant glimpse into the future suggests that photons (light) may replace electrons (electricity) in an increasing number of storage and computing functions.

Storage Networking

Storage networking represents the emerging model for storage subsystems and consists of SANs, NAS, DAS, IP Storage, switches, virtualization, storage management, the network interfaces, bandwidth and the associated storage appliances. Numerous new storage and network interfaces are being developed to move the industry away from an aging base of legacy interfaces. History has consistently shown that moving away from de-facto standard legacy interfaces occurs far more slowly than expected. The IP storage family consisting of iSCSI, iFCP and FC IP that enable running block-level data over IP networks are all overdue and will add to the capability of the base SCSI and Fibre Channel interfaces, not replace them. Each will have to lean more heavily on hardware-based offload engines to achieve acceptable performance levels in order to become successful.

Bandwidth Feast and Famine

Assembling the pieces of the storage networking puzzle together into a workable solution has made interoperability possibly the biggest storage-networking challenge and one that may not be met anytime soon. Bandwidth is abundant everywhere except to your home and the price of bandwidth is now falling at over 50% per year. In Europe and the United States, the supply of bandwidth exceeds demand between two or three times. With bandwidth consumption much lower than planned, much of the unlit (unused) fiber or "dark fiber" may already be obsolete and not capable of being lit. With costs from $250,000 to $1 million per mile to dig, lay conduit and pull fibers, a shortage in fiber could occur again and prices could actually rise in attempts to lay upgraded fiber. The landscape is changing every day, the time to prepare is now.