Innovation, litigation place increased demand on e-mail archiving.

While network managers struggle to keep pace with the digital content storage requirements, they increasingly need to implement an economic and reliable hierarchy of storage so data can be reliably stored, searched, collated, assembled, analyzed and otherwise used. Organizational and individual data is not only important for making critical business decisions; it is also vital for legal protection.

With daily e-mail traffic expected to almost double by 2006--from 31 billion in 2003 to 60 billion, according to International Data Corporation (IDC)--simply upgrading hard-drive capacity to meet this demand is not only expensive but is impossible to manage. According to Forrester Research, each e-mail is an average of 59KB and the daily flow of e-mails worldwide is currently 1,829 terabytes. In one year, the total will grow to 3.35 petabytes.

The volume of e-mails that are being generated is fueling the growth of new e-mail search solutions (such as Google's and Yahoo's recent acquisitions), more robust corporate e-mail archiving solutions and the increased importance of nearline storage systems. The opportunity for most integrators to tap into this market was recently highlighted by Radicati Group analysts who noted that despite the continued rise in corporate e-mail traffic and volume, only about one-fourth of the companies surveyed have adopted a formal e-mail archiving solution.

According to Dr. Victor R. McCrary, Business Area Executive for Science and Technology at the Johns Hopkins University Applied Physics Laboratory, digital information preservation has become a major challenge for storage mangers and CTOs who struggle to meet regulatory and budgetary requirements. While virtual storage and SAN (storage area network) are being hyped as the ideal solution to the dilemma, they usually incorporate only disk subsystems, which only address part of the storage puzzle. Dr. McCrary and others emphasize that the best (most reliable and most economic) solutions encompass a range of device/media types to deliver the optimum solution for ease of use, scalability and cost effectiveness.

According to recent studies, 80% of online information is static and unused. This is especially true of critical data sitting on notebook and desktop systems at the outer edges of the network where the information is least protected. Rather than continuing the challenge of centralizing this storage. IT managers are implementing individual and departmental backup/archive activities using products such as NTI's BackupNOW! that automatically save e-mail to CD or DVD and then migrating the discs to increasingly centralized optical libraries.

Storage virtualization becomes increasingly vital for the migration and archiving of e-mails to lower-cost storage solutions such as optical libraries. The ability to remove storage media (cartridges) from the device and store it away from the primary environment to maintain capacity and improve server performance is not only logical and economical; it is also compliant with legal requirements for storing e-mail.

Today, the IT administrator spends an estimated 40% of his or her budget on storage. A number of proven optical storage solutions are now in the mainstream that enables them to control this "mix and match" storage environment. These libraries--MO, DVD (+/-R, RW, RAM, DL) as well as next generation UDO, Blu-Ray and HD DVD--can deliver some of the most efficient and most effective centralized storage.

This approach enables firms to significantly lower their distributed storage management administration costs while migrating static data to lower-cost nearline or offline optical storage systems. IDC has also found that this approach also enables IT management to reclaim a large amount of their current online storage investment.

All data (especially e-mail) is not created equal and should not be treated equally. Disk subsystems are well suited as primary storage; but e-mail archiving is one of the applications that is ideally suited for secondary optical storage. What's needed is an approach to solving the tradeoffs between capacity, access speed and cost.

Storage solutions (HD or optical-based) don't exist in a horizontal or vertical scaleable plane. As IDC has so succinctly illustrated, the business users' priorities and requirements exist in three dimensions. Understanding the market, application and relative importance of each of the requirements will undoubtedly be more important than the specific storage technology.

As the mountains of data seem to increase exponentially on a quarterly basis, business and IT management focus on making the information useable so they can make effective decisions in real-time. In this aggressive environment, optical technology is proving its value as a high-capacity, long data life, random-access writable technology.

Organizations need to choose the best archiving solutions based on their data access requirements and their legal exposure for storing data. The key for integrators is to provide users with the blended solution that delivers the highest levels of performance, data security, and cost-effectiveness. As Microsoft and others facing legal action have found, the ability to manage and track information (especially e-mail documents) across the organization has become extremely important.

Data that is used and/or changed frequently belongs on magnetic disk where access speed is critical. However, the vast majority of data on RAID arrays is not accessed frequently. Static data such as e-mail must be offloaded to more cost-effective optical storage media and libraries. This managed approach enables IT management to not only manage their growing data storage requirements but also decrease their operating downtime and dramatically reduce the chances of lost or corrupted data. The challenge today is to maximize the benefits of corporate data assets throughout their data lifecyles on the most cost-effective and reliable medium available. Optical (MO, CD and DVD) is becoming increasingly important as a key component in today's virtual storage picture--especially for organizations concerned about data permanence, security, litigation and keeping costs in check.

G.A. "Andy" Marken is president of Marken Communications Inc.

[e-mail]