Alleviating high storage costs while complying with e-Discovery laws.

Many small to mid-sized companies face a seemingly insoluble problem-new eDiscovery laws that mandate they be able to capture and reproduce email traffic, but they cannot afford to comply. They either lack sophisticated archiving software, or simply cannot afford additional storage area network (SAN) storage for Microsoft Exchange. However, new email solutions are emerging that take a very different, and far less expensive, approach to storage.

Exchange's architecture is hard on storage systems because it requires about five systems operations for one send. All these writes, rewrites, and commands clog I/O, and each of those operations is stored in Microsoft's Jet database. As a result, Exchange requires storage with many fast, expensive disks-a requirement that is out of the price range of most small and medium-sized businesses.

To get around these limitations- and to deal with the need for backup windows-Exchange administrators limit the amount of storage available to users and employ fiber-channel-driven, dedicated SANs. While SANs are great storage appliances, they can drive the cost per TB of storage well above $15,000. To compound matters the growth in message storage requirements, and the resulting increase in backup and restore times, is the most serious problems facing messaging management.

Limiting the size of mailboxes isn't a viable solution to the storage problem; it simply forces users to move their messages and other data elsewhere, usually to .pst files stored locally on their individual machines. The inability to access locally stored data leaves many organizations unprepared to meet new requirements for eDiscovery under the December 1, 2006 changes to the Federal Rules of Civil Procedure. Non-compliant organizations risk incurring millions in fines, losing countless IT hours, and damaging their reputations because they cannot respond- or respond quickly enough-to litigation requests for electronic evidence.

Given the storage issue, what should small and mid-sized companies do? Risk not complying with the law or compromise their finances? Neither is an attractive option.

Leveraging Linux to Solve Storage Problems

Today a third option has emerged, one that enables companies to comply with the new eDiscovery laws and pay only about $2,000 per terabyte for storage. This option is a new type of open-source-based email server that provides a plug-compatible alternative to Exchange but has a very different approach to storing data. Such an email server offers a simple solution to storage problems by using the Linux file system for its storage and giving users bottomless email boxes on low-cost commodity storage configured as a live email archive. Because the server requires only a single connect to the file system and uses single-instance storage for both send and receive, it works with a far less expensive processor and requires less space on much lower-cost storage systems. Using the file system as the primary data store instead of a database can also simplify the search and retrieval operation.

This approach to storage thus eliminates the Jet database altogether in favor of leveraging the speed, flexibility, reliability, and efficiency of modern file systems found in Linux operating environments. This approach makes the email server's storage as easy to manage and maintain as any file server's and eliminates the need for an intermediate database that can fragment or become corrupted.

This approach also simplifies the storage model; each user has his or her own folder within the store, and each folder contains subfolders corresponding to such Exchange categories as Calendar, Inbox, Sent Items, and others. Within an individual subfolder, each message is represented by a file. Thus the basic approach of the store is "one file per message." In addition, "single instance" storage for large files means a large attachment sent to many users is stored only once.

Leveraging the file store this way offers a significant performance improvement and potential cost savings. Performance improves because the file system approach, unlike the database approach, does not have multiple levels of indirection between the email and the storage subsystem. This performance improvement allows the organization to choose from a wider array of lower-cost storage options.

"Lean Infrastructure" for Storage

A "lean infrastructure" approach to storage employs direct attached storage (DAS). With DAS, the first level of redundancy is typically provided by a software or hardware RAID-5. An external disk array that supports hot-swapping of drives and RAID-5 enables fast data transfer rates and easy drive replacement if a failure occurs.

Total cost of a DAS storage configuration can be quite modest-as little as $2,000 for one that includes a redundant pair of internal drives that provide 80GB of storage for the operating system and applications and 1500GB of usable storage in an external enclosure that includes a four-way RAID-5 with drive hot-swap. Note that this high-density, low-cost storage is not particularly high performance. However, when this storage is combined with open-systems-based email server software that has been carefully optimized to use storage efficiently, the result is a high-performance system. Contrast this with Exchange, where use of such low-cost storage would deliver poor performance. The combination of low-cost storage and high-performance, low-cost, server software makes it significantly easier for organizations to meet eDiscovery requirements.

IT can configure either Exchange or the open-source email server to forward a duplicate of every email (both inbound and outbound) to the open-source server. IT must configure the server as an eDiscovery archive using standard SMTP-based email forwarding. Access to this secondary archiving server is limited to "privileged" users who are allowed only to search for and retrieve court-required email. This approach costs far less than enabling Exchange's journaling feature. No additional Exchange servers are required, and no Exchange-specific archiving products are needed.

An open email server environment also provides advantages for distribution list handling and for security. Distribution lists are completely expanded on the single server, rather than on separate servers as with Exchange, an approach that leaves open the possibility that a complete distribution list expansion may be missed because of an error on another server. In addition, the open-source server sends the data onward to the archiving/eDiscovery server in a single step before the email is written to the mail store, thus reducing errors and the possibility of dropped messages. Compared with Exchange 2007, the Linux-based Exchange alternative is more flexible and less expensive to operate when configured as a live eDiscovery archive.

Eliminating complex mail storage--SAN, storage cards, configuration issues, implementation time- and leveraging a live eDiscovery archive based on simple commodity storage using a Linux-based email server system gives small and medium-sized organizations a low-cost alternative to meeting eDiscovery requirements.

Scott Young is Vice President of Marketing at PostPath

www.postpath.com