New Strategies In Performance Storage Solutions.

It is a very exciting time for the disk storage systems market. With the growth of video data content, music files, high resolution photographs, etc., there appears to be a never-ending demand for more and more disk storage solutions. The Internet has created vast depositories of information all over the world which contain every imaginable form of data. Once we talked mainly of gigabytes, then terabytes, and now it is petabytes. Where will it end? It is difficult to see any slow down at this time, so we will soon be talking about exabytes and zetabytes. The re-purposing of data into digital formats for Internet consumption is one of many significant factors in this exploding demand for disk storage solutions.

Complementary product enhancements have helped pave the way for the dissemination of these large files. Host processors, of course, are continually improving their handling of large video, music, and picture files. The delivery transport has improved with the availability of DSL, cable modems, and more affordable T1 lines. Performance fibre networks to support large databases are improving and becoming more cost effective, and, disk technology continues to advance in speed, storage capacities, and performance user interfaces.

Changing Storage Requirements

With these new tools available, it is time for a new look at the architecture of current storage products today and how they could be enhanced to meet the challenging growing needs of the video, music, download program files and picture storage. The challenge for storage solutions is to handle vast amounts of large block files as efficiently as possible. Even heavily compressed music and picture files can be multiple megabytes each, and video files can be gigabytes. The demand today is for even greater picture and video quality which will drive the file sizes up by a minimum of four times.

The requirements for large block files on the surface sounds a lot like any storage application, but there is a twist to the requirements that cannot be overlooked. These requirements include:

* Faster throughput

* Guaranteed performance (real-time)

* Higher storage capacities

* Scaleable configurations which build throughput as well as capacities

* Redundant/protected data

* Lower costs per gigabyte

Faster throughput means the ability to deliver large files in the least amount of time. This is a bandwidth issue as well as an I/O issue, since video, music, pictures/photographs, and program downloads are contiguous transfer applications. Moving a large file typically clogs the path over the delivery channel. The key is therefore to move the file through as fast as possible so that the path is clear for the next transfer request.

Guaranteed performance is actually an extension of faster throughput. For the video or music creation application, guaranteed performance means that the content can be reviewed without interruption. For the information/content provider, it means that the provider can get maximum utilization of the system hardware. For example, if a supplier is paid by the amount of data he or she delivers, losing throughput capability by 25% in the event of a drive failure means that his revenue stream will also be cut by 25% until that failed drive is replaced. This is an extremely important issue for Internet product providers.

Video, pictures, and music files are storage hogs, with video being the guiltiest culprit. A standard length, medium resolution music file is easily 2MB. A photograph of average resolution is 14MB. A video file stored in low resolution MPEG1 format is over 1GB, while a DVD quality MPEG2 file averages 3GB.

Scaling storage systems to meet higher throughput as well as capacity requirements means that parallel access must be easily facilitated by the storage system. Piling up more capacity behind the same delivery path is no help, once the data path is maxed out. Parallel (multiple) data paths to the storage system allow not only higher I/O access, but higher throughput capability, and can insure higher guaranteed performance.

Redundant capability in storage solutions for large block applications is extremely important. You can reload a text file in seconds, but to reload a single DVD quality video from a tape backup system requires 10-15 minutes. For the poor uncompressed video user, that backup requires six hours for an average length, standard definition video. If the database is a collection of hundreds of videos, the restore time is not an attractive activity to think about.

Lower cost storage is a term that seems to be filler material in this article, but actually, if storage costs are not minimized for large file applications, it can become economically problematic. The best example I remember is a request from a customer at a trade show approximately four years ago. This gentleman had a collection of videos/films of old television shows that he wanted to preserve and have available via computer monitor. He had approximately 1200 episodes that he wanted to digitize and store on disk. Additionally, he needed the resulting digital content to be quality equal to that of the original videos, because he needed this digital content to be the new master, since the videos and film were deteriorating. If each video was a 30 minute episode and it was stored in standard uncompressed form (Dl format) the required disk storage would be approximately 45TB. Four years ago, with the prevailing disk storage capacities at 9GB and a cost of $1,000 per disk (Cheetah Variety) the customers' raw dis k costs would be approximately $5 million. He decided to wait. Another very popular example is the video-on-demand fiasco that peaked for the first time approximately seven years ago. This application was impacted greatly by the economics of disk stored digital video.

A Solution For Consideration

One approach to solving the growing large block storage needs is the "module" solution. This different approach to handling data has many advantages over today's large expensive storage arrays. The module is a physically small storage block, which appears as a virtual single high performance drive although it may have multiple physical drives inside. The interface to the module should be one that easily facilitates scalability and is matched in performance to that of the total throughput potential of the virtual disk storage inside. Fibre Channel is such a bus that meets these requirements. The module must be capable of handling large block files without interruption, and it must be constructed using the most cost effective elements possible. Also the module element itself should protect the valuable content if it is to be a complete scalable building block for small to large applications.

Configuring The Solution

For data intensive large block databases, the storage media that should be used is IDE or ATA disks. This typical desktop home computer product is by far the lowest cost hard disk drive on the planet. Looking back at my video customer with the 15TB requirement, we find that today's current raw disk costs using IDE product would be $56,000. Darn, I could have had that sale! In contrast, raw disk costs for performance Fibre Channel or SCSI disks would cost my customer $200,000. Probably still a sale, but I would not want to be the competitor with the $200K solution if my opponent was offering the $56K option.

I know what comes to mind about now: "My customer would not want to settle for a low performance low reliability desktop, home computer disk solution. He needs performance, quality, and integrity of data." However, the ATA drives of the past are not the same ATA drives of present. At 80GB per disk, they actually exceed the storage capacity of today's Fibre Channel and SCSI disks, which are currently 73GB. IDE disks are fast, with media rates off the disks of over 45 MB/sec. That is 89% of the speed of the fastest Fibre Channel or SCSI disks of similar capacity. Fibre Channel and SCSI disks have the edge in rotational speed of 10K RPM over the IDE disk's current 7200 RPM capability, but in large block applications, that difference is also nearly undetectable. As for reliability, IDE disk MTBF calculations have crept up over the years to the point that they are approaching 1 million hours, like their SCSI and Fibre Channel counterparts.

The final potential objection to IDE disk use, which must be overcome, is the error handling of IDE disks. In this area, the SCSI disk is more intelligent with superior error handling capabilities. Error handling on the current IDE disk is slower and less robust. To handle this possible objection the RAID controller handling the IDE disks must compensate for the IDE drive by providing error handling enhancements for the overall storage solution. This can not only be a neutralizing factor, but an opportunity for error handling superiority. The controller can be designed to offload any error handling routines that would be done by the disk or the host itself. By using the redundant drive, time wasting retries can be eliminated by enabling real-time correction on the controller for failed or slow responding disks. Additionally, the RAID controller can add performance enhancements to the system through the use of CTQ handling and specialized segmented cache buffers for video-streaming applications. This allows th e resulting RAID storage solution to surpass the capabilities of current more expensive arrays for large block applications.

The controller serves another function in that it becomes the bridge from IDE disk drives to the desired Fibre Channel host interface. The controller can provide multiple FCAL ports eliminating the need for hubs in smaller shared storage solutions. Fibre Channel provides a vehicle to facilitate maximum throughput from multiple IDE drives to the host or storage network. Additionally, with the use of Fibre Channel as the host interface, external hub and switch technologies can be combined for larger applications, allowing a large number of ports for increased accessibility or I/O and parallel streams for increased throughput or bandwidth. If constructed correctly, each port from the RAID to the switch will be maxed out so that all performance inherent in the configuration is realized.

In the module strategy, the IDE disks and controller are encapsulated in a small module or block so that they can be configured either alone on desktop or side-by-side in a rack for high density storage. The module is totally self contained, including internal power and RAID protection, so that it appears to the host as a very large, fast, error free fibre channel disk with multiple ports, except that the resulting solutions costs half that of comparable Fibre Channel disk solutions.

Configuring For The Application

Using the module as a "super disk," configurations for large block applications can be handled easily. No host based striping software is required to build bandwidth configurations. One module with multiple Fibre Channel ports can become a performance entry sever. Through the use of a fabric switch or switched hub, a single module can be added to each port to provide scalability in both performance and capacity. In combined solutions, standard RAID technology can work alongside modules in a switched network, allowing the module to handle the large block accesses while traditional I/O intensive RAID products handle the lower capacity, smaller record files.

When addressing storage applications for large blocks, such as files for download, photographs, music files, and video streaming, the storage capacity required can be astronomical. Selecting the lowest cost disk media for the system solution will be key to the acceptability of the solution. When combined with enhanced RAID technology, low cost IDE drives can be used effectively, matching or even exceeding the performance of higher cost SCSI and Fibre Channel disk storage systems. When combined in a small module, scalability of system design can allow economical performance storage solutions for data intensive applications.

Martin Bock is the president of Storage Concepts (Tustin, CA).