Anatomy of a Serial ATA storage enclosure.

As a company's demand for on-line and near-line data storage grows, the job of balancing the cost of equipment against performance and reliability becomes more and more challenging. New storage products based on Serial ATA architecture are promising lower costs while still providing new levels of high performance. Of course, as with the other storage systems, all SATA-based storage enclosures are not alike. Ideally, if all these systems were built the same, the selection process would be greatly simplified and the acquisition could be based on cost and features. Unfortunately, this is not the case.

The design and construction of these storage enclosures are as varied as the number of manufacturers that produce them. Some of these manufacturers design storage solutions with great care and intricacies while others produce products that are no more than sheet metal boxes haphazardly loaded with a bunch of inexpensive disk drives. It is important for those tasked with acquiring data storage systems to understand that a properly constructed storage enclosure needs to meet certain levels of structural, electrical, and environmental integrity. If these levels are not met, the resulting consequences can appear in a number of unappealing, poor performing, and short-lived systems. By having a good, fundamental understanding of what constitutes a properly built storage enclosure, a lot of these issues can be avoided and the expectations of performance, reliability, and low system cost can be attained.

Serial ATA storage enclosures are built in variety of sizes and shapes, depending on the application. Some are designed for desktop or deskside applications while others are designed to be mounted into racks. The physical bits and pieces that constitute a SATA enclosure are referred to as the enclosure's mechanicals and have a major influence on a product's cost, performance, and reliability. There are a number of items that should be checked and verified when evaluating the mechanicals of a storage system. Some of the more important and relevant of these include:

* Verifying that the overall structure of the enclosure is sufficiently rigid and soundly constructed

* For rack-mounted configurations, confirming that the enclosure dimensions are within the parameters specified by EIA standard, RS-310-D

* Ensuring that the hard disk drive (HDD) carrier is built robustly with good isolation for the disk drive and that its mechanical interface to the storage enclosure is not loose or sloppy 

* Confirming that any removable disk drive carrier(s) or customer replaceable component(s) function properly for mechanical swap

* Verifying that all circuit board modules fit correctly within the storage enclosure and are properly grounded

* Confirming that electromagnetic interference (EMI) shielding is properly installed and making contact with mating surfaces

* Ensuring that all assembled components are properly secured into the enclosure and that there are no loose or ill-fitting parts.

Although the reasons for examining these points seem fairly obvious, it is worth expanding on them to better understand their relevance.

[FIGURE 1 OMITTED]

Enclosure Structure

There are a number of reasons why having a rigid, soundly constructed enclosure is important. One is that a properly built enclosure tends to prevent or restrict the generation of disruptive noises and vibrations. If not properly confined, vibrations from internal sources like cooling fans can migrate into sensitive areas such as the disk drive bays. If these vibrations are significant enough they will affect a drive's ability to accurately position its read/write heads over a requested disk location. Depending on the frequency and magnitude of the vibration, the resulting problems from this can range from mild system performance degradation to catastrophic loss of data. It is worth noting that the disk drives themselves can also be a source of unwanted vibration. The rotary actuators used to position the heads over the disks tend to be the culprits in this situation. In a heavy I/O environment this can be the source of some considerably large vibrations. This phenomenon is referred to in the storage industry as Rotational Vibration (RV) and the effects can be just as disastrous as those previously mentioned.

In addition to better controlling vibration, a well-constructed, rigidly built storage enclosure will also ensure minimal body flexing at key interface points. This is particularly important when rack mounting a system. All enclosures sag from their own weight when mounted into a rack. A rigidly constructed system will have minimal sag with little or no impact to the operation of the enclosure. A poorly or improperly built storage enclosure will have a large amount of sag that will adversely impact the fit and operation of the system in a number of areas. One of the areas most affected by a large amount of sag is the fit of disk drive carriers and other customer removable components. If not properly supported, the bays that these items fit into become distorted making insertion and extraction of the devices difficult. This distortion can also adversely affect the EMI sealing of these removable components, which in turn can produce significant leak points for EMI emissions. Since rack-mounted systems must fit within a specified vertical envelope, having a large amount of sag can also cause the enclosure to breach this envelope and affect adjacent systems.

Construction and Fit of HDD Carrier

A hard drive's I/O rate is greatly influenced by the magnitude and frequency of vibrations created both internally in the drive and those generated externally inside the enclosure. Therefore, it is vital that the HDD carrier that the drive is mounted into be capable of isolating itself from external vibrations like those generated by high-speed cooling fans. It is equally important that the carrier be able to dampen the vibrations created by the high frequency I/Os and high-speed spindle of its own drive. Isolating these drive-induced vibrations will prevent them from migrating and affecting neighboring drives. An HDD carrier designed with a thin sheet metal shell and some stamped grounding dimples will more than likely not be able to provide sufficient isolation to the drive. If this is the case, then the overall performance and reliability of the storage enclosures will be affected by degradation in the drives' I/O rates--or even possible data loss.

Not only is it important that the HDD carrier be built correctly but the fit within its mating enclosure bay must be proper and secure as well. A loose or bad fitting carrier can cause a number of problems that could affect the performance of the system or the integrity of the stored data. A loose carrier can potentially amplify the adverse affects of performance-degrading vibrations. It can also cause problems at the drive's connector interface by permitting relative movement between the drive and the printed circuit board the drive plugs into. This can cause an intermediate signal problem or prematurely wear-out the connector system. Several enclosure manufacturers have addressed this loose carrier problem by incorporating a pre-load mechanism into the carrier. Once the drive carrier is slid into place and latched, a fore-aft pre-load is applied and movement becomes greatly restricted. Once restricted, the associated problems from drive carrier movement are all but eliminated.

[FIGURE 2 OMITTED]

Fit of Circuit Board Modules

Depending on how the enclosure is laid out and configured, there can be a number of printed circuit boards (PCBs) within a storage system. Figure 1 shows an example of a possible SATA storage enclosure and illustrates just how many module boards could be encountered.

It is important to verify that all circuit boards within the enclosure fit correctly and that they are grounded properly. An ill-fitting board, especially one that connects to other PCBs, can cause alignment problems or excessive loading at connector interfaces. The results of this situation can adversely impact the system's performance and reliability over time. It is equally important that all module boards within the SATA enclosure be grounded in some manner.

Examining a SATA Enclosure's Cooling

Many of today's SATA enclosures strive to insert maximum amounts of storage and processor power into a minimum amount of space. As manufacturers succeed at this high-density integration on the physical side, they are finding that controlling the high thermals generated within these enclosures to be a more difficult and daunting challenge. Disk drives, high-speed processor chips, and power supplies are but a few of the sources responsible for the high temperatures seen within these storage systems. If not kept under control, excessively high heat can cause accelerated wear of internal electronic and mechanical components. Adding more fans to the system would seem to be an obvious and simple remedy to this problem. Unfortunately, space constraints typically restrict the size and number of blowers or fans so much that just getting enough cooling devices to keep the enclosure from melting is considered a victory by the designers.

To effectively cool today's storage systems really requires the use of lower volume, higher static pressure cooling devices. After all, it is static pressure that moves the airflow inside a chassis--especially in tight spaces like those found between the hard drive bays. A larger fan with a high volume rating but lower pressure will not be as effective as a smaller fan with a slightly lower volume but much higher pressure rating.

One of the best cooling layouts for an enclosure is an exhaust-only cooling system. This configuration consists of fans or blowers mounted strictly at the rear of the chassis and does not use other intermediate cooling devices like fans mounted on a central, internal bulkhead. With the fans positioned at this location, re-circulation of heated air inside the chassis is virtually eliminated. This type of layout also allows for greater control of the airflow within the enclosure. This is accomplished through strategic placement of baffles and intake vents. Having an exhaust-only cooling system is certainly preferred but often cannot be achieved due to the limited available space on the chassis's rear panel. In these cases, cooling fans are typically mounted internally towards the center of the chassis and provide a positive-pressure cooling arrangement to the rear half of the enclosure. It is necessary, with this kind of configuration, that the fans be mounted to a sealed bulkhead to prevent any air from re-circulating back to the less-pressurized front half of the chassis. Depending on the form factor of the chassis and space available, a variety of fans and blowers can be used for this layout to obtain a more efficient cooling design.

In addition to the fans, the enclosure's intake and the manner by which the airflow is channeled through the chassis are also very important. For a high drive-count storage enclosure where the majority of the system's intake comes through the drive bays, it's critical that the drives be spaced far apart, and the carrier bezels have as much intake area as possible. In instances where the enclosure's backplane (the module board that the disk drives plug directly into) extends across the full width of the system, it is important that air flow holes be present across the backplane and that their combined area be comparable to the combined area of the intake. Anything less than this will cause impedance within the enclosure and restrict the amount of cooling air available to the rear of the chassis.

Examining a SATA Enclosure's Electricals

The electricals within a Serial ATA storage enclosure are generally divided into two areas: those associated with the system's data signal and those associated with the enclosure's power. Evaluating a SATA enclosure's power normally consists of verifying that the power provided to the system is sufficient to support all its electrical needs over a range of demands. Examining the system's ability and efficiency to handle its data signal requires an evaluation more focused on performance and data integrity. Although the individual electrical components used within a storage enclosure are very much the same from product to product, the way and manner in which they are integrated ultimately dictates the performance and signal integrity of the storage enclosure.

[FIGURE 3 OMITTED]

Powering the System

The internal power of a Serial ATA storage enclosure is typically provided by one or more power supplies. Some enclosures offer customer removable power supplies that may or may not be hot-swappable. A storage system with redundant power supplies provides a more reliable environment by eliminating the problem of having a single point of failure. This kind of option normally comes at price and is only effective if each power supply is plugged into an independent circuit at the wall. Figure 2 is an example of a dual-power supply enclosure.

Serial ATA Data Signal

As the name infers, Serial ATA is a serial architecture that moves data from a point-to-point basis. SATA creates packets of data and transfers them at high speed (1.5 Ghz) across specially built cable/connector assemblies to and from the host. Serial ATA integrates Cyclic Redundancy Checking (CRC) on both command and data packet levels for enhanced bus reliability. Through its cyclic redundancy code, SATA is able to detect all single and double-bit errors, making it a very robust and reliable architecture.

In order for a Serial ATA disk drive to achieve its full transfer data rate of 150 Mbytes/sec, it must use specially designed cables or printed circuit boards. The length of distance between the SATA drive and its controller is limited to 1 meter in cable applications. A single signal cable consists of 8 individual wires (4 signal lines and 4 ground lines). These wires are carefully cut to identical lengths (to minimize skew) and soldered to a 7-contact connector (the middle two grounds are connected together). An overmold is then placed around the cable/connector assembly for strain-relief. Figure 3 shows one end of an assembled single SATA signal cable.

Although the width of these cables is relatively narrow (7.9mm/0.31" across the flat), the materials used in their construction create a rather stiff composite. This can create some difficulty when it comes to routing these cables in tight areas that require small bend radii. This routing can become even more challenging if there are a large number of signal cables involved, such as in a 12-drive, 2U enclosure.

A new, innovative solution for being able to manage a large number of SATA signal cables has been introduced. Based on the InfiniBand cable system (SFF-8470), these signal cables combine 4 individual cables into one. Referring back to the previous 12-drive example, the number of multiport cables needed to support this enclosure would now only be three--a fairly significant reduction from the previous count of 12. By reducing the number of cables within the enclosure, not only is the issue of routing reduced but an improvement in the system's cooling (from fewer blockages) is achieved as well. There are several enclosure manufacturers that are just coming to market with Serial ATA enclosures based on this multi-port technology. On the controller side, 3ware, Inc. has just introduced their Escalade 9500S Series of Multi-lane Internal (MI) controllers. 3ware's twelve-port, multi-lane controller--the Escalade 9500S-12MI--has a 4-port cable.

Figure 4 shows an example of a modular SATA backplane from AIC (Advanced Industrial Computers). This backplane supports four hard drives and utilizes a four-port, multi-lane connector system.

The frame that the backplane is mounted to allows up to three of these modular boards to be attached.

A storage enclosure's overall performance and its data integrity is very much dependent on the I/O performance of the system's disk drives. The controller card used and available I/O bandwidth can obviously play a very important role, but it is the hard drive's I/O rate that is most influenced by the structure of the enclosure itself.

[FIGURE 4 OMITTED]

Multi-lane Internal (MI) controllers increase reliability, and simplify installation by significantly reducing the number of required connection cables.

Conclusion

Serial ATA solutions are enterprise-ready and are appealing to business and vertical markets because it combines performance, capacity and affordability. With the number and variety of Serial ATA storage enclosures on the rise, the task of deciding which products should be considered and which should be avoided becomes challenging. Although low cost can be a strong lure, a system's reliability is ultimately what needs to be considered most important. Ensuring that this reliability is attained requires that the mechanicals and electricals of a potential storage enclosure candidate be examined and evaluated. The enclosure's structure. fit of removable components, and quality of assembly are all important points that must be inspected. Likewise, the system's ability to meet its internal power needs and effectively route data signals inside and outside the enclosure are all areas that should be scrutinized. Lastly, it is important that the RAID options being offered with the storage system be reviewed and that the advantages and disadvantages of these options be understood.

After all, data is priceless and building the right storage solution will protect the digital assets of any company, day after day.

www.3ware.com

www.aicipc.com

J. Peter Herz is senior vice president of Business Development at 3ware (Sunnyvale, CA)