The challenges of testing SATA and SAS: part 3: testing wide SAS.
One of the first obstacles to testing SAS devices is getting an analyzer connected into the point-to-point bus. The similarities between SAS and FC have made it possible to borrow from Fibre Channel techniques, but making sure the analyzer can adapt may still require special cables.SAS specifies four different types of cables and connectors. There are single-link, dual-link and four-link cables designed for use inside an enclosure. There is also a four-link connector specified for external use, such as connecting a server to an external RAID box. Since SAS analyzers today use the single-link connector, connecting an analyzer to a dual-link or four-link bus requires special adapter cables. Fortunately, these are the same cables used in standard SAS storage applications so they are starting to become available from cable manufacturers.
[FIGURE 1 OMITTED]
The single-link connector (Fig. 1) is the same as the SATA connector and is manufactured by many different companies. The dual-link connector (Fig. 2) is new and physically similar to the SATA/SAS single-link connector, but adds additional pins on the unused opposite side, providing for a redundant set of connections between the drive and host in case the primary physical connection goes bad.
SCSI marketing departments have always maintained that a significant differentiator between ATA and SCSI drives is reliability, and the redundancy of the new dual-link connector for SAS is further proof of this difference going forward. The internal four-link connector (Fig. 3) is just like a very-wide SATA connector. RAID applications inside the box will use the internal four-link connector. The external four-link connector is based on the same MicroGiga connector used by 4x Infiniband, but uses jackscrews instead of the quick-release latches that Infiniband cables use. External four-link SAS applications include expanders and RAID boxes.
Testing "What If" Scenarios
SAS provides an unusual level of flexibility when it comes to some of its handshaking using primitives. Table 1 shows some areas where SAS state machines are more flexible than their FC equivalents.
The flexibility of SAS primitives makes it difficult to verify that a SAS device is robust enough to support the primitives, no matter when they occur. Typically, those designing SAS silicon will make or buy the simulation code they need to verify the RTL (Register Transfer Level) code, ensuring that the different scenarios have been verified.
[FIGURE 2 OMITTED]
One company that develops and sells SAS protocol verification tools is Perfectus (Santa Clara, CA). Their verification engine generates the packets and primitives in software to test all the possible timing and sequence possibilities. The Perfectus SAS verification tests are also translated to the Data Transit PacketMaker hardware for verification of the final device--not just a simulation. Figure 4 shows the sequence for verifying RTL prior to finalizing the hardware, and later verifying the final hardware with the same tests.
[FIGURE 3 OMITTED]
Perfectus tests are categorized according to the different protocol layers of PHY, Link, Transport, and Command. They give the developer the ability to adjust a variety of test 'knobs' which vary the timings and occurrences of SAS Errors, OOB, Primitives, and Frames within legal specifications, or beyond specifications if the user chooses. The test case generation is coupled with a protocol checker, which analyzes a device's response to the generated traffic to determine if the device followed protocol. The protocol checker is implemented in software and verifies both the simulated results of the pre-silicon design, and the actual results of the final hardware as received by the PacketMaker.
[FIGURE 4 OMITTED]
Table 2 shows the tests performed in each protocol category by the combination of the Perfectus verification engine and the Data Transit PacketMaker.
[FIGURE 5 OMITTED]
Obstacles to Analyzing Wide SAS
Wide SAS uses multiple links in order to handle multiple commands simultaneously. In this multi-link environment, each link carries the data for a different command. The links cannot be aggregated to achieve higher bandwidth for a single command, but multi-lane performance gains can be achieved at the application layer when a host adapter breaks a data request into multiple commands, using a different link for each command. Thus, the data can be sent down multiple lanes simultaneously even though commands can only be issued on one lane at a time.
Certain features of wide SAS make it impossible to analyze each link as a distinct SAS bus using a single-link SAS analyzer. On a wide SAS bus, a command will be issued on whichever link is available, rather than waiting for a specific link, which would add more latency.
One of the difficulties of analyzing commands on a wide SAS bus is that the connection on one link can be closed before the data transfer for that command is complete, such as when the drive empties its buffer and closes the connection in order to free-up the link while it refills its buffer.
In this example, the original link may be busy when the drive is ready to continue, so a different link is opened in order to continue the transfer. Thus, the command was executed using different links at different times, preventing a single-link analyzer from capturing all the data associated with a command.
Special analyzers, such as the Data Transit 4x-SAS Analyzer (Fig. 5), can simultaneously capture and correlate all the links, putting commands back together even if they used multiple links before completing. The following trace display (Fig. 6) shows how the analyzer correlates the data from multiple links into a single display. The display allows the user to display the traffic in a time-ordered, command-ordered, or connection-ordered format.
DVDs, CD-ROMs on SAS/SATA Bus
Hard drives supporting SATA are shipping in volume, but CD and DVD drives are just now in development. The new CD/DVD drives for SATA will continue to use the ATAPI protocol as they did on parallel ATA, even though ATAPI adds a significant amount of protocol complexity.
The ATAPI protocol is necessary for compatibility with existing host drivers, just as it was necessary when the first SCSI CD-ROM drives made the transition to the low-cost parallel ATA bus. The history of CD and DVD interfaces is interesting because it demonstrates how much protocol baggage current devices must carry in the name of driver compatibility.
The original CD-ROM drives were SCSI devices, and using them on a PC required the purchase of a SCSI host adapter. Later, in order to eliminate the extra cost of the host adapter, the ATAPI specification was added to the ATA standard as a method for transmitting SCSI commands across ATA hardware by encapsulating or packetizing much of the SCSI protocol into standard ATA data transfers.
[FIGURE 6 OMITTED]
CD-ROM drives were then able to use their tried-and-true SCSI command set on the pervasive PC architecture while eliminating the cost of a SCSI host adapter. The encapsulation adds overhead, but for optical drives in the PC market, price is much more important than performance.
[FIGURE 7 OMITTED]
Figure 7 shows the interface migration of low-cost CD-ROM drives from SCSI to ATA to SATA. Today, even though SATA is already a packetized interface and it would be a much simpler protocol to discontinue the encapsulation of SCSI commands into ATA data transfers, the ATAPI encapsulation will still be used so that the OS and its drivers won't see a difference between a Parallel ATA DVD drive and a SATA DVD drive.
Protocol Analyzers used with the new generation of SATA optical drives must support the ATAPI protocol with special decoding and triggering functions. In the future, CD/DVD drives designed for performance instead of low cost will become available with a SAS interface instead of ATAPI over SATA. Figure 8 shows a state listing of an ATAPI protocol SCSI command encapsulated into a SATA data transfer.
Conclusion
SAS presents a number of special challenges and obstacles to engineers working with this new storage technology: Cabling and the multi-lane data paths characteristic of SAS being just two of them. Overcoming these challenges takes knowledge and the right test tools.
[FIGURE 8 OMITTED]
Table 1 Feature SAS Primitives start with pos or neg disparity YES Allows deletable primitives in frames YES-ALIGNs Allows credit primitives in frames YES-R--RDYs Handshake primitives in frames YES-ACK, NAK Zero primitives between frames SOF can follow EOF Feature Fibre Channel Primitives start with pos or neg disparity NO-Must start with neg disparity Allows deletable primitives in frames IDLE not allowed in Frames Allows credit primitives in frames R--RDY not allowed in Frames Handshake primitives in frames ACK Frame instead of primitive Zero primitives between frames Must transmit 6 IDLEs between Table 2 PHY LINK TRANSPORT OOB Signals Primitives FIS construction SASRESET, and decomposition SASINIT and SASWAKE Host Phy Disparity All FIS types Initialization transmission and State Machine reception checking Device Phy Data and CRC Host and Device Initialization Transport State State Machine Machines Power Management Link Idle, Transport Layer State Machines Transmit, Error Handling (Host/Device) Recieve and Power Management State Machines Phy Layer Error Link Layer Error Handling Testing Handling Testing PHY COMMAND OOB Signals Power-on, SASRESET, Deviceidle and SASINIT and Reset protocol SASWAKE Host Phy PIO & DMA data-in Initialization and data-out State Machine protocol Device Phy Read and Write Initialization DMA-queued State Machine Power Management Command Layer State Machines Error Handling (Host/Device) Phy Layer Error Handling Testing
Parts 1 and 2 of this article appeared in the January and February issues of CTR. To obtain copies of CTR, please contact the circulation dept.: renee_hieblinger@wwpi.com
Dale Smith is CTO and founder of Data Transit Corporation (San Jose, CA)
www.datatransit.com
 Printer friendly
 Cite/link
 Email
 Feedback
| |
| Title Annotation: | Connectivity; Serial Adapted SCSI |
|---|---|
| Author: | Smith, Dale |
| Publication: | Computer Technology Review |
| Geographic Code: | 1USA |
| Date: | Mar 1, 2004 |
| Words: | 1594 |
| Previous Article: | Transitioning to SAS technology: a comprehensive comparison between SAS and parallel storage. |
| Next Article: | Data protection strategies: are they too complex? |
| Topics: | |