Access Density Key To Disk Performance.

Tremendous increases in disk drive capacity have been offered and the capacities are increasing at a geometric rate. For the customer who buys disk storage, this has resulted in significant price reductions per megabyte of disk. There is a negative side to the capacity increases: performance. Although capacities have increased, the corresponding factors that are part of the performance characteristics of disks have not increased at an equal pace. The effect on performance is so significant to the use of the storage for each new capacity level of disks that performance expectations must be modified or some other compensating factor must be introduced (more caching, more sophisticated controller mechanisms, etc.).

This article will demonstrate one aspect of how increased storage capacity increase can impact performance. The aspect to be addressed is called access density.

ACCESS DENSITY

Access Density is a number that is used as a measure of performance. The generally accepted definition of access density is 'the number of I/Os per second per gigabyte of storage.' This measure can be the amount being done for a particular workload on a storage subsystem or it can represent a maximum amount of I/O operations that can be achieved for a given type of workload.

The maximum amount is a characterization of how many I/Os a storage device can do. The parameters to consider in the workload are reads versus writes, random I/O vs. sequential I/O, etc. Typically, the worst case access density occurs when 100% random I/O is being done. Performance characterizations are normally done both with the worst case situation and a mixture that would simulate a specific type of application or customer environment. For the purposes of this article, we are interested in the 100% random read case, which would establish a baseline for maximum access density.

The unit of measure can be applied to a disk drive or a storage system (i.e., a RAID system). Obviously, the more I/Os that a storage device can perform in a given period of time, the better its performance capability. Access performance is an integral part of overall system performance. Although it is much easier and more interesting for most people to talk about processor MIPs, most of the time computers are moving data. The devices used to move data, rather than the CPU performance, become the limiting factor. Access density is a little known but key component of overall system performance.

Access density shows itself in the simplest form when a random I/O is done on a disk drive. The usual measure is the number of random I/Os per second. The access density metric provides a limit of how many I/Os that disk drive can do, but does not take into consideration I/Os to multiple drives on the same path or system bandwidth limitations.

DISK TECHNOLOGY

Disks have increased areal density through technology advances in several areas. Most of these advances require an advanced physics degree to understand in depth. In simple terms, multiple technology changes have combined to provide for improvements in the disk capacity.

* The formulations of the media have advanced to allow more magnetic flux changes to occur per inch.

* The head properties have improved to detect finer granularities of magnetic reversals through use of Magneto Resistive (MR) and Giant Magneto Resistive (GMR) heads.

* Actuator mechanisms have improved to give finer alignment for location registration.

* Advances in electronics allow faster read channels, etc.

Combined with these innovations, disk drives have one or more built-in processors to handle the interface, control the servo positioning, and handle error recovery. These functions complement the capacity increases.

The I/O rate supported by a disk drive is a result of performance of multiple components:

* Interface overhead

* Data transfer time

* Electro-mechanical positioning--seek and rotation latency.

* Internal algorithms for request ordering and other performance improvements.

The electro-mechanical time overwhelmingly dominates the time consumed during a random I/O request. Drive manufacturers have improved the disks in this area by reducing seek times and increasing rotation speed (to over 10,000rpm in recent drives). The problem is that these increases have not kept pace with the increases in density, leading to the increasing problem in access density (Table 1).

Another improvement made in disk drives is the incorporation of an actuator level buffer on the drive controller card. This buffer can be used to mask the rotation of the disk somewhat and for prefetching of data. In sequential I/O operations, the buffer can improve the I/O rate by overlapping the overhead time for the next command with data transfer from the media to the buffer and eliminating the rotational positioning miss. As access density has improved over time, the size of the buffers has increased. Newer technology drives offer a 4MB buffer. Unfortunately, this makes little difference in random I/Os.

EFFECTS

In demonstrating the progression of disk technologies and the effect on access density, I chose to use the first generation of new capacity points on 3.5-inch disk drives. I chose common capacity points of 1GB, 2.1GB, 4.5GB, 9GB, and 18GB. Due to unavailability, I did not test the latest 36GB and 50GB drives. To represent the effect of the larger capacities, using the first generation of the drives at those capacities kept the factors somewhat equal among the drives chosen. Subsequent generations at the same capacity points have introduced such refinements as faster rotation speed, larger cache buffers, and faster electro-mechanical devices. Since the second generation drives at a given capacity point were not necessarily the leaders in the market and some capacity points did not have a second generation, I chose to exclude those from this analysis. I also chose to use SCSI drives because of their common use in more performance critical systems.

To measure the number of I/Os that could be done on each generation of disk, I used a specifically developed test program. The program generated random I/Os across the entire usable area of the disk.

Table 2 plots the year of delivery for the identified drive capacity points against the maximum I/O rate supported by that particular drive for random I/O of 512 byte blocks with a tagged command queue depth of 1. Table 2 graphically illustrates that while density is doubling, the maximum random I/O rate is relatively constant. Table 3 takes the same data and represents it as the maximum access density (maximum I/O rate divided by capacity). This shows that maximum access density is actually decreasing.

One of the improvements that came with advancement of the SCSI interface to the SCSI 2 standard was the ability to queue commands at the device level, allowing the device to reorder execution in order to optimize the electro-mechanical motion. Looking at the access density with the queue depth set to a higher value for random I/O shows the effect of this improvement. Tables 4 and 5 show the results under the same conditions as the previous charts except that the queue depth was increased to 24. These tables show an access density improvement over a queue depth of 1 when command queuing to the device is implemented.

By increasing the disk capacity without a comparable increase in the electro-mechanical properties necessary to improve the I/O rate, there is an obvious reduction in access density. For the user of the storage system, this results in less costly disk storage, coupled with a degradation in the maximum performance that the storage system can provide. Disk drive and system designers have tried to compensate for the disparity with caching in the controller (or host), buffering at the drive level and queuing of commands to overlap mechanical motion. Caching and buffering are effective in workloads with sequential access or some locality of reference in the random I/Os. As shown in Table 5, command queuing, while helpful, does not overcome the trend. The judgment about what is useful for improving access density depends on the individual workload characteristics. These techniques usually do not overcome the density increases. It is clear that the larger capacity drives are working against system performance and different approaches to system storage must be used to overcome this situation. By putting more data on fewer actuators, the cost per megabyte of storage has gone down significantly, but the system level performance capability has suffered dramatically. When a new generation of higher capacity disk drives is used, the performance expectation must be adjusted or some compensating factor must be introduced.

Randy Kerns is the director of engineering for enterprise disk at StorageTek (Louisville, CO).

          1GB drive-1992 18GB drive-1997 Improvement
Seek Time    10.2 ms          5.7 ms         56%
Latency      5.56 ms         2.99 ms         54%
Capacity     1.05 GB         18.2 GB       1733%
                RANDOM I/O - 512 BYTE BLOCKS, QUEUE DEPTH OF 1
YEAR OF INTRODUCTION Capacity I/O Rate
1992                    1        60
1993                    2.1      77
1994                    4.5      70
1995                    9        71
1997                   18        75
                RANDOM I/O - 512 BYTE BLOCKS, QUEUE DEPTH OF 24
YEAR OF INTRODUCTION Capacity I/O Rate
1992                    76       1
1993                   121       2.1
1994                   133       4.5
1995                   121       9
1997                   140      18