Notes From The Lab.

The Ecrix VXA review (page 38 in this issue) is an important one. This isn't to say that other reviews won't be of comparable importance-indeed, all product reviews will be run as if a company's future sales count on it, but the Ecrix VXA review is the first and may be something of a model for future reviews. At the risk of giving the wrong impression, let me make it clear that my goal in reviewing products for Computer Technology Review has little to do with helping or hurting a company's sales. The goal is not to enhance or reduce my reputation as a reviewer. My primary goal in writing reviews is the same today as it was nearly two decades ago when I wrote my first review--to give an honest, balanced evaluation that can be used by the reader to make an educated decision whether to buy or use a reviewed product or not.

Developing a fair review of the Ecrix VXA-1 drive presented a number of challenges, some of which forced reexamination of the test platfonn. The VXA-1 unit I reviewed is configured as an External SCSI drive. Connecting and configuring the drive were, as expected, pretty easy. My test system used an Adaptec 2940U2W host bus adapter. Using an internal cable included with the adapter, I was able to connect the external cable coming from the Ecrix drive to the end of the internal cable. I know this may sound confusing--the special cable has two connectors for internal drives and an external connector that mounts to the back of the computer and that can be used for connection to 50 pin mini-SCSI connectors, like the one that works with the Ecrix drive. Setting a SCSI ID, connecting a cable, and attaching a terminator aren't rocket science. Installation was easy.

Unfortunately, operation was erratic. My primary backup application was Novaback 6. This was chosen for a few reasons. Among these was flexibility of configuration and a sophisticated progress screen that makes it easy to determine, minute by minute, how fast the drive is writing data. The configuration options made it easy to turn compression on and off and to turn archive bit reset on or off (important if you want to do testing on files without marking them as 'saved').

In addition to Novaback 6,I also did tests using Dantz Retrospect 5 and two versions of Veritas Backup Exec (3.0 and 4.2). The primary goal was to attempt to extract measures of performance from each application that may not have been available with other software.

I used a few different backup sets. Initially, I tested basic performance using the contents of a single IDE drive, about 12GB in size with a mix of file types ranging from applications and operating system files to compressed audio and video files. It was clear that compressed files are not good candidates for backup to a drive that has hardware compression enabled--in some cases, such files may actually slow a drive's performance and take more space to store compressed than they take to store as written to hard drive.

I also used data sets that ranged from 30GB to 51GB, spanning over 13 logical and physical drives. These drives also contained a mix of file types and ranged in size from a few hundred megabytes to 47GB. It is believed that this mix of drives may approximate what can be found on a small network with small drives on older clients and a larger drive on the server.

I think it's important to test with at least one data set that really challenges the drive. I've seen reviews that were done using data sets as small as 2GB or 3GB, recorded over and over and over again. Except for small desktop applications, I doubt the value of testing drive performance with such small data sets. Statistics for backing up 300GB over a 24-hour test period may be of little value if the test was for backing up a 3GB test set 100 times. Why not back up a 100MB set 3000 times? Although running a drive over a 24-hour period may be useful to determine whether a drive will fail after so few hours, I don't think there's a lot of real world value to be gained from such tests. Backing up multiple drives with data sets ranging in size from a few hundred megabytes up to more than a dozen gigabytes seems a lot more real world to me. Let me know what you think ([e-mail]). During initial testing, backup speeds fluctuated wildly and somewhat unpredictably, ranging from a high of about 9GB/hour to less than 700MB/hr. Clearly, something strange was happening.

In order to rule out many of the issues that may have produced poor performance, I made a number of changes. I replaced the Adaptec 2940U2W with a new 2940U2W adapter. Next, I made a clean install of Windows 98 Version 2 to a new directory on a Seagate Cheetah hard drive. I installed a minimum of drivers and applications, hoping that a leaner installation will reduce the overhead required to support other applications that run in the background.

Perhaps most significantly, I replaced the motherboard. The original motherboard was an EPOX MVP-3G. This board had a 1MB on board cache and seemed to be functioning appropriately. The replacement board was an EPOX MVP-3G5 with support for ATA-66 (although I didn't use this feature) and a 2MB cache. The performance improvement seemed to be immediate. Much of the improvement may have been due to the larger cache, as improbable as this may seem.

A tool that I use monitors the use of memory, both RAM and cache. The program, MemTurbo, is a small shareware application that performs near magic. MemTurbo monitors memory usage. If you choose to, you can view, real time, how much memory is available. Some applications don't manage memory very well. When these applications close or stop a particular process, they may not release the memory for use by Windows.

Until I discovered MemTurbo, I would often run into what appeared to be an out of memory situation. For a computer with 384MB of RAM, this was somewhat confusing. The solution, before MemTurbo, was restarting Windows or, at the least, logging off and then logging back in. MemTurbo (www.memturbo.com) can be a useful tool for clients and servers. It can be set to automatically recover and defragment available memory at specific time intervals or to recover memory when the amount of available memory drops below a certain level. Installing MemTurbo across a network may reduce support calls from users who may be running many applications and whose computers are prone to memory leaks.

In the case of the Ecrix review, MemTurbo gave me different information. When running a backup, I was able to open MemTurbo's monitor screen to watch what happened to RAM. During a backup, the available RAM dropped continuously, a few megabytes at a time, in what appeared graphically to be a series of steps down a staircase. When the memory hit the threshold level for memory recovery, MemTurbo kicked in, restoring memory that was then eaten away each time data was moved from the disk being backed up into a buffer for backup.

After installing the new motherboard, installing Windows 98 to a new drive and directory, and replacing the Adaptec HBA, the memory leak stopped. My guess is that the new motherboard, perhaps as a result of the larger on board cache, made it possible to move data from the disk drive through the SCSI card and onto the Ecrix drive without having to use any system RAM to temporarily store the data.

Testing with the new configuration drove performance up. Backup rates after the upgrade were about 9GB/hr, regardless of the types of data being backed up, and fairly independent of the application used for the backup. The only area of concern was an apparent lack of compressed storage capacity. Even though the Ecrix claims a 33GB native capacity for its V17 tape cartridge, backup applications kicked the tape out after storing 34GB, suggesting that there was either a problem with the data being uncompressible to tape or compression not enabled.

Ecrix technical support assured me that the default for the drive is to automatically compress the data. They suggested that performance might be improved by running the drive on its own SCSI adapter.

My last few tests were run with the drive connected as the only device attached to a second 2940U2W installed in my system. If possible, I'll do whatever I can to give any item being reviewed as fair a test as possible.

Additionally, I've defined backup sets that exclude such compressed file types as JPG, MP, ZIP, WAV, GIF and others. I've tried to exclude whatever files were already compressed with the hope that a more accurate look at extra capacity can be attained, even if this diverges from the real world mix of files across many networks.

At this time, later in the morning, Backup Exec has finished writing 25GB to a cartridge. I'm about to run the same backup again, appending it to the tape cartridge and seeing if the extra capacity is available.

After running the backup set a second time, results showed an increase in capacity. Before the compressed files were excluded, somewhere between 33GB and 35GB of data could be stored onto an Ecrix V17 cartridge. After excluding many of the compressed files, the storage capacity increased to 38GB. Although this isn't an earthshaking amount of increase (10%-20%), it demonstrates that the compression algorithms actually work and points out a problem with specifications based on data compression.

I have a few more thoughts regarding compression. First, though compression sounds great, it may be that, in many cases, using hardware compression may not boost the backup performance. As it was explained to me by Ecrix technical support (in the case of the VXA drive, at least), the compression processor worked more quickly than the drive's write performance. Data, whether compressed or not, would still be written at maximum drive speed, as long as data was being moved from the server to the drive at maximum speed. The purpose of compression in this case was only to increase the amount of data that could be stored on the tape-performance was not an issue here.

Secondly, performance, rather than compression, is probably more important. I have to refer back to one of my opening comments--unless you've got an automated tape system, there's no such thing as unattended backup if you have more data than you have tape capacity. If someone has to wait to replace a full tape with a new tape, reducing the time required between tape swaps is an obvious plus. Unless, of course, your drive takes 16 hours to fill a tape, in which case, you can start backup at the end of the day and switch tapes next morning.

Where performance may be a bigger issue is in restoring files. In a typical restore scenario, you select files, directories, or even drives that you want to restore from a catalog that your backup software created during backup. Once the files are selected, the tape drive searches for the files and begins to write them back to their target. A high performance drive should be able to find the data on the tape more rapidly and transfer the data to the target more quickly. When you're restoring critical files, time matters. It should be less important how much data is on the drive than it is how fast your drive can get to the data on the tape and move it back to the target drives.

That's enough for now. In the next Notes, I'll discuss some of the tools I use to check my network connections and the quality of my link to the Internet and, of course, I'll explore issues related to my next set of reviews. Your comments are always welcome. Please send them to [e-mail].