The incredible shrinking petabyte: how compression technologies are helping store more data in less space.

The Internet brought with it a round-the-clock business environment where Web traffic, e-mail and e-commerce transactions drive data creation, storage and business continuity requirements that continuously change. Traditionally, large organizations dealt with the data deluge by simply buying massive storage libraries, filling up large data warehouses with wall-to-wall archival solutions to hold and manage it all.

But just as technology advancement and dropping manufacturing costs eventually brought the power of refrigerator-sized computers of the 1950s to desktop PCs of the 1980s, storage hardware and media are rapidly shrinking and becoming ubiquitous to every business and trade. The ability to store large volumes of data in a smaller area is a growing priority for companies looking to improve their cost of doing business and take up less space on their data room floor. And the creation of smaller and smaller storage hardware is being matched by innovations in the media needed to store that data.

Today, innovations in data compression and coating technologies are enabling more information to reside on a smaller surface. As a result, it will soon be possible to fit a terabyte worth of information--an equivalent to 500 million text pages generated from 50,000 trees--onto a cartridge that fits in the paln of your hand.

Data storage has a long and wonderful history filled with tremendous innovations that have kept up with and enabled the burgeoning information age. And while today's data warehouses may be cumbersome, the quest to reduce the size of those data stores is in fact as old as the computing industry itself.

Tubes to Tape: The History of Storing Data

It took nearly 3 million years for humans to figure out that etching animals and good hunting sites onto stone for preservation and information-sharing was a valuable--and life-prolonging--tool. Thankfully, data storage technology and compression is evolving considerably faster since the birth of modern information technology just 50 years ago.

Before the 1950s, large rooms filled with vacuum tubes and wires were used to perform even the simplest mathematical calculations. Information was kept on paper, in books at libraries or--if you were really high tech--punch cards. Personal computers, hard drives and digital data storage were the stuff of science fiction novels and Saturday matinee movies.

The baby boom in the United States in the 1950s forced the government to create a better way to track the numbers and needs of a rapidly growing population. The result was a burgeoning U.S. census bureau and Social Security administration that became the catalysts for innovation to find a better way to store data. In 1951, two UNIVAC I computers were delivered to the U.S. Census Bureau that together weighed 32,000 pounds and contained over 10,000 vacuum tubes. The UNIVAC's tape storage device called UNISERVO I held 1,000 words with 11 digits. UNIVAC was state-of-the-art, and handled a whopping 1,000 calculations per second at a cost of $159,000 for the first mass-production unit. To give you some perspective, that would be nearly $1.5M in 2004 dollars. That was some investment for a machine quite different from today's toaster-sized computers capable of 2 billion calculations per second--and cost less than $1,000.

As refrigerator-sized computers began to proliferate, a series of enhancements and firsts for tape storage quickly followed, to help reduce and improve efficiency of the space that computing power took up on the data center floor.

In 1986, an IBM compression development known as Improved Data Recording Capability (IDRC) provided a significant improvement in data compression for tape of at least 2X. IBM's Enterprise Systems Connection Architecture (ESCON) channel interface, delivered in 1987, enabled tape storage to be managed across a network instead of at the central server. Tape subsystems could then be deployed for better data management and across several kilometers--a distance previously unheard of.

In 1991, the 36-track drive was introduced, which used a new extended-length chromium dioxide tape and provided 800MB of storage. With the IDRC, capacity was expanded to over 2.4GB--the highest data capacity available at that time.

As data storage has evolved, other types of storage have followed magnetic tape storage and have led to the development of everything from real-time storage servers to the floppy disk and zip drives of the average PC. CDs changed the world of audio recording while DVDs are currently doing the same thing for video data storage.

Higher Densities, Smaller Footprints

All of the great digital storage innovations over the past 50 years owe their development to new ways of compressing data onto smaller and smaller formats. Compression requires some form of mathematical formula, or algorithm, to reduce redundant strings of data and uses a code to create patterns of data that fit onto a smaller space. Not all compression technologies are equal, however, and some recent advances are allowing for unprecedented amounts of data to be compressed to smaller sizes than was ever before thought possible.

In addition to making the data smaller, the storage media itself is being enhanced to hold more of the data on an increasingly smaller space. The first tape storage products utilized magnetic particles coated onto a thin ribbon of metal substrate. As technology evolved, engineers searched for a way to make the layer thinner and thinner--thereby increasing the overall capacity and quality of the recorded information. In 1992, Fujifilm introduced ATOMM technology (Advanced Super Thin Layer and High Output Metal Media), a second, non-metal particle layer to enhance the quality and improve recording densities--a huge industry breakthrough that led to the development of new tape storage formats such as Digital Linear Tape (DLT).

In 2001, Fujifilm engineers took the lead to introduce Nanocubic technology, an entirely new coating technology utilizing both metal and barium ferrite particles to make magnetic recording layers 10X thinner than the super-high resolution of metal particle technology.

This ultra-thin layer coating process controls the thickness of the magnetic layer on a nanometer scale. This is crucial for layering microscopic magnetic particles that must be uniformly coated in order to achieve ultra-low signal-to-noise ratios and higher recording bit densities required. This dispersion technology uses a special organic binder material that has the ability to thoroughly disperse the particles in the coating solution so that a uniformly packed magnetic layer is realized. The technology employs two types of super-fine magnetic particles, both tens of nanometers in scale: Acicular Ferromagnetic Alloy particle and Tabular Ferro-magnetic Hexagonal Barium Ferrite particle.

Nanocubic coating technology will lead to the achievement of the super tape storage capacities, both for helical scan and linear recording mid-range and enterprise class formats. This has great implications on the thriving market of tape storage, and today you'll find tape just about anywhere you find information stored electronically. From the early 1950s, the U.S. Census Bureau and U.S. Social Security Department continue to store demographic data on tape; bank teller transactions are recorded on tape to provide a permanent record for auditors; and insurance companies back up policyholder data on tape. Storage tape is playing a critical role in today's search for new energy sources as well. Nearly all the major geophysical and petrochemical companies are using tape media to record massive amounts of seismic data as they search for pockets of fuel.

The Future of Compression

There will be a huge demand for the capacity enabled by compression technology from customers in fields such as life sciences and digital media. These industries will create enormous amounts of data via digitized images--on the magnitude of many petabytes (a petabyte = 1,000 terabytes, or the equivalent to 20 million four-drawer filing cabinets full of text) that digital images create. The life sciences industry, in particular, is seeing remarkable information growth and will benefit greatly from this breakthrough technology. In medical imaging alone, the use of PET, SPECT, CT and MRI imaging technologies is estimated to grow up to 67% through 2006, according to health care business intelligence organization Solucient. Fujifilm Medical Systems (one of our sister companies) credited for inventing the digital x-ray, is working with its customers to continuously drive new digital solutions via its software-based digital image and information management system--generating more than 1.8 million images a day.

It's not hard to envision how such increased use across the industry will generate hundreds of millions of digital images each year requiring terabytes of data.

Another application of compression technology will be seen in Disaster Recovery implementations. More and more companies today are seeing the need to protect business critical data and to implement disaster recovery solutions. Seventy percent of companies go out of business after a major data loss. Data storage media companies are providing the high quality tapes to back up data in case of system failures; and compression technology will make this even more feasible and easily deployable for businesses of all sizes. Clearly, compression technologies will shape removable data storage systems for years to come.

The extraordinary capacities and levels of performance that can be achieved through the breakthrough in compression technology come at a time when the corporate demand for data storage solutions seems insatiable. Ten years ago, the concept of a terabyte seemed foreign. Today, a terabyte represents the entire digitized x-ray library at a single, large hospital.

Not to let physical limits hold the industry down, new storage technologies are emerging to match the accelerating digitalization and global sharing of information. Some have estimated that by 2010 we could see media capable of handling more than 10 tera-bytes of information in a single cartridge. To put that in perspective, that's the equivalent of 2000 full-length feature movies stored on a single cartridge.

All in all, the journey from those first 32,000-pound systems of the 1950s to the palm-sized computing and storage technology of tape cartridges, DVDs, CDs and USB drives today has been an incredible achievement with an ongoing legacy. In the future, data compression and coating technology will lead to products that meet the ever-growing data demands and ensure the continued growth and leadership of the data storage media business.

www.fujifilm.com

RELATED ARTICLE: Next-Generation Tapes Change the Rules

For years, tape technology lagged the technological progress and innovation of disk drives. In the late '90s, the magnetic tape industry began to respond by delivering numerous and significant design improvements. By 2000, magnetic tape cartridge capacity had surpassed the capacity of the largest disk drive for the first time ever. Historically, the preferred choice for backup, recovery and archiving, recent tape enhancements have positioned tape for a variety of new applications. Even with the advent of low-cost disk arrays entering the backup and recovery market, today there is no truly cost-effective storage strategy without a tape component.

Excerpted from Storage: New Game, New Rules by Fred Moore, president of Horison Information Strategies (www.horison.com)

Richard Gadomski is vice president of marketing, recording media division, Fuji Photo Film U.S.A., Inc. (Valhalla, NY)