The role of microfilm in information management: current and past problems with digital long-term storage have been well publicized. Is it possible to guarantee the preservation of history and allow for future processing of it? (Tech Trends).

Despite all the benefits digital technology has provided, recent history has taught us that the seemingly endless hardware, media, and software changes render digital systems unreliable for long-term preservation. Current and past problems with digital long-term storage have been well publicized; one can only speculate how widespread the problem will become in the future.

A study conducted by The Collaborative Electronic Notebook Systems Association, "Looming Information Age Crisis Expected to Cause Trillion-Dollar Losses," identifies the primary causes of the impending digital disasters created by the rush into the digital future. The report suggests that the only solution is standardization of information technology (IT) systems: "For society to move all the way into the Information Age, we need to replace paper and microfilm as the default permanent storage media for data created electronically."

While digital technology has changed the world for the better by allowing information to take many forms and move rapidly around the planet, it has actually taken a step backwards for preservation purposes. For example, a 1987 electronic version of the Doomsday Book, which was first compiled in 1086, has become unreadable. It seems ironic that the 16-year-old high-tech version is unreadable while the ancient copy remains perfectly usable. Computer expert Paul Wheatley reflected, "We're lucky Shakespeare didn't write on an old PC." The problem is obvious, but the solution is not so clear.

Unfortunately, many computer-oriented people continue to promote technology-dependent digital preservation systems and the abandonment of human readable forms (i.e., paper and microfilm) as being "obsolete." Cynics believe the computer industry operates on the basis of planned obsolescence, that it actually plans endless upgrades and migration. Digital technology allows proprietary software, hardware, and media, and the dominant software vendors have certainly taken advantage of that fact. Upgrades are a major revenue source. Planned or not, this highly profitable strategy provides incentive for more of the same.

"Replacing Paper: Bad News for Trees" addresses the hotly debated suggestion that e-books will replace paper books. There are two reasons why even the most ambitious champions of digital technology still do not think they can kill ordinary paper. One is the problem of durability. Atoms tend to persist, but electronic signals disappear unless they are recorded. Unfortunately, both the recording media and the software used to retrieve digital data quickly become obsolete. The other disadvantage of digital is that people are not digital; they are physical objects who live and work in three dimensions. People prefer to work and play with objects arranged around them, and their memories depend on cues provided by spatial location.

Breaking Down the Problem

To understand the basic issue, separating information management into active and inactive stages, using what is commonly called "the document life cycle" is important. The active stage varies by application, but its utility is usually one year or less. At some point, the need for handling this set of documents slows, stops, and the documents become inactive. Retention in the inactive phase may be three to five years or even permanent.

Technology to store records for a long period (e.g., digital tape) often differs from the media used to create records (e.g., magnetic hard drives). That a lower-cost--and sometimes less stable--medium is often used for long-term storage suggests that concern for digital preservation is not high.

Fortunately, today's media technology has given us longer-life paper, microfilm and, to an extent, digital media. Long-life stable materials can be stored in human- or machine-readable forms, or both. Only paper and microfilm, however, have certified life expectancy (LE) of 500 years: ANSI LE 500 microfilm and acid-free LE 500 paper. Both are in human-readable form. So while creation and manipulation using digital formats may work well in the creation or active stage, they do not hold out as much hope for longer life--the inactive phase--as do paper or microfilm.

Digital vs. Human-Readable

Several real-world examples point to the issues related to digital and microfilm as "rivals."

Example 1

Many think that the 2000 U.S. census statistical data must remain in processable (digital) form to allow comparison with future census surveys. The paper census forms, on the other hand, are being captured on archival microfilm in 2002 and are static. When it is determined that there is no longer a need for further processing of the statistical data, ANSI LE 500 microfilm in human-readable form is required.

Example 2

When the 1930 census was microfilmed, no one knew that later it would be possible to scan, digitize, and send it anywhere on earth via the Internet. Likewise, one can only speculate now as to what type of system will be used in the year 2500 to read the same human-readable microfilm and possibly send it to a settlement on Mars. The critical point, of course, is that it will be readable.

Example 3

The British House of Commons debated replacing parchment with high-quality linen paper to preserve British law, which has a permanent retention period. The House of Commons decided to continue using parchment because the oldest parchment records are 1,000 years old and continue to be perfectly legible. Because parchment has proved to last a millennium and can be digitized on demand, why would they risk a switch to another medium?

Long-Term Preservation Methodology for Human Works

Humans have spent thousands of years perfecting their ability to communicate with one another. They do it by language, both written and spoken, a method that not only provides direct human interaction but also has the ability to preserve it. In a world of instant global communications, people want the ability to communicate as needed on a worldwide basis. Telecommunications, the Internet, and e-mail are fueling the need for a single common language. People want and need a standardized methodology for written and verbal communication.

All present-day digital preservation systems, however, are technology dependent and are constantly evolving and changing. Because history teaches that there is the additional possibility that today's digital technology may be replaced by a more efficient processing method, all machine-readable information technologies are subject to obsolescence. This is in stark contrast to the physical world, which remains constant. Some human inventions, however, stand the test of time; these include the Latin alphabet and Arabic numerals, which will remain reliable and accessible for centuries.

Consider what actually happens in our "digital" world. Humans program computers using programming languages of the day. Then, another group of humans enter information as alphanumeric characters. At that point, those characters are converted to machine-readable binary code (zeros and ones). Then, the data processing begins. At some point after processing, the coded information is converted again back to the human-readable form it was created in.

Computer input and output is in the form of human language, so what is the purpose of machine language? It is little more than a methodology to speed up the processing of human-originated information. Computers receive human input, process it, and provide the processed information as human-readable output. The popular phrase "born digital" is misleading because information is born in the brain of a human being and entered into a computer. Digital is a processing technology, not a human communication method. People communicate with speech and writings. One can only speculate why some continue to misrepresent computer technology.

A Question of Credibility

Concerns about preservation of digital records and documents extend beyond the issues of physical preservation and into those of credibility. The authors of "Looming Information Age Crisis" note that "the integrity of electronic records and data are easily broken when they are removed from the technology that was used to create them. Once broken, an electronic record's integrity cannot easily be restored."

Diane Carlisle, CRM, director of professional resources for ARMA International, correctly raises this issue in AIIM's E-doc (September/October 2002): "A key to any archival record, everyone agrees, is that it be preserved in its original form at the time of creation." What may be considered "original form" may change over time. In the case of a letter typed into word processing software, for example, is the original form the letter as displayed on the monitor, the paper printout (if one exists), or the binary representation stored on the hard drive? Because a person created the letter, its original form is that displayed on the computer monitor in front of its creator. The display, however, is temporary. To save it in its original form, it is printed out on paper or optically reduced in size and captured on microfilm.

Standards: A Breakthrough?

Mankind's track record on preserving its works is a sad story and an obvious case of history continuously repeating itself. Since recorded history began, there has been a need for reliable communication standards. The biblical story about the Tower of Babel is an example of what happens without such standards. Today, languages vary from culture to culture, but there is a common thread: Language is a methodology that provides access by humans without reliance on an external technology. A couple thousand years ago, the Western world came close to adopting Latin as the common language, but then the Roman Empire collapsed and new languages developed. All the Western European languages, however, continued to use the Latin alphabet. The digital computer has continued the use of the 26-letter Latin alphabet. A worldwide numeric standard has been achieved already--all use "0" through "9" the Arabic numeral set. Further, the growing popularity of English suggests the world is well on the way toward adopting it as its common language.

In the digital world, things are quite different. Technology, rather than human interaction, is primary. Standardization that would facilitate long-term access is not a priority.

Some believe that as long as the computer industry is an unregulated for-profit industry, standards will be not be supported by that industry. Authors of the "Looming Information Age Crisis" note vendors' most often-cited reasons for not supporting particular standards are as follows:

* cost--it takes scarce time and resources away from competitive features to define and implement standards. Standards are always a cost center, never a profit center for vendors.

* poor current acceptance--no one wants to be the first to say goodbye to their proprietary technology

* short lifetime of most standards

* the need to "enhance" standards--"our ideas are better," or "there were always some unacceptable compromises made"

* the "not-invented-here syndrome"--everybody wants to be the standard; nobody wants to follow someone else's standard

History teaches that regulated industries can be standardized, at least temporarily. Good examples are the regulated or government-owned telephone monopolies, such as AT&T and British Telecom. Standards were established and maintained. All the world's telephones were interconnected. Telephone monopolies unfortunately stopped innovation, and the calling rates became too expensive. When deregulation began, the connectivity standards remained, and innovation gave the world the Internet.

Long-term preservation of visual human works including human writings, however, is a different kind of challenge. They must be available as readable (human decipherable) objects for decades, centuries, or millennia. A system subject to technology changes not only complicates the problem but also adds substantial risk of loss. How then, does one guarantee the preservation of history and allow for future processing of it? The answer may lie in the development of a computer processing system that can read analog media accurately and at high speed.

One such technology is already in use. John P. Whitley of Image Graphics notes that "electron beam microfilm recorders are used for ... the State of Virginia, Library of Congress, Cornell University, and others. They want the highest image quality attainable for future generations. The output from electron beam recorders is a primary, eye-readable information storage media that will last for centuries. It allows 100 percent information recapture when scanned and digitized as computer input microfilm. The high-resolution and sharp-edge acuity of text and data recorded on electron beam film enables a high-resolution scanner to recapture 100 percent of the text information content through the optical character recognition process."

Into the Future

Simply stated, today's processing power allows computers to convert human language media, on demand, back to the processable code of the day. Today's code is binary. One can only guess what the future holds. However, future conversion from analog microfilm to the prevailing technology of the day is predictable. When all else fails, retrieval can be accomplished using simple magnifying optics and a light source--technologies that will always be available.

Part of the long-term solution will be an optical character recognition (OCR)-friendly alphanumeric font that that allows high-speed digital conversion with 100 percent accuracy. It must also be easy to read when optical projection is used. An OCR software package must be optimized for the selected font. Existing standardized 16mm and 35mm roll microfilm formats can be used into the foreseeable future.

By interactively combining the 500-year migration-free strengths of microfilm and the access capabilities provided by digital technology, a document management system can be designed that will satisfy all known life cycle requirements in the most economical manner for the longest period of retention needed.

At the core

This article:

* Explains the problems presented by digital preservation of information

* Explores long-term preservation methods

References

Lysakowski, Rich and Zahava Leibowitz. "Looming Information Age Crisis Expected to Cause Trillion-Dollar Losses Over Next 20 Years" (also known as "Titanic 2020--A Call to Action"). The Collaborative Electronic Notebook Systems Association, 2000. Available at www.censa.org/html/Publications/Titanic2020_PDFv3-03-22-20.pdf (accessed 25 November 2002).

"Replacing Paper: Bad News for Trees," The Economist. 19 December 1988.

Whitley, John P. "`E-Beam Film' Exposes Service Bureaus to Specialty Vertical Markets." Image Graphics Advanced Digital Image Technology. Available at www.igraph.com/PressReleases/WP_ebeamfilm.htm (accessed 25 November 2002).

Russ Burkel, former Vice President of International Sales and a Founder of Eye Communications, is now a consultant on microfilm and digital information systems and heads the Film-Based Imaging Association. He may be reached at [e-mail].