Managing Digital Information: The Emerging Technologies.

Technology and information are inseparable aspects of the information environment. Technologies that will significantly affect the future of information and records managers are identified and discussed here. Some are already before us, though not widely known. Some are on the threshold; and some are noted in anticipation of what will come. Storage and retrieval technologies will require in-depth understanding from information and records managers to gain the best benefits, assure adequate digital records management, and avoid incompatibilities that result in orphaned information in obsolete systems.

On the threshold of the millennium, technology moves more quickly than organizations can comfortably react. New digital media emerge in ever-faster technology development cycles. The volume of digital data outstrips that in hard-copy records. Thanks to high-speed networks, information is in constant motion around the globe. Information managers struggle daily to understand the new technologies and their applications. They must race to stay ahead of the curve. The struggle will not get any easier.

Dramatic strides are now being made in the laboratory, strides that will further enhance our record keeping ability but that also will make our professional lives more complex. Three developments promise dramatic increases in computer storage capacity: holography, nano-CDs, and electronic paper/ink. To find the enormous amounts of information stored on these new media, structure-based mark-up languages, new eigenvector search engines, and intelligent agents are tools that may offer new modes of effective information retrieval.

Information managers need to understand both the storage and the retrieval technologies in order to gain maximum benefits, assure adequate digital information management, and avoid incompatibilities that result in orphaned information in obsolete systems.

Holography

We know holograms as familiar anti-counterfeit devices on our credit cards and now on paper currency around the world. But they are much more than that. Holograms are devices that store data through the interaction of light waves on special materials. Because holographic images have three dimensions, information can be stored in layers inside a hologram. Magnetic and optical disks store data serially -- that is, in sequence, one piece after another. Holographic storage can stack about 40 pages of information in the same small location within the hologram. The data is read by tilting the angle of the beam of light used to read it.

Although early holograms were created on photographic film, today's photopolymers and photo-refractive crystals provide greater storage possibilities. Chemically, a polymer is a long chain of several single molecules, or monomers. Plastics and Plexiglas[TM] are examples of polymers familiar to us from daily life. Polymers that have potential for holography have excellent recording properties (i.e., high light sensitivity, high resolution) even when very thick. Photopolymers are used in WORM devices; photo-refractive crystals are erasable.

Scientists are currently researching issues related to the angle, wavelength, and placement of images to increase storage capacity. Lasers are the most common light source both for recording and reading these holograms.

One example of a holographic storage system is IBM's HESS technology. At its current stage of development, the system can store more than a dozen times the capacity of current mainframe hard disks. It can read and write data 10 times faster than today's systems since many pages of information can be accessed simultaneously. Each page of holographic information can contain more than a million bits, and the system can store thousands of pages on a crystal no larger than a dime. With this capacity, a computer the size of a watch could hold gigabytes of information. But this is only the beginning. NTT of Japan has an experimental technology that can record 30 hours of video on a crystal the size of a fingernail -- a density of 10 trillion bits per square centimeter.

Even though current laboratory efforts have shown that holography can store massive amounts of information within the tiny crystals, until recently retrieval of the information has often meant destruction of the information on the media. The information in a holographic crystal is stored in regions of differing brightness and darkness created by two laser beams that have the same wavelength but differ in phase. (Holograms are created by beams of light with the same wavelength, whose starting points, however, are not the same. This out-of-synchronization [i.e., not in phase] is the condition that allows the holograms to be created.) Holographic crystals rearrange their electrons in response to these beams, thus creating a necessary interference pattern.

During the retrieval process, laser light of the same wavelength was used to illuminate the crystal and to recreate the original interference pattern. However, when the illumination once again excited the electrons, the act of retrieval destroyed the pattern. The latest approach uses specially treated crystals and a combination of laser and ultraviolet light to "brand" the information onto the holographic crystals. The combination of two light sources creates the interference pattern, but only one, the red laser, is required to retrieve the information.

Holographic crystals have a theoretical storage limit of 125 MB per cubic centimeter or about 1,000 times more storage than currently available on RAM chips. When this type of capacity is available to the ordinary user in five to 10 years, information managers will face an increasingly difficult task of managing these volumes in a structured and rational manner, particularly since the holographic media will be compact enough that an individual will no longer need to depend upon a server to store large amounts of information.

Nano-CDs

Moving from light waves to molecules, we enter the world of nano-technology and nano-metrics. "Nano" is the term used to designate a billionth of something -- in this case, a billionth of a meter. At Princeton University's Nano Structures Laboratory, experiments proceed to store information at the molecular level on silicon disks the size of small coins. At its current stage, the technology is similar to CD-ROM publishing.

The process itself is called "nano-imprint lithography." This approach avoids the need to irradiate the storage material (again a type of polymer) with electrons or X-rays, a costly and commercially infeasible approach. Before the storage disk is created, a mold with microscopic grooves is prepared. Then, a soft polymer about the size of a penny is stamped with the mold. The data is recorded as tiny pits in the grooves. The data bits are 10 nanometers (billionths of a meter) wide and 40 nanometers apart. These data bits are more than 25 times smaller than those on current CDs. With this compaction ratio, they can fit the same information as a normal CD into a space about 800 times smaller.

A silicon "needle," the nano-modifier, is used to read the information on the disk. The needle vibrates. As it comes close to the disc, there is molecular interaction between the needle and the disk. As a result, the vibration of the needle changes. By tracking the changes in vibration (zeroes and ones register different vibrations) the recorded data can be decoded.

Reading the information on nano-CDs also requires a special technique since the nano-CD's pit and track sizes are nearly two orders of magnitude smaller than the wavelength of the laser light used in current CD players. The solution was to use a tiny needle to pick up information from the nano-CD surface, similar to the way the needle on a record player does except that the needle does not completely touch the nano-CD. Researchers found that with a special method, called "tapping mode," a silicon needle can read a disk millions of times without causing significant wear.

The storage capacity of a nano-CD is 50 GB. That provides a significant increase over a CD's current capacity of 650 MB. For information managers, the enormous storage capacity would complement the requirements for storing multimedia or graphic information. A nano-CD could hold five hours of commercial quality video (about 800 times the storage capacity of current CDs) or archive hundreds of thousands of CAT-scan or magnetic resonance images. However, the latter application will need to wait until the development of a rewritable nano-CD. Currently, the technology permits only CD-ROM type publishing.

Electronic Paper/Ink

Science fiction has long described newspapers that can be regularly updated with current information. The stories change on the page as new information is broadcast by the publisher. Engineers at Xerox's PARC research center and the MIT Media Laboratory are creating versions of what might be called "electronic paper" or "electronic ink" which would change science fiction to simple fact. Both efforts focus on paper-like products that can display digital information, text, and graphics.

In both instances, the technology is similar. It is based on the principal of electrophoresis. This term describes a phenomenon where molecules separate based on size, charge, and their speed through an electrical field.

Xerox's electronic paper is composed of three basic elements: balls, binder, and oil. The balls are plastic, tiny (1 micron to 1 nanometer), and specially treated to retain a specific electrical charges. Each half of the ball is colored differently, most commonly black and white. The binder is transparent and usually silicone based. It is heated into a thin film and spread with a single, regularly arranged, tightly packed layer of the balls. After it has cooled and been cut into sheets, the composite material is soaked in an oil. The oil is absorbed and forms lubricating pockets around the balls encapsulated in the binder. The oil allows the balls to rotate.

The difference in the approaches by Xerox and MIT is in the final application. Xerox has focused on thin, flat objects, such as book pages, hence the name "electronic paper." MIT, on the other hand, has centered its development on a substance that can be printed on paper and other substances, hence the name "electronic ink." In MIT's approach, the ink is in liquid form until it is cured on the final substrate, or recording layer.

Electronic paper/ink works in the following manner. Each half of each ball carries a different electrical charge. When the ball is exposed to an exterior charge, the ball rotates so that the side of the ball with the opposite charge is nearest the exterior charge. That is, the side of the ball with a positive charge moves toward a negative exterior charge, and the side of the ball with a negative charge turns to the positive charge. The result is that only one of the two sides of the ball, black or white, is now visible to an observer. The patterns of balls, of course, can be programmed to display text or graphics. By varying the completeness of the rotation, exposing only part of the white or black sides, shades of gray are also possible. Current research focuses on how to mix balls of various hues to create full-color images.

Initial tests show that the electronic paper/ink is durable. Xerox has shown media image stability for more than two years and 3 million cycles of create text/erase text. The first applications are expected to be billboards or retail signs. An alternative application under development would permit the creation of text directly on the electronic paper through the use of a stylus with the appropriate charge to change the direction of the balls from "off" (white) to "on" (black), thus creating handwritten text on the "page."

The ultimate application of interest to information managers will be electronic books whose pages can be created, updated, and ultimately erased by electronic signals fed to the "book." Each page would have a set of electrodes connected to a chip in the spine of the book. The electrodes would permit the electric charge to be passed to each ball on the page. The major limitation today is the absence of a technology that will allow electronic signals to be transmitted through the spine to the pages. Research continues on the development of a flexible backplane that would hold the assembled pages, power sources, and transmission circuitry.

Structure Based Mark-up Languages

One of the major problems that information managers face with digital information is the need to identify the type of documents being maintained on the myriad of disks and servers in their organizations. Retention cannot be assigned, nor disposition carried out, if the content and value of the information cannot be discerned. However, almost all documents have some type of structure. This structure relates both to content (text, graphics, tables, etc.) and to the role of that content (e.g., in a report, a management summary has a different content, structure, and meaning than a footnote, which functions differently from the caption of a graphic or the tabular data extracted from a database table, etc.).

One new development that can assist in the identification process has actually emerged from the laboratory and is quickly being implemented for documents on the Internet. This approach uses a structure based mark-up language to identify structures in a document. The current effort has focused on the extensible mark-up language (XML). XML defines a standard way to mark up documents with coding that specifies both structure and content.

To accomplish this task, XML uses tags similar to those used in the more familiar HTML. However, XML does not predefine the tags as HTML does. Rather, XML is a meta-language -- it provides the rules for creating tags that can be interpreted by applications or style sheets to identify content and structure within a document. In other words, XML provides a facility to define tags and the structural relationships between them. Since there is no predefined tag set, there are no preconceived and therefore limiting semantics.

XML is powerful because there are theoretically no limits to the type of data that can be structured. For example, XML could define <customer> as a data concept. Within this data concept would be included the components and relationships that belong to <customer> such as <customer name>, <customer ID>, <tax ID number>, <address>, etc. Once these are defined, they can be used within documents and can be recognized across applications that share the <customer> definition. A major practical example is the need for financial institutions to be able to exchange information. An effort is under way to convert the Open Financial Exchange (OFX) specification to XML. The result will be the ability of institutions and customers alike to retrieve and exchange data without worrying about the specific applications used, as long as each of the applications supports the OFX standard.

For information managers, this approach can provide an opening to the difficult problem of document identification across a LAN/WAN. For example, an organization could define the data concept <contract>. Within <contract>, various components could be defined: <contractor>, <contractee>, <initiation date>, <expiration date>, etc. Once defined, an intelligent agent (cf., the last technology discussed in this report) could be routed through the LAN/WAN to identify all <contract>s with an <expiration date> prior to the current date and then to calculate if the retention period had been completed and the document eligible for final disposition. On a practical level, this approach would be most effective through the use of sty]e sheets employed by the user while creating documents.

Eigenvector Search Engines

Despite the promise of structure-based mark-up languages, there are other occasions where a tool like XML will not work. One instance is when documents of varying types have a relationship to one another (e.g., a proposal, a contract, and a financial analysis all related to a specific project). This information management problem will become increasingly evident as organizations move to an intranet-based technology used by widely dispersed work teams.

IBM is developing a promising approach to create order out of the Internet/intranet chaos. It is a search engine that uses what is called "eigenvector analysis." (The eigenvector engine searches widely first and gathers many sites [the "root" set]. Then, from that large number -- say, 100,000 -- the engine uses a linear algebra formula to identify and weigh "authority" sites to which many other sites link. Thus "vectors" are created to the most important and authoritative sties.) A more mundane definition might be a search application that discovers the home, or source, location of a topic and then the paths (vectors) to the related items.

Internet tools such as Yahoo! and Infoseek are effective because humans help to organize the material that the search engine indexes. However, the exploding growth of Web information and the need to catalog information on intranet or virtual private networks not open to these search tools requires a more automatic approach. IBM's solution, named CLEVER (client-side eigenvector enhanced retrieval), is to compile automatically a resource list on any widely present topic on the Web. Through automation, the available resources are identified more swiftly and the topic indexes updated and maintained more frequently. IBM has studied the quality of its results in comparison to human-generated lists and claims that the automated approach does not result in a significant loss of quality.

CLEVER is different because it analyzes how documents are linked to each other on the Internet. The core of the system is an algorithm that brings together both the context of each page's content -- an improvement over merely literal "hits" on discrete words -- and the hyperlinks that create the overall topic space. CLEVER identifies what it calls "authorities" cited by other documents on a subject and the "hubs," or sites, with multiple links to authorities.

For example, a search for "New York taxation" might generate an initial list of 200 to 300 pages. The search would then expand automatically to include documents linked to and from those up to a predefined limit, perhaps 10,000 items. The engine then analyzes and ranks those by assigning greater importance to the most frequently cited pages -- the authorities. Based on the number of links to the authorities, the system then identifies the hubs or focal sources. Note, however, that in this case, both "New York State" and "New York City" references would be intermixed. No engine has been developed to infer that type of distinction from a simple search phrase. Of course, with Boolean qualifiers, a more refined search is possible.

For information and information management purposes, such an engine may be an effective tool for designing a directory structure for a wide-spread LAN/WAN. A specific set of searches on key record series or functional document types (e.g., XML-identified contracts) would create a logical structure of the interrelationships among various objects and permit an information manager to plan a set of directory structures that would foster the maintenance, archiving, and disposition of digital documents. Together with intelligent agents, the possibility of effective control over digital information comes closer to reality.

Intelligent Agents

Intelligent agents are software applications that follow instructions given by their creators and which "learn" independently about the information they are programmed to gather. They save time and simplify the complex world of information management. An intelligent agent can anticipate the need for information, automate a complex process, take independent action to improve a process, or communicate with other agents to collaborate on tasks and share information.

The following characteristics help to differentiate intelligent agents from other software applications:

* They are always autonomous -- each agent controls its own actions.

* They are goal driven -- each has a specific purpose and acts in accord with that purpose.

* They are reactive -- an agent responds to changes in its environment.

* Each is continuous -- it is always in operation.

Some agents have the following characteristics:

* social -- agents which interact or communicate with other agents

* adaptive -- some agents can change their actions based upon previous experience and new encounters

* mobile -- agents move from machine to machine

* believable -- the user can see or hear the agent

The characteristics of an ideal agent include:

* small size -- 4-80KB so that mobility of the agent is not impeded

* intelligence -- the agent must be able to learn as it operates

* persistence -- the agent should not require owner intervention for operation

* flexibility -- the agent should be able to negotiate alternative routes

* mobility -- an ideal agent must travel the correct course and return to its "home site"

* hyperoptic -- a good agent recognizes related information even if not explicitly included in its search criteria

* security -- an agent is resistant to interception, immobilization, and tampering 

* discretion -- agents are invisible to both the user and the systems through which they travel

Intelligent agents, sometimes called "smart assistants," are currently being used for a wide number of tasks. In one application, the agent learns what drugs a doctor in a given practice prescribes under certain circumstances and in response to particular symptoms. In great detail, the smart assistant can recommend a particular drug (generic or branded), the dosage, the frequency of administration, duration of treatment, and circumstances for administration (e.g., taken before or during meals). With this tool, the physician's assistant can prepare a prescription for review by the doctor as an independent task during an office visit.

Other examples include customer help-desk applications that provide a coherent structured answer to a customer's problem from the past experience it has gained, the knowledge it gains or has gained from examining the user's system, and basic information about the hardware and software downloaded from the manufacturer. Finally, there are those agents that add intelligence to Web browsers. These applications can track the user's wandering through the Web as well as bookmarks or favorites and the paths traveled to get to them. They can then provide a unique, customized, personalized view of the Web, with suggestions for additional paths related to explorations that users may wish to follow.

Managing Information in Digital Forms

The most sophisticated agents are memory-based reasoning systems that rely on "associative" memory to accomplish their tasks. The key to this approach lies in its focus on the relationship between attributes of different cases, :not the content of the cases. Thus, it is not necessary to maintain comprehensive databases of the content of past decisions. With an integrated rules-based component, these agents offer promise as a tool for effective digital information and information management.

Digital information management agents would encompass the following capabilities:

1. They would travel throughout an organization's networks -- from desktop PCs, to network servers, to mainframe data farms, to the organization's Internet, intranet, or extranet sites.

2. They would each have basic initial definitions and rules about a specific type of record created by the organization, the retention rules the organization has defined for that record, the holder of the "official" copies of the record type, the retention process and medium for each type of record, and the expected actions (e.g., allow to "remain," "move to an archive," or "purge").

3. Based upon the above rules, they would evaluate digital objects throughout the various networks, classify them, and then act upon them in accord with the defined procedures.

4. When agents discovered new or ambiguous types of information, they would attempt to classify them in accord with existing information management rules and structures, and recommend a tentative mode of action. Final actions would depend upon authorization by the information management operation.

5. The agents would maintain an audit trail of all their actions and authorizations to provide the organization with the means to reconstruct how a particular record was maintained.

Conclusion

The old adage proclaims that no one can avoid death or taxes. Information managers in the 21st century will need to add technology to that list. The six technologies discussed here will clearly influence how we manage information. Diligent future-focused attention to these and other developing technologies will allow information managers to anticipate changes and to stay ahead of the technology curve during the next 50 years. In order to understand in depth the nature and uses of these and other emergent technologies, information and records managers must make a commitment to keeping currently aware of the technologies through reading the literature, attending trade shows, and visiting exhibits at professional conferences.

AUTHOR'S NOTE: Information presented in this report was gathered from the Internet using the search engines available through www.theminingco.com, www.dogpile.com, and www.isleuth.com. (Regrettably, these resources are not eigenvector based.)

Alan A. Andolsen, CMC, CRM, is President of Naremco Services Inc., a management consulting firm that has created information and records management programs since 1948. During the past decade, Andolsen has pioneered practical techniques for the management of digital records. He is Vice Chair of the Board of Directors of the Association of Management Consulting Firms and Chair of its Code of Ethics Committee. He is the author of "Office Systems Management" in the AMA Management Handbook, editor of Management Consulting: A Model Course, and has served as Visiting Lecturer at Rutgers University and the Ecole Superieure de Commerce de Paris.