The indiana University Electronic Records Project: Lessons Learned.

For records professionals, the decade of the 1990s will undoubtedly be remembered as a period of intense and passionate debate about a host of issues related to the role of the archivist and records management professions in managing and preserving society's documentary heritage.

Among the questions being asked: What do archivists and records managers contribute to society? What is their relationship to other information management professionals? What theories, principles, and techniques will continue to guide records professionals in their work? When one looks for the primary factors fueling this debate, two primary contributors stand out: the rapid technological changes of the last two decades and the increasing dependence of society on electronic or digital documentation (Barry 2000; Bearman 1994; Cook 1994).

Various research projects have addressed the challenges presented by electronic records. The most prominent are those devoted to developing basic requirements for recordkeeping systems (See DoD-STD-5015.2; National Archives of Australia) and to identifying documentation or metadata that must be present to create reliable and authentic records (See University of Pittsburgh 2000; National Archives of Australia; McKemmish and Acland 1999). However, few implementation projects have been designed to test any of these theories and concepts. The Indiana University (IU) Electronic Records Project is an implementation project begun five years ago. It was designed to develop a strategy and methodology for incorporating recordkeeping requirements into IU's transaction processing and information systems. The project staff has asked the following questions:

* What is the archivist/record manager's primary contribution to the management of digital objects?

* Will traditional methods of identifying, appraising, and describing records still have value in managing electronic records?

* Will archivists/records managers have to change significantly the way they do business?

* What new skills will be needed?

* What changes need to be made to transaction process systems to make them function effectively as recordkeeping systems?

* How does one insert the archives program into the process for designing, analyzing, and auditing electronic information systems on the IU campus?

* Who are an archivist's strongest allies in the management of electronic records, and which issues will resonate with these partners?

These questions, along with lessons learned in the implementation process, are addressed next.

Defining the Archivist/ Records Manager's Unique Contributions

LESSON 1: Records professionals must define their primary and unique contributions to managing digital resources. The profession must not only define itself but also articulate the mission of archives/ records management in relation to the goals and objectives of other related data and information management professionals.

Organizations collect, create, and use a wide variety of recorded documentation:

* data, or the raw facts, about the organization and its business transactions

* information, definable as "data that has been refined and organized by processing and purposeful intelligence" (Whitten and Bentley 1998)

* documents, "a grouping of formatted information objects regardless of medium or form that can be accessed and used by a person" (Sutton 1996)

* knowledge, which is defined as something more than information because it includes the expertise, logic, and reasoning developed by accomplished experts in a specific field to solve problems and make decisions (Turban 1993)

* the concept of a business record, which archivists argue is a specific and unique type of information quite different in its creation and purpose than any of these other types of recorded documentation

Archivists have identified two distinguishing characteristics of records. First, records reflect business processes or individual activities; a record is not just a collection of data but the consequence or product of an event. Of course, this concept is not new; older definitions identify records with a process or an activity. What is new is an insistence on defining more precisely and conceptually when a record is created by the business event or personal activity. The other part of the definition of a record stresses that records provide evidence of these transactions or activities. In other words, recorded documentation cannot qualify as a record unless certain evidence about the content and structure of the document and the context of its creation are present and accessible (Cox 1994; Cox 1996; Dollar 1992).

What do archivists/records managers contribute? What is their role in any partnership? The IU Archives team has defined its mission and its contribution as (1) the identification and appraisal of records generated in the context of business processes and (2) the creation of systems that capture, manage, and preserve these records.

In other words, records and recordkeeping systems are its main and primary responsibilities.

In the past few years, some archivists and records managers have argued that these professions need to become more involved in related disciplines such as information management and knowledge management. This change would be a grave mistake. Recordkeeping is what archivists and records managers do best, and few others in the data and information management community are committed to or trained in preserving reliable and authentic records. Recordkeeping is itself a full-time job, and it is archivists' and records managers' unique, though complementary, contribution to management of the institution's resources.

Partnerships

LESSON 2: Forming partnerships with other information professionals is essential.

Effective management of electronic records requires the archivist/ records manager to form partnerships with various professions (Bearman 1995). What has not been sufficiently demonstrated through experience is who the most important partners are and how these partnerships will work. Three partners at IU have been the most valuable -- decision support personnel, systems analysts, and internal auditors. Of the three, the IU Archives has had the most experience and success working with internal audit.

Why is internal audit such a useful partner for the archivist/ records manager? The answer can be found first in the fact that the missions of the two professions share many concerns. Both professions are interested in creating systems that are accountable, compliant, and trustworthy; systems that produce accurate, reliable, and authentic records. Both professions acknowledge the importance of risk assessment and the value of understanding business requirements. More than their similarities in mission, however, make this partnership or any partnership work effectively. Participants must also gain something valuable and unique from the collaboration. To be effective, the partnership must result in a mutually satisfying scenario.

In the IU Archives/Internal Audit partnership, mutual benefits occur in several ways. The audit process employs a methodology based on detailed sampling of certain transactions while the objectives of the archives methodology is to achieve a broad, but less detailed, analysis of all system functions. When combined, these two methodologies complement one another, and both partners agree it results in a much more detailed, more complete analysis of the system.

The two professions also complement one another in terms of the functions or issues they focus upon in their analysis of systems. For auditors, the primary concerns include data authenticity and accuracy, system security, adequate internal controls, documentation (written procedures and instructions), and backup procedures and contingency planning. The archives staff, on the other hand, tends to focus on recordkeeping issues such as record identification and capture, record metadata, access, and long-term preservation. Again, the result is a complementary analysis, which provides each partner with new and valuable information. The auditors gain access to previously unavailable analyses of systems from a recordkeeping perspective, and the archivists gain detailed knowledge about system security, data accuracy and input procedures, and system documentation.

Ultimately, however, the greatest advantage for the archival program in this partnership may be that it involves archives staff in the authorized and routine review of information systems. The value of partnering in the systematic, daily review of systems with a unit such as internal audit, which has an institutional mandate and the authority to conduct these reviews, cannot be overemphasized. Working with audit is an effective strategy for inserting the archives/records management program into the mainstream process of designing, analyzing, and modifying electronic information systems (Bantin 1999).

Skills Required

LESSON 3: Most archivists involved in electronic records management recognize that the archival profession needs to add some new skills to its "tool kit" to be effective in a world of increasingly automated records.

Based on the IU experience, the most difficult question is not whether new skills are needed but rather which skills and in what amount. In other words, how much specialization is needed? How do records professionals determine when they have acquired enough information to do their jobs? At what point does one hand the ball off to partners to let them do what they naturally do best?

In addressing these questions, an observation by Margaret Hedstrom is valuable: "The challenge is to strike a balance between teaching known methods and techniques -- built around traditional archival functions -- and teaching archivists how to evaluate, apply, and modify theory to address the changing needs of advanced technology" (Hedstrom 1993). In other words, to be a good manager of electronic records, a solid grounding in basic archival principles and techniques is essential. Beyond that, however, experience has demonstrated that new and different managerial and technical skills are required to be successful in the world of automated records.

The new skill sets of importance can be grouped into three basic categories. The first is basic knowledge of automated systems and how they process data. Of particular importance here is acquiring a good working knowledge of the most prevalent systems presently being employed in most organizations: transaction processing systems (TPS), database management systems (DBMS), management information systems (MIS), decision support system (DSS), data warehouses, and electronic document management systems (EDMS).

In addition, one must obtain a thorough understanding of all metadata systems such as data dictionaries, information resource dictionary systems, and transaction logs. Understanding these systems and how they manage digital objects throughout their life cycle is the starting point for developing recordkeeping systems. Before archivists can begin designing recordkeeping systems, they must first be able to articulate why a TPS or EDMS is not performing as required.

A second set of essential skills form around information system analysis and design. Of particular importance is the ability to create conceptual models for representing records and system requirements. Of these models, the most important are business process models. These models depict the business functions and business processes -- transactions and the inputs and outputs -- required to respond to business events. The value of business models for archivists is that they can depict precisely when, where, and how record creation occurs. Process models are the blueprints, the maps for record identification and capture. They provide the archivist with a conceptual model based on depiction of real-life activities of the context for creation. Consequently, these models provide the information needed to describe and define for system designers those pieces of data that need to be captured as evidence of business transactions. The ability to create conceptual models of business processes as an absolutely critical and essential skill for the 21st century records professional.

Skills in a third set include those management skills required to translate this knowledge into a strategic plan. More than ever, records professionals must be effective managers able to translate a set of goals and objectives into a realistic and effective implementation project. More specifically, records professionals must learn effective techniques for communicating records program needs, capabilities, and contributions. They must develop strategies for positioning the archives/records management program within the broader context of information resources management. They must also learn how to define mutually beneficial activities and how to negotiate mutually beneficial alliances.

Finally, records professionals need to maintain the proper perspective or balance in the acquisition of new skills. They need to keep in mind that the goal is to manage and preserve records created in the normal course of an organization's business. Records professionals are not managers of data, information, knowledge, or documents. Keeping focused on records management issues will help us stay focused on the basic skills required. From a technology point of view, the goal is not to become a programmer, systems analyst, or decision support specialist but rather to be a records professional who can speak the language of the technologist, understand how various data and information systems function, and be able to perform some basic tasks related to modeling and describing business processes.

Recordkeeping Deficiencies Commonly Encountered

LESSON 4: The primary dam and information systems employed by most institutions are poor recordkeeping systems.

The most basic business system and the heart of most organizations is the transaction processing system (TPS). A TPS is a computerized system that performs and records the daily routine transactions necessary to conduct business.

The primary goal of these systems is the automation of computing-intensive business transactions such as those undertaken in the financial and human resources functional areas. The emphases are on processing data (sorting, listing, updating, and merging), reducing clerical costs, and producing documents required to do business such as bills, paychecks, and orders.

The guiding principles of these systems focus on creating data that is current, accurate, and consistent. To achieve these goals, these systems employ database management system (DBMS) software. A primary advantage of DBMS is its ability to limit and control redundant data in multiple systems. Instead of the same data field being repeated in different files, the information appears just once. Another advantage of DBMS is that it improves data integrity. Updates are made only once, and all changes are made for that data element no matter where it appears.

TPS: A Two-Edged Sword

Without question, TPSs are very good at supporting current business needs for information, minimizing the amount of data stored in the system, improving overall efficiency of the system, removing obsolete data, and providing an organizational resource of current data. Archivists universally agree, however, that these systems do not routinely and systematically capture records. In these automated systems, business records are not routinely stored as stable, finite, physical entities. Rather, these systems create records by combining and reusing data stored in discrete units organized into related fields that form files. Once created, a record of a business process may not -- indeed, likely will not -- be captured as a physical entity.

Not only will the record not be captured at the time of creation, it may be impossible to recreate it at some later date. Databases are dynamic, volatile systems, in a state of continual change. Data updates occur frequently, and with DBMS software managing the system, these revisions are made in every file containing that revised data element. Moreover, databases typically maintain only the current value for any given data element. Historical data, if kept at all, is usually incomplete or summarized. Consequently, historical snapshots of a database do not routinely capture the data values needed to reconstruct a specific record.

Even if all data values are captured in historical versions of the database, archivists argue that the system is still not capturing and preserving records. Retaining database tables preserves data, not records. In most automated systems, the physical relationship between the record content and the metadata that gives the content meaning often does not exist. Vital links between metadata and the record content data may exist only in a computer software program or may not be a part of the automated system at all. It may exist only as a paper document totally disassociated with the records it describes.

Archivists have also discovered that system metadata as defined by systems designers and technologists are typically not as complete as necessary to describe a record. Transaction logs maintained in a typical TPS do contain some critical data on updates and revisions, but on the whole, archivists generally agree that these logs do not provide sufficient evidence.

Of particular concern is the relative lack of metadata related to the context of creation and use -- metadata that addresses the questions of why the record was created, who were the users of the record, and who had custody of the record. The availability of this contextual metadata, archivists argue, could make the difference between a useful and a useless record, particularly when viewed over longer periods of time.

Another deficiency of typical system metadata from a recordkeeping perspective is the absence of some critical documentation on the structure of the record. Of particular importance is structural metadata describing how to open and read a record as it was originally created and viewed. Taken as a whole, the absence of critical metadata has meant, as one archivist has noted, "most collections of electronic data, electronic documents, or information are not records because they cannot qualify as evidence" (Bearman 1994).

New Techniques and Methodologies Required

LESSON 5: Traditional records management strategies established for paper records need to be altered in significant ways to accommodate electronic records.

Experience would strongly suggest that records professionals have to implement new techniques and methodologies to be effective in the digital world. Whether this change results in a major overall reengineering of the process is not clear. Very likely the changes will involve more than a mere tweaking or refining of traditional methodologies for managing records. Records professionals will have to devise new visions and new strategies for addressing some of the most basic issues, such as how to identify and capture records in automated environments, how to ensure that inviolate records are preserved for as long as necessary, how to appraise the value of records, how to document records, how to ensure that a complete record is captured -- and at what point records professionals undertake these tasks. Two profound changes that need highlighting are discussed next.

Archivists Involved Throughout the Records Continuum 

Experience at IU and other institutions would strongly indicate that the archivist's role in managing electronic records as articulated in the traditional life cycle model is far too limited. Experience would suggest that to be successful in managing electronic records, the archivist must be part of the team managing records throughout their life cycle -- from their creation (and in the system's design before creation) through to the preservation and use of records as archives.

Experience suggests that a more effective model or framework includes a more expansive role for the archivist such as that defined in the newly formulated conceptual framework known as the "Records Continuum Model." Perhaps the most basic difference between the continuum model and the life cycle approach is that although the life cycle model proposes a strict separation of records management responsibilities, the continuum model is based on an integration of the responsibilities and accountabilities associated with the management of records.

It is a strategy that undercuts and destroys the distinction between active and inactive, and archival and nonarchival records, and blurs or wipes out the defined set of responsibilities associated with managing records at each stage. As a result, archivists and archival functions are propelled forward in the records management process. According to the continuum model, strategies and methodologies for appraising, describing, and preserving records are implemented early in the records management process, preferably at the design stage, and not at the end of the life cycle.

Advocates of this newer model argue that, absent a strategy for active and early intervention by the archivist in the records management process, those electronic records documenting vital transactions may never be created, may never be fully documented, or may never survive (Upward 1996).

The Value of Conceptual Models

A second profound change is the creation of an overall strategy that views conceptual model building as the primary methodology for dealing with many or most of the issues the profession faces in attempting to manage records in automated environments. In other words, rather than physically reviewing records and systems to conduct such basic activities as appraisal and description, records professionals would be creating and employing conceptual models designed to analyze and document record-producing systems.

Using this approach as the equivalent of a traditional records survey would assist the creation of business process models (i.e., conceptual models of functions and transactions that identify business records). Appraisal of records would still be undertaken by employing traditional appraisal values, but the analysis would be based on conceptual models of the processes and records rather than on a physical review of data content. Evidential values would be derived from business process and metadata models. Informational values would emerge from reviewing data and metadata models. In documenting records, a complete, authentic, and reliable record would be captured, not by physically reviewing the record but by analyzing metadata and data models and by comparing the results to an established set of metadata specifications and recordkeeping requirements.

Conclusion

The digital world presents great opportunities but great risks as well for records professionals. Plenty of evidence exists of what appears to be an international concern that automated systems are out of control and need to be better managed. Records professionals are increasingly viewed as part of the solution, even if their role is not yet truly defined or clearly understood. Therein lies the opportunity.

The risks records professionals face are that they will not adapt to the changing environment and, consequently, will not meet the challenge. In meeting the challenge, they must at the very least continue to make the case for recordkeeping systems and not abandon their role or mission as the primary manager of this type of digital resource.

However, it will require more from them than that. Experience at IU has demonstrated that records professionals working with automated systems will need to make some significant changes in the way they do business. Traditional methods for identifying, appraising, and documenting records will not be effective without major modifications. Records professionals will need to develop new strategies and techniques to gain entry to the systems and access to information managers and system documentation. In other words, the mandate and lines of authority archivists/records managers created for paper records will not necessarily carry over into the automated world. New partnerships and new management skills will be needed to place records professionals firmly within the process for designing, analyzing, and auditing electronic information systems.

A rigid allegiance to strongly held notions of the past on how to manage records seems counterproductive in the present environment characterized by rapid changes in many aspects of records professionals' lives. What is needed are archivists who are willing to experiment with creative combinations of ideas, old and new; who are courageous enough to seek out and form partnerships with information specialists whose language and methodologies are presently foreign to them; who are motivated to learn new skills; and ultimately, who are committed to developing realistic strategies for managing electronic records, no matter where this journey may lead them (Bantin 1998, 30).

AT THE CORE

THIS ARTICLE EXAMINES:

* how records professionals must define their primary and unique contributions to managing digital resources and to developing new skills and new partnerships

* ways in which primary data and information systems employed by most institutions are poor recordkeeping systems

* altering traditional records management strategies established for paper records to accommodate electronic records

REFERENCES

Bantin, Philip C. "The Indiana University Electronic Records Project Revisited." American Archivist 62, no. 1 (Spring 1999): 153-163.

--. "Strategies for Managing Electronic Records: A New Archival Paradigm An Affirmation of Our Archival Traditions?" Archival Issues: Journal of the Midwest Archives Conference 23, no. 1 (1998): 30.

Barry, Richard. "The Changing Workplace and the Nature of the Record." Available at: http://www.rbarry.com/aca-pv16/ACA-PV16.html (accessed 9 December 2000).

Bearman, David. "Archival Strategies." American Archivist 58 (Fall 1995): 380-413. --. Electronic Evidence: Strategies for Managing Records in Contemporary Organizations. Pittsburgh: Archives and Museum Informatics, 1994.

Canadian State Archives. "Recordkeeping in the Electronic Work Environment -- Vision." Available at: http://www.archives.ca/06/0603_e.html (accessed 9 December 2000).

Cook, Terry. "Electronic Records, Paper Minds: The Revolution in Information Management and Archives in the Post-Custodial and Post-Modernist Era." Archives and Manuscripts 22 (November 1994): 300-328.

Cox, Richard. "The Record in the Information Age: A Progress Report on Research." The Records and Retrieval Report 12, no. 1 (January 1996): 1-16.

--. "The Record: Is It Evolving?" The Records and Retrieval Report 10, no. 3 (1994): 1-16.

Dollar, Charles. Archival Theory and Information Technologies. The Impact of Information Technologies on Archival Principles and Methods. Macerata, Italy: University of Macerata, 1992.

Hedstrom, Margaret. "Teaching Archivists About Electronic Records and Automated Techniques." The American Archivist 56, no. 3 (Summer 1993): 427.

Indiana University Electronic Records Project. "Functional Requirements for Recordkeeping Systems." Available at: http://www.indiana.edu/~libarch/funcreqs.html (accessed 9 December 2000).

Indiana University Electronic Records Project. "Information on the IU Electronic Records Project, Phase I and Phase II." Available at: http://www.indiana.edu/~libarch/phase.html (accessed 9 December 2000).

McKemmish, Sue, and Glenda Acland. "Accessing Essential Evidence on the Web: Towards an Australian Recordkeeping Metadata Standard." Available at: http://ausweb. scu.edu.au/aw99/papers/mckemmish/paper.html (accessed 12 December 2000).

National Archives of Australia. "Designing and Implementing Recordkeeping Systems." Available at: http://www.naa.gov.au/recordkeeping/dirks/summary.html (accessed 12 December 2000).

Sutton, Michael J.D. Document Management for the Enterprise. Principles, Techniques, and Applications. New York: John Wiley & Sons, Inc., 1996.

Turban, Efraim. Decision Support and Expert Systems: Management Support Systems. New York: Macmillan Publishing Co., 1993.

Upward, Frank. "Structuring the Records Continuum. Part One, Post-custodial Principles and Properties," Archives and Manuscripts. 24, no. 2 (November 1996): 268-285.

U.S. DoD STD-5015.2 Functional Requirements for a Records Management Application. Available at: http://jitc.fhu.disa.mil/recmgt/#standard (accessed 9 December 2000).

The University of Pittsburgh Project. "Functional Requirements for Evidence in Recordkeeping." Available at: http://www.lis.pitt.edu/~nhprc/prog1.html (accessed 9 December 2000).

Whitten, Jeffrey L. and Lonnie D. Bentley. Systems Analysis and Design Method. 4th ed. Boston: McGraw-Hill, 1998.

Philip C. Bantin is the University Archivist of Indiana University, Bloomington, Indiana. He has more than 20 years' experience as a records professional and specializes in institutional archives management. He is a member of the Society of American Archivists. He and his institution have received grants totaling $293,511 (1995-2002) from the U.S. National Historical Publications and Records Commission (NHPRC) for developing, implementing, and testing methodologies for the management of electronic records. Bantin received the masters in library science from the University of Wisconsin-Madison in 1976. The author may be reached at [e-mail].