Library economic metrics: examples of the comparison of electronic and print journal collections and collection services. (Academic Libraries).
THIS ARTICLE DEALS WITH A FRAMEWORK of library economic metrics including service input and output, performance, usage, effectiveness, outcomes, impact, and cost and benefit comparisons. Examples of these measures are given for comparison of library electronic and print collections and collection services based on a recent cost finding study at Drexel University where the library has converted almost entirely to an electronic journal collection. These data are complemented with recent readership surveys of scientists at Drexel University, University of Tennessee, Oak Ridge National Laboratory, and members of the American Astronomical Society which describe changing information-seeking patterns and use of library electronic and print collections.
The introduction of the World Wide Web, electronic publishing, and digital library initiatives has had profound and continuing impact on libraries of all types. The emerging technologies have caused libraries, their funders, and their users to rethink what libraries are and how libraries can best serve their constituents. Sentiments have ranged from the extreme opinions that libraries will no longer be necessary, to explanations as to why the Web is not a library and that the new technologies will actually strengthen the role of librarians and other information professionals (e.g., Griffiths, 1998). Regardless of how this scenario will play out, it is clear that we must stay on top of these changing events by clearly understanding their economic implications.
This article addresses some economic metrics that can continue to shed light on the evolution of educational, research, and professional communication systems. While these economic metrics have been demonstrated for the scholarly journal system and its participants such as authors, readers, publishers, libraries, and other intermediaries (Tenopir & King, 2000), the focus of this article will be on comparisons of library electronic and print journal collections and collection services. This aspect of the system is particularly important because of the steady shift to electronic collections and the resulting difficult decisions that must be made by librarians in an increasingly complex economic environment.
Librarians continually face the need to make decisions on the selection, acquisition, access, and service policies and procedures related to electronic publications and to negotiate legal and financial arrangements with publishers, consortia, aggregators, and so on (King & Xu, 2003). More specifically, they need to decide whether or not:
* to rely exclusively on electronic journals or purchase both electronic and print subscriptions and, if so, at what price;
* to subscribe to or rely on single article demand for certain journals;
* to discard print issues or rely on them as a backup for archival purposes;
* to negotiate site licenses;
* to deal directly with publishers or rely on intermediary services such as consortia, aggregators, gateways, etc., and if so, at what price;
* to depend, in some cases, on information freely accessible on the Web as a substitute for costly electronic resources.
These complex decisions require a sound economic underpinning as well as good judgment in applying economic information and metrics.
Griffiths (2002) has briefly described the evolution of library performance measurement over the past thirty-five years starting with the Morse (1968) pioneering adaptation of operations research analysis to library performance. In the 1970s and 1980s, the Public Library Association, Association of Research Libraries, Council on Library Resources, National Science Foundation, and others sponsored a series of studies in the U.S. to develop library economic metrics, methods, and models for decisionmaking and planning (e.g., Baumol & Marcus, 1973; deProspo et al., 1973; Hamburg et al., 1974; Clark ,1976; Palmour et al., 1980; Zweizig & Rodger, 1982; D'Elia & Walsh, 1983; Buckland, 1983; Kantor, 1984; Cummings, 1986; McClure et al., 1987; Van House et al., 1987). Also during this period, Lancaster (1977, 1993) produced books on the evaluation of libraries. More recently, McClure and colleagues (e.g., McClure & Lopata, 1996; Shim et al., 2001) have focused on measures for library networked services and electronic collections.
The economic framework used in the comparison of library electronic and print collections and services in this article has evolved over thirty some years as a result of what was learned from hundreds of studies. The genesis of the framework was first published in 1971 as a result of a National Science Foundation (NSF) funded study (King & Bryant, 1971; King & Lancaster, 1969) and a similar approach that was being developed in the UK as well (Griffiths, 1977). Many of the studies performed by King Research in the 1970s were founded on this initial framework, including a range of applications such as information retrieval systems (e.g., King et al., 1972), scholarly publishing (e.g., King, et al., 1981), federal clearinghouses (e.g., McDonald et al., 1981), and scientific communication (e.g., King et al., 1976), among others. In the late 1970s, Vernon Eugene Palmour joined the staff of King Research and began to build on the framework (King & Palmour, 1974) with specific applications in the public library community (e.g., Palmour et al., 1980 a,b). In the early 1980s, some of the staff that was concerned with public library studies (i.e., Rodger, Van House, Zweizig) moved on and through the years developed one particular approach to library planning and economic assessment. This has led to recent studies of the electronic journals in libraries mentioned above. Meanwhile, the King Research staff took another approach to economic analysis of libraries, which evolved to the most recent version of the framework used in this paper. This economic analysis mostly involves numerous studies of special libraries (e.g., Griffiths & King, 1993), public libraries (e.g., Griffiths & King, 1989), library networks and consortia (e.g., Griffiths & King, 1991), studies of a few academic libraries (e.g., Griffiths & King, 1989; Montgomery & King, 2002; King & Montgomery, 2002).
A FRAMEWORK AND DEFINITION OF LIBRARY ECONOMIC METRICS
One premise of the framework is that metrics are designed to serve the perspectives of library staff and management, library users, the funders of the library, and the higher-order community served by the library. A second aspect of the economic framework is that it is applicable to the entire library, general library functions, specific services, or resources used to perform the services. The framework is described in well-established economic terms such as inputs, outputs, performance, effectiveness, usage and demand, cost-benefit, and so on. It first defines five specific metrics and then derives relationships among these metrics.
A schema depicting the framework of metrics is given in Figure 1. In this framework, one set of metrics involves inputs which include the amount of resources used to perform a service or provide a product where such resources can include staff, equipment, systems, facilities, a library collection, and so on. The input resources are often measured in the common measurement unit of dollars. There are also attributes of the input resources that can be measured or characterized. For example, staff attributes include level of education and experience or other indicators of competence. Collection attributes might include comprehensiveness, type of materials, age, and medium (i.e., print, electronic, microform). Outputs include the amount of services provided or number of items produced such as number of items loaned, reshelved volumes, electronic journal article hits or printouts. Attributes of output might include quality, timeliness, availability, accessibility, and sometimes, a price or fee charged. Performance is defined as a relationship between input and output that reflects how well a resource or service is performing, such as staff productivity (e.g., output quantity divided by staff time) or unit cost (i.e., cost per unit of service or item provided).
[FIGURE 1 OMITTED]
We feel it is important, if feasible, to measure usage in terms of the use of information provided by the library service such as the information content of an article that is read from access to the electronic collection. Too often the service output does not adequately reflect the amount of actual information use. For example, an electronic article hit or printout might not involve a reading and a reshelved issue in a current periodicals room may not have been read or might involve the reading of several articles. For this reason, we depend on a readership survey to provide estimates of the amount of reading from a library collection service and to establish factors that affect the amount of reading from this particular source. Such factors include user effort, ease of use and cost of use; availability of alternative sources of the information and their relative ease of use; purposes of reading; importance of and satisfaction with the attributes of the journal access services; and awareness of the services and their attributes (King & Tenopir, 1999).
Libraries constantly strive to improve their services in order to have a positive effect on the amount of use (reading) and on factors that affect use. Effectiveness is defined as a relationship between service output attributes and usage such as the amount of reading as a function of availability or accessibility of the collection, timeliness and speed of delivery of collection services, and the price required of users in dollars and/or their time. Similarly, usage metrics can be related to the service input costs such as the service cost per article read. In this article we define such derived measures as cost-effectiveness.
Outcomes, by our definition, are consequences of having used the information provided by the service such as a library-provided article that is read. Such outcomes are best determined by relating them to the purposes for which the information is obtained such as for primary research, teaching, life-long learning, consulting or advising others, administration, and so on. Here again, readership surveys can provide evidence of such outcomes that affect one's work (or other endeavors) such as improving the quality, timeliness, and productivity of work. For example, an outcome might be the extent to which the article information affects the quality of research or the effectiveness of teaching. With students, one can demonstrate a correlation between use of periodicals and grades (GPA). It is also useful to consider outcomes that are important to the library funders or community served such as helping achieve the goals of the organization or enhancing the disciplines of science or of society in general such as improving quality of life.
Over the past quarter of a century, there has been much made of the "value" of information and information services or products. Economists distinguish between two types of value (e.g., Machlup, 1979):
1) Exchange Value which is what one pays for information in dollars exchanged and in one's time and effort (which are usage measures) and,
2) Use Value which is measured by the results of having used the information (i.e., outcome measures).
Impact is the relationship between (1) inputs, outputs, and/or usage and (2) outcomes. For example, spending more for input resources can improve output attributes which in turn results in more use and, therefore, the possibility of more favorable outcomes. Thus, impacts can be relationships among several of the specific economic measures.
Domain metrics are characteristics of the community served by a library service such as number of persons served, how many of them are users, their education, and experience. Such characteristics can have an important bearing on the other metrics. For example,journal publication costs and, therefore, price required to break even or make a profit depend on the number of readers in a discipline (Tenopir & King, 2000) and unit costs of library services depend on the number of users served because of economies of scale (e.g., Cooper, 1979; Griffiths & King, 1993). Level of education and experience of the library's community can affect who will use library services and the extent to which they will use the services. There are positive and negative externalities inherent in the domain that can also affect the other economic metrics. For example, administration and funder attitudes can tremendously influence the library budgets and even users. In one organization studied by us, potential science users were told not to spend too much time in a company library because the library was thought by a high-level manager to be "recreational."
The heart of the comparison of electronic and print journal collections and services is a cost-benefit analysis. We rely on Bickner's approach to cost-benefit, which is a comparison of a service or product with some alternative to it (Bickner, 1971). In this case we will compare the library electronic collection with the traditional print collection, as well as the collection services related to these two media. The comparison can involve any of the metrics previously discussed such as input cost or comprehensiveness of the two collections; output and unit costs of the two collections; input, output, and performance of collection services; amount of articles read or purposes for which the articles are read from the two collections; outcomes from reading from the two collections; and domain number of potential users who can access the two collections.
In such comparisons, if the comparison is favorable to the electronic collection or service, it would be considered a "benefit." If the comparison is unfavorable, it would be a "cost" or perhaps better termed a "detriment." Note that the dollar cost of input (or dollar cost of users' time) could be either a "benefit" or "cost/detriment" as will be demonstrated in examples given later. For example, publishers could charge libraries less for electronic than print subscriptions in which case the comparison in dollar price paid would be a "benefit" for the electronic collection. On the other hand, if a publisher charges a higher dollar price for electronic subscriptions the comparison with print subscription would constitute a "cost/detriment." Of course, there are two sides to this coin in that one could compare print collections to electronic collections in which case the "benefits" and "costs" would be the reverse. Other examples of cost-benefit comparisons would be purchase of journal titles compared with use of interlibrary borrowing, document delivery services, pay-per-view, or comparison of a library collection with having no collection at all. Below we provide some recent examples of the cost-benefit comparisons of library electronic collections and services with print collections and services.
RECENT STUDIES OF LIBRARY ELECTRONIC AND PRINT COLLECTIONS AND SERVICES
The examples below are for cost-benefit comparisons of the collections and of services, which provide (1) access to electronic journal collections and (2) access to print journal collections. We use the term "access" in a generic manner since a library or its organization may not actually house an electronic journal but subscribe to its use on the Web or have access on a "per view" basis. Similarly, libraries can provide access to their current periodicals collection of issues or bound copies found on the shelves and special libraries often provide access to print journals through journal routing. Libraries also provide access to their collection to other libraries through interlibrary loan (ILL) and obtain copies of articles for their users through interlibrary borrowing (ILB). All of these journal access services are included in the discussion below.
Examples given below are from three recent studies performed by the authors involving economic cost analysis of library collections and readership surveys. The analysis of electronic and print collections includes cost finding for activities and resources associated with inputs, outputs, and performance (unit costs) of collections and services in special libraries and at Drexel University, Hagerty Library. Steps in these cost-finding studies are to identify relevant activities performed, establish all the resources needed to perform the activities, allocate the amount of resources applied to perform the activities, assign a dollar amount to the resources, and sum across the relevant activities to estimate fixed and variable costs of the collections and fixed and variable costs of the collection services. From this, unit costs of the collections and their collection services are estimated.
Recent readership surveys were conducted with scientists and engineers at the Oak Ridge National Laboratory; scientists, engineers, and medical personnel at the University of Tennessee; and scientists, engineers, and others at Drexel University. Some economic metrics are based on a survey of users of an electronic journal system developed by the American Astronomical Society (AAS). This remarkable system of core astronomy journals has a number of special features including electronic full text of all the core journals back 150 years, bibliographic links, citation links to and from journal articles, a relatively complete searchable database of abstracts, and links to numeric data and images. The journals are available to individuals and libraries in electronic and print media. Two user surveys with over 1,000 responses provide data and information on reading from library electronic and print subscriptions to these journals. In particular the surveys provide data concerning the relative extent to which library electronic and print versions (as well as other sources) are read and the surveys highlight factors that affect usage such as ratings of importance of system features; awareness of features; time spent by readers in identifying, locating, obtaining, and printing out or photocopying articles; purposes of reading; the age of articles read; and time spent reading the articles.
INPUT, OUTPUT, AND PERFORMANCE OF ACQUIRING LIBRARY ELECTRONIC AND PRINT COLLECTIONS
The first analysis involves the input and output of acquiring library collections, which in turn becomes one of several resources applied to provide journal access services. The input cost of the library print collection resource obviously involves much more than the price paid for the journals. The input to this resource includes staff, equipment, systems, and facilities for collection-related activities such as collection development, serials acquisition, mail processing, serials check in, and collection maintenance. These activities, of course, constitute the components of fixed costs of various collection access services. Some of these activities are common to electronic and print collections, but require a different level of effort. Below we provide some cost-benefit comparisons of the collection resource inputs (e.g., costs of staff, space, equipment, subscription price paid, etc.), outputs (e.g., number of titles acquired in the library collection, comprehensiveness, age, etc.), and performance (e.g., cost per title).
We have performed in-depth cost finding for these activities as they relate to print collections in special libraries (Griffiths & King, 1993) and print and electronic collection at the Drexel University, Hagerty Library (Montgomery & King, 2002). From 1998 to 2002, the Drexel Library migrated from a print collection of 1,710 to 370 rifles and from an electronic collection of 200 to 8,634 unique titles. Drexel is not retaining the print counterparts of electronic titles unless the electronic version is not a satisfactory equivalent (e.g., low-quality graphics). The increased comprehensiveness of the electronic collections is a clear "benefit" for them. In comparing the electronic collection resource with the previous print collections, the following differences in the price paid have been observed:
* Arguably, we could say that subscription prices paid "on the whole" are less for the Drexel electronic collection (i.e., perhaps a "benefit" for electronic collections). However, this simple statement ignores a complex set of factors that must be considered to make a comparison between electronic and print subscription prices paid meaningful. A "subscription" in the electronic world does not involve a simple payment for the annual content of a journal title. An annual subscription often brings with it several years' of back files. And the price models and electronic content vary so radically that Drexel has found it necessary to define three electronic journal types by the criterion: How stable is content? "Pure" Electronic Journals are individual subscriptions or publisher's packages that may or may not be a part of a consortium "deal" (e.g., acquired by purchase through a subscription agent or from the American Chemical Society, the American Institute of Physics, etc.). Aggregator Journals come from vendors that provide access to different publishers' journals with no possibility of content dropping, only adding. The collections started as full-text content and added searching (e.g., JSTOR, MUSE). Full-Text Database Journals come from many different publishers but with no title or issue-level subject or index access (except ProQuest). Journals are added or removed regularly from these databases according to the database vendor's contracts with publishers. They often have an embargo on current issues of six months or so (e.g., Lexis/Nexis, INFOTRAC's Expanded Academic).
Subscription prices vary greatly among the three types: at Drexel, at the beginning of 2002, the average per-title price paid varied from over $300 for the individually subscribed titles, about $90 for the aggregator titles, and $5 per title for the full-text database journals. Most academic libraries do not include the full-text database titles in the journal counts. However, use of the titles in these databases is so high (more about that later) that we feel it is misleading to exclude them from the total picture.
Price comparisons between electronic and print subscriptions will vary from library to library, depending upon the collection choices, agreements with publishers, consortia arrangements, ability to negotiate, choices regarding cataloging and inventory control and, surprisingly, size of library. Two common subscription models favor smaller academic libraries in a "rich get poorer and poor get richer" scenario: (1) access to a publisher's entire electronic collection for a premium over the money spent on the publisher's print subscriptions at the time of the "deal," and (2) access to all of a publisher's electronic journals held by any member of a consortium for a small premium over the money spent on the publisher's print subscriptions at the time of the "deal."
Regardless, libraries are compensating publishers largely for (1) adding value to article information content and (2) distribution through the print version or access to the electronic version. The cost to publishers for adding value to the article information content is about the same for both versions. The publisher's cost to distribute the print version is typically about $30 to $40 per annual subscription and the publisher cost to provide access to the electronic version depends on value-added features provided (e.g., the AAS features mentioned earlier). However, if both versions (i.e., electronic and print) are made available to the library by subscription, the publisher should be compensated only once for adding value to the article information content. Thus, subscribing to both versions should not cost the library much more than the price paid for one version.
In addition to the price paid for library collections, there are input costs associated with collection development; input processing and shelving for print collections; servers and systems for electronic collections; inventory control for both; and subscription maintenance for both. Comparisons of these costs are as follows:
* Collection development costs are generally higher for electronic collections because of the cost of personnel needed for the time-consuming process of negotiating licenses, the additional variables to be considered (e.g., interface, inclusion of visuals, perceived stability of source, commitment to archiving, existence of back files, linking from electronic databases) and the variety of sources from which a journal can be purchased. This is a "cost" for electronic collections compared with the print collections.
* Mail processing, serials check in, and shelving are nil for electronic collections, but even including electronic collection server and systems costs, there is a savings of about $70 per title (i.e., a "benefit" for electronic collections).
* Cataloging or inventory control costs depend on the library's policy. There are two basic approaches with many variations: (1) libraries may catalog all three categories of titles--which is very time-consuming to maintain; (2) catalog only the electronic titles in the first two categories listed above--which gives an incomplete picture; (3) catalog only the electronic equivalents of print titles--also incomplete; (4) maintain HTML lists (created from databases in the more technologically advanced libraries)--far less costly than cataloging to create and maintain but does not provide "one stop shopping" for journal holdings; (5) catalog the rifles and provide access via lists--obviously more costly than cataloging only. Thus, the "cost" or "benefit" of electronic journals depends upon the approach taken.
* Subscription maintenance can be higher for electronic collections due to the volatility of the collection (i.e., a "cost" for electronic collections).
The electronic collection at Drexel is more comprehensive than the previous print collection, but the annual per-title costs appear to be much less than equivalent print subscriptions. Thus, on balance these collection-input costs appear to be a substantial "benefit" in comparing electronic with print collections. And since labor costs are higher during the transition, the benefit should increase over time unless subscription prices increase dramatically. Another attribute of electronic journals, the access to back files as part of current subscriptions, makes the economic picture for electronic journals even better at the time of purchase.
INPUT, OUTPUT, AND PERFORMANCE OF LIBRARY ELECTRONIC AND PRINT COLLECTION SERVICES
Library print collection services include access to a current periodicals room, access to older journals maintained in stacks, articles provided through interlibrary loan (ILL), as well as journal routing provided by many special libraries. Articles are also obtained from elsewhere through interlibrary borrowing (ILB), document delivery, and pay-per-view. Library electronic access services involve access to internal or external full-text databases from in-library computer workstations and readers' office desktop computers by means of the campus network and, most importantly at Drexel, from their homes and elsewhere through the public Internet.
Input of library collection access services includes the cost of resources used to provide individual services in addition to allocation of the collection-related resources discussed in the previous section. Outputs of the access services are the quantities of services provided (i.e., hits or downloads and items reshelved) and the service attributes such as timeliness, availability, and accessibility.
The input costs of print collection services include (1) allocation of the fixed costs to each service and (2) the variable costs associated with the service access to the collection (i.e., the costs associated with each use). As shown in the previous section, the fixed costs of print collection services are largely due to the subscription price and processing of journal issues. To that is added the cost of the shelves and space allocated to the current periodicals room, or to older journals in the stack. The variable costs include the costs of activities associated with specific service use. Variable cost includes directional reference to print journals, photocopying of items read, and reshelving issues or bound volumes. ILL activities are ILL processing, photocopying, and reshelving. Journal routing requires such activities as list maintenance, routing processing, and reshelving. Typically photocopying by library staff costs about $2.70 per article (and $1.10 when coin machines are used) and reshelving about $.30 per item reshelved. These costs are estimated with all resources (i.e., staff, equipment, space, supplies, etc.) and overhead included. The total variable cost of the services, of course, depends on the amount of access or use.
There are both fixed and variable input costs associated with electronic collection services as well. There are two kinds of fixed costs. The first kind includes the collection-related resources and allocation of computers, servers, systems, space, and so on. The second kind of fixed cost is more related to the number of users involved. These costs include resources used to train users and to provide promotional and education materials for users. The variable costs associated with electronic access services include directional reference and help to users in the library, as well as support and help services provided to network users. Also, most reading from electronic library-provided journals is done from articles that are printed out. In fact, based on our recent readership surveys, about 80 percent of articles read are printed out. The cost to the library of printing an article obtained from a library computer is typically about $1.00 per article printed including allocation of printer costs (i.e., equipment, maintenance, toner, paper, space, furniture, etc.).
The service output quantities are usually measured by the number of times a service is used. Unfortunately, there are several definitions of use of library collections (King & Tenopir, 1999). For example, access to the periodical room collection and stacks is sometimes measured by counting issues and bound volumes reshelved (i.e., counted by observation or bar code). These measures are not the same as the amount of reading since an issue or bound volume might not be read at all or have many articles in them read. In fact, from exit survey observations, reshelved bound volumes tend to have fewer than two articles read per volume and reshelved issues average about four articles read. The Drexel survey data also indicate that about 25 percent of print journal users regularly use more than one article from a specific volume during a single use.
We have also observed, by survey, the annual number of times users say they have used these two print collections. In academic libraries, it is thirty-five and thirty-one uses per user per year of current periodicals and volumes in the stacks, respectively, and twenty-eight and twenty-five uses in special libraries (Tenopir & King, 2000). At Drexel, annual output metrics are: 15,000 issues reshelved and 8,800 bound volumes reshelved. Output measures also include attributes of the services such as availability and accessibility of the current periodicals room and the stacks. Hours of opening and the distance of the library to readers, of course, limit use of the print collections. ILL attributes include speed of response, quality of photocopying, and fee (if charged). ILB has similar attributes. The most critical attribute of journal routing is the number of persons on a routing list, since this attribute determines to a large degree when the reader will receive an issue.
An example of print service performance is the unit cost per item reshelved. In special libraries, after allocating the print collection (fixed) costs, we estimate the average or unit cost per use as being $13.00 per current periodical issue reshelved, $15.30 per bound volume reshelved, $25.70 per person receiving a routed title, and $24.30 per article obtained through ILB. At Drexel, the recent unit costs of access from the periodicals room is $8.50 per issue reshelved. The unit cost of the stacks tend to be much higher because the amount of use is down due to electronic access, and the size of the bound volume collection is larger than typical special library collections. The access unit cost per volume reshelved is $30.00.
Generally, uses per print title in libraries have increased, largely because of reduced personal subscriptions. Over a twenty-five-year period we estimate this to be about three times more use per title in academic libraries and seven times more in special libraries (Tenopir & King, 2001). This means that the unit costs per use are less than they would be if this phenomenon had not occurred. Of course, the increased prices of print journals have partially offset the increased use, when considering unit costs.
Output quantities of electronic collection services have also been difficult to measure (e.g., Luther, 2000; Shim, et al., 2001; Odlyzko, 2002; Miller & Schmidt, 2002; Goodman, 2002; Kidd, 2002). The metric of service output is currently reported to libraries by publishers, vendors, or consortia providing the electronic journals. These metrics are uses of full text such as hits or downloads of articles, which vary in definition among these sources. Even so, these indicators of reading are probably closer to the amount of reading than counting items reshelved. At Drexel, a full-text use is defined as an HTML view, PDF download, or print when possible. With an estimated 400,000 electronic uses, Drexel's estimates of per-use costs are $4 for individual subscriptions, $3 for publisher packages, $2 for aggregator titles, and about $1 for full-text titles. With broader collections available to smaller institutions, ILB will likely decrease and pay-per-view article access is sometimes (not always) available at less than the cost of ILB. All of these reductions in cost are, in effect, "benefits" for electronic collection services compared with print collection services.
USAGE AND EFFECTIVENESS OF LIBRARY ELECTRONIC AND PRINT COLLECTION SERVICES
Usage is measured by the extent to which articles in the library electronic and print collections are actually read. It is useful to make a distinction between type of output metrics of use of library collections mentioned above and metrics of the use of information content provided by the collection access services. We have done that through over 20,000 readership survey responses involving professionals, particularly scientists located in universities, national laboratories, industry, and government. Some of the readership surveys were performed for publishers (e.g., Science).
In recent years (2000 to 2002) we have surveyed readers in four distinct circumstances. Two surveys were performed at sites in which libraries have continued print collections, supplemented with electronic journals. One site is at the University of Tennessee (UT) where scientists and medical staff were surveyed, and the other involved users of a special library at Oak Ridge National Laboratory (ORNL) where scientists were surveyed. These two sites had been surveyed before electronic journals were on the scene, thus providing before and after observations. Another readership survey was done in May 2002 at Drexel University where most journals were replaced by electronic versions in 1999. Even though all faculty and doctoral students were surveyed, we limit observations here to scientists in order to provide direct comparisons with readership patterns of other scientists discussed here. At the end of 2001, we initiated and conducted a large-scale readership study of the American Astronomical Society (AAS)journal system (under partial funding by NASA). This study, in particular, provided substantial evidence of the readership of both library electronic and print collections. Furthermore, the AAS electronic publishing services are particularly advanced.
All of these readership surveys rely heavily on the critical incident method, where the last reading of an article is the focus of observation. A series of questions concerning the last reading include age of the article read; depth of reading; how the article is identified; where it was obtained (highlighting print vs. electronic sources); the amount of time spent by readers identifying, locating, obtaining, and reading the article; purposes of reading; outcomes from reading; and so on. This method is particularly useful in cross-classifying these observations. Comparison of the information seeking and reading patterns from electronic and print collections is given below.
The frequency with which electronic journals are used varies substantially among the surveyed groups of scientists, partially reflecting access to library electronic collections. For example, at UT and ORNL the proportions of readings from electronic sources are 23 and 32 percent respectively, but 46 percent at Drexel where the current collection is largely electronic.
Because of the early start and sophistication of AAS electronic publishing, the AAS members have come to rely much more on electronic journals than many other user groups. For example, 75 percent of readings by AAS members are from electronic sources. However, only 35 percent of the AAS member electronic journal reading comes from electronic library collections because of the availability of AAS electronic journals to society members and to value-added features.
The estimated annual amount of reading and proportion of reading from library print and electronic collections are given in Table 1. As might be expected, the proportion and amount of reading from the electronic library collection is by far the highest for the Drexel scientists. This electronic access may also account for the fact that less reading at Drexel is done from nonlibrary sources such as personal subscriptions. While we do not have before and after comparisons, it appears that the switch to the electronic collection has, if anything, increased readings from the Drexel library.
AAS members tend to rely on library electronic collections rather than library print collections, regardless of how the article is identified. For example, if an article is found by searching an online abstracting and indexing (A&I) database (e.g., ADS, PubScience, SPIN) or a Web search engine (e.g., Yahoo, AltaVista, Excite, Google), it will be obtained about 90 percent of the time from a library electronic collection rather than a print collection. At Drexel about 76 percent is from the library electronic collection. When an article is identified through browsing, about 70 percent of the articles will be from an electronic collection, but lower at Drexel (29 percent). Clearly, a library electronic collection is often the source of choice for these scientists. This is not necessarily true for UT/ORNL readers. Online searches (mostly from A&I databases) provide articles that are more often obtained from their library print collections (64 percent of these readings), largely because the older articles, identified by online search, are not yet available electronically. On the other hand, about two-thirds of articles found by browsing are from their library electronic collection as opposed to the print collection.
As mentioned earlier, effectiveness is the relationship between access service outputs (and their attributes) and usage measures. In a real sense, the collection medium (i.e., electronic and print) is an attribute of the collection-related services. Special attributes of the library collection from AAS are the age of articles in electronic medium, forward and backward linkages, preprint access, machine readable data tables, links to the NASA Astrophysics Data System (ADS), and inclusion of images and color. Below we discuss the comparison of the two types of library collections and their services with respect to information-seeking patterns and age of article read.
A survey of Drexel users in May of 2001, after a substantial electronic journal collection had been in place for two years, showed that they prefer electronic journals for many reasons. Four hundred student and faculty respondents responded (on a scale of 1 = no agreement; 10 = strong agreement) to indicators of satisfaction as follows:
Mean * E-journals save time 7.7 * E-journals make work easier 8.6 * E-journals result in better quality research 8.1 * E-journals enable me to find more 8.5
Eighty-four percent of the respondents preferred electronic journals to print; and use of electronic journals at Drexel is increasing, a pattern also reported by Guthrie (2000).
Astronomers were also asked to rate the importance of specific attributes or features of their journals. The average importance of these features in order of results are: machine-readable data tables (4.1 average importance rating), links to references (3.9), links to data centers (3.9), links to future citations (3.7), and inclusion of movies and color (3.1). While we do not know the relationship between these features and extent of reading electronic library collections, there well may be a positive correlation.
Time spent by readers (or someone on their behalf) varies substantially depending on how the articles are identified, located, and obtained. When articles have been identified, it takes an average of about nine minutes less time to locate and obtain the articles from the library electronic collections than from library print collections, and time spent browsing a library electronic collection is about eight minutes less per article found. It appears to require about three minutes less to download and print electronic articles than to photocopy print articles. As mentioned, the proportion of readings printed or photocopied is remarkably similar for the two media. While these differences may appear minor, they can add up to an appreciable amount of time with as much reading as scientists do. For example, AAS member use of library electronic collections alone (fifty-nine readings) can save them an average of about ten hours per year. Surveys over the years clearly show that scientists and medical professionals are aware of their time spent and they tend to choose information services and products based on ease of use and minimizing their time. Drexel's survey data also show that users believe that electronic journal use saves time. These results all point to "benefits" of the library electronic collections.
Since the Drexel Library has JSTOR and other older electronic collections, we observed that 69 percent of readings from articles published more than two years prior to reading were from the electronic library collection. Guthrie's preliminary usage data from all libraries using JSTOR indicates surprising use of older articles (Guthrie, 2000). He states that the "average age of the top ten articles most frequently printed and viewed was 13 years. More dramatically, in the file of mathematics, the average age of the most used articles was 32 years." The JSTOR data also show much heavier use of the electronic than print journals. This is confirmed by the Drexel experience. Use of the JSTOR journals is far heavier than the comparable bound volumes (even when adjusted for number of volumes held in the Drexel collection) and, in spite of the two-to-five-year "moving wall," it is even heavier than the combined current issue and bound volume use.
The survey of AAS members provided an opportunity to gain a glimpse of information-seeking and reading patterns for older materials since their electronic journals go back to 1849. The age distribution of articles read from astronomy core electronic journals is almost exactly that observed by scientists generally in recent years and even in 1960 (Tenopir & King, 2000). Again, we examined the readings by AAS members of articles published over two years prior to the surveys. Of articles obtained from library collections, 23 percent were over two years old. Most of these older articles were obtained from a library electronic collection (70 percent of readings). Most of these older articles were identified through citations in a refereed journal (35 percent of readings from a library electronic collection) or online search (58 percent). While four percent of readings from this electronic service were found by browsing, 18 percent of readings were found this way from library print collections. Nearly half of older readings from print collections were identified from citations and 27 percent from online searches.
The average amount of time spent by AAS readers (or someone on their behalf) obtaining the older electronic articles was the same as with newer articles. However, the time spent obtaining older print articles was somewhat greater than with the newer ones, thus yielding an additional or greater "benefit" of the library electronic collection. Also of interest is that the average time spent reading older articles is forty minutes per article, compared with twenty-five minutes for recent articles, as might be expected given the purpose of use.
One indicator of print collection effectiveness is the proximity of the collection to readers (i.e., its accessibility). Every survey we have done comparing distance (in minutes) of readers to the print collection shows the overall use of the library, use of its journal collection, and amount of reading are inversely correlated with the distance to the library. That is, those closer have higher use, although it is found that readers further away from the collection tend to read more when they do visit the library. Evidence of the effect of distance on reading is as follows:
* 66 percent of the readings are from library print collections when the readers are less than five minutes away;
* 48 percent of readings are from there when five-to-ten-minutes away; and
* 34 percent of readings are from there when over ten minutes away.
A study by Charles River Associates (1978) for the NSF used a stochastic model to determine the probability that scientists will subscribe to a journal. The two most important factors, of many factors contributing to a low probability of subscribing, were found to be: "availability of the journal in a library frequently used by the scientist" and "convenience of location of the library to the scientist." This also accounts for the fact that scientists closer to their library tend to subscribe to fewer journals than those further away.
Thus, one of the clear "benefits" of library electronic collection services is that distance does not affect use of the collection and the extent of reading from the collection. Another is that the electronic collection is available at all hours. Finally, scientists further away from their library tend to subscribe to more journals (incurring additional cost), which would not be necessary with the electronic collection services. Drexel results seem to confirm this since scientists there average fewer personal subscriptions than scientists at UT.
COST-EFFECTIVENESS OF LIBRARY ELECTRONIC AND PRINT COLLECTION SERVICES
In an earlier section we discussed the unit cost of several journal access services. The unit costs were based on the fixed costs of purchasing and processing the journal collections and the variable costs associated with provision of the collection services. The average (or unit) costs were based on use measured by issues and bound volumes reshelved for print collections and hits and downloads for electronic collections. Cost effectiveness, by our definition, is a relationship between service input measures and usage measures. Perhaps the most obvious such derived measure is the input cost of services divided by readership resulting from the services. In our special libraries studies the estimated cost-effectiveness measures are:
* Reading from print current periodicals--S4.20 per reading
* Reading from print collections in the stacks--S9.70 per reading
* Reading from routed journals--S4.80 per reading
Note that these unit costs are much less than cost per use of these services. In the discussion of effectiveness of print collections, we mentioned the effect distance has on amount of reading. Of course, special libraries tried to increase reading from their collections by routing journals to their users. This, as shown, has been relatively cost-effective.
Unfortunately, we do not yet have direct comparisons with electronic library collections in special libraries. At Drexel the cost-effectiveness is $3.90 per reading for print current periodicals; $23.50 per reading for print journals in the stacks; and $2.00 per reading electronic journals. Thus, cost-effectiveness is a significant "benefit" for electronic journal collection services.
OUTCOMES FROM READING ARTICLES FROM LIBRARY ELECTRONIC AND PRINT COLLECTION SERVICES
We consider outcomes as the consequences of having read and used information found in articles obtained from library collections. Since we began surveying readership in the 1970s, we have tried to assess such outcomes from several perspectives, particularly considering the purposes for which reading is done (i.e., research, teaching, learning, etc.). Examples of outcomes include:
* the importance of the information in achieving these purposes;
* the relative importance of the information, as a resource for performing work, compared with other resources used in doing that work;
* the amount of dollar savings achieved from reading;
* the extent to which reading affects readers' performance such as the quality and timeliness of work, improvement in students' grades, and so on;
* a correlation between amount of reading and productivity;
* other favorable consequences such as initiating new ideas, broadening options in work, and so on;
* achievements of parent organization and societal goals.
Most of these outcome indicators have been observed in special library environments (Griffiths & King, 1993), although below some of them are compared from readings of library electronic and print collections based on the UT, ORNL, Drexel, and AAS surveys.
The purposes for which information is used depends somewhat on the scientists' work setting, field, and type of work. Scientists in universities indicate that about one-half of readings are for current awareness or professional development. When applied to work, they are used to support research (75 percent of readings), teaching (41 percent), and administration (13 percent). Over a period of one year the scientists indicated that twenty-three of the readings from print collections were absolutely essential to their research and thirteen readings were absolutely essential to teaching. Almost identical results were observed from the Drexel survey of scientists when reading from the electronic library collection. For example, 79 percent were read for research and 25 percent of these were absolutely essential to this purpose. Nonuniversity scientists indicated that 30 percent of readings were for current awareness, etc., conducting primary research (17 percent), background research (26 percent), design or other R&D activities (11 percent), administration (19 percent), writing (7 percent), and presentations (7 percent). When compared with other resources (e.g., computing, instrumentation, support staff, etc.), the information found in documents was rated second highest in importance for primary research and rated highest for most other tasks.
Comparing the principal purposes for information read from library electronic and print collections, the purposes given by AAS members are very similar for the two collections: primary research (44 percent of readings from electronic collections vs. 48 percent print); background research (33 percent, 28 percent respectively); writing proposals, reports, articles, etc. (10 percent, 15 percent). Importance of the information to the principal purpose is rated from 1--not at all important to 3--absolutely essential. Average importance ratings are almost the same (2.21 for readings from an electronic collection; 2.19 print) and the proportion rated absolutely essential is the same for both collections (22 percent of readings).
A separate opinion survey of AAS members illustrates just how valuable they believe the electronic journals to be for their work, both for keeping up with current developments as well as for obtaining definitive information. For example, 72 percent of them rate electronic journals as either "very useful" or "essential" for keeping up with recent developments. When seeking definitive information, astronomers value the electronic journals even more highly. Virtually all astronomers (96 percent) rate electronic journals as either "essential" or "very useful" for delivering definitive results.
This overwhelming approval rate reflects the effectiveness of the whole electronic information system used in astronomy, particularly seamless links between the electronic journals and the highly effective NASA ADS (an A&I service plus a database of historical full-text journal articles). The same survey indicated that 97 percent of AAS members knew about the ADS, over 50 percent of them use it at least every other day, and 27 percent of AAS members use the ADS every day. ADS usage statistics confirm this level of activity.
We also asked readers if they had previously read the most recently read article and, prior to the first reading, if they knew about the information reported or discussed. The results are similar for proportion of articles that had been previously read (20 percent of readings of electronic, 24 percent print), but less for prior knowledge of information in electronic collections (42 percent electronic vs. 50 percent print). Thus, there may be some "benefit" in provision of more new information read from library electronic collections. One explanation for this phenomenon may be that scientists are observed to read from a broader range of journals than they did twenty-five years ago. That is, in 1977 scientists on average were estimated to read at least one article from about thirteen journals, but that number has increased to over twenty based on observations in the last two years. This broadened reading may be partially due to the easy accessibility of electronic journals and to the discovery tools--the abstracting and indexing databases. As a result, more readings from journals not previously read may provide additional new information.
A series of earlier readership surveys (Griffiths & King, 1993) showed that journal reading resulted in saving time and money. In fact, about 32 percent of readings from library-provided articles resulted in such savings compared with 23 percent read from other resources (e.g., personal subscriptions). Furthermore, the estimated amount of savings was $360 per reading and $260 respectively. Reasons given for achieving such savings included avoiding having to do some primary research (49 percent of readings in which savings were incurred), provided confirmation of research (27 percent of such readings), stopped an unproductive line of research (10 percent of such readings), modified research or engineering design (12 percent of such readings), and modified analysis methods (16 percent of such readings). Clearly, one should not interpret the amount of these savings as being "typical" from readings since the average savings are calculated from highly skewed distributions with 25 percent of readings contributing no savings and one or two percent accounting for most of the estimated savings.
In other surveys, we asked how journal reading affected the quality and timeliness of work. For example, respondents were asked the activity for which reading was done, whether the reading affected the quality of the activity and, if so, they were asked to rate the quality of the activity before and after reading with I being low and 7 high quality. About 70 percent of readings were for activities in which quality is applicable; 44 percent of all readings resulted in improved quality; and, following readings, the ratings of quality improved from 4.04 average rating to 5.82. About 60 percent of the readings involved activities for which timeliness is relevant and 32 percent of these readings resulted in faster performance or completion of the activity.
In one company, a stated goal was to increase the speed of products from discovery to the marketplace. We identified about twenty major processes involved in going from discovery to the marketplace and asked whether reading affected the speed of completion for each of these processes. About 31 percent of readings of library-provided documents led to the work being completed faster.
We developed five indicators of productivity of professionals in organizations where outputs included number of formal records of work, number of consultations given, number of presentations made, number of written proposals or plans, and number of formal publications written. In all instances the productivity measures were positively correlated with the amount of reading. Higher productivity and improved work performance would suggest the potential of receiving formal recognition of work through achievement or technical awards and other forms of recognition. Our surveys revealed that recipients of such awards tend to read more articles than nonrecipients. For example, such award winners read 32 percent more articles in a year. Persons asked to serve on high-level projects or problem-solving teams or special committees read 21 percent more articles. In one company, the personnel office provided us with the names of twenty-five particularly high achievers. These twenty-five high achievers read 59 percent more articles than cohorts with equivalent degrees, fields of specialty, and years of experience and who performed the same kind of work. This finding holds true for both university and nonuniversity scientists.
Thus, there are several ways of assessing the outcomes of reading journal articles. In our recent surveys we do not have all of these indicators for library electronic collections. However, we observed that a high proportion of readings from library electronic and print collections improved the result of the principal purpose for which reading was done (36 percent of electronic reading, 41 percent print); inspired new thinking or ideas (36 percent electronic, 33 percent print); narrowed, broadened, or changed the focus (14 percent electronic, 19 percent print); resolved technical problems (9 percent electronic, 7 percent print). Some said that the reading had no effect on the principal purpose, but was valuable nevertheless (17 percent electronic, 26 percent print). It appears, considering outcomes alone, that there is no clear "benefit" or "cost" attributable to electronic collection services compared with print collection services, but both have highly favorable outcomes. On the other hand, readings from library-provided articles almost always have more favorable outcomes (e.g., King & Montgomery, 2002).
SUMMARY AND CONCLUSION
We have provided an approach to assessing the economics of electronic journals in libraries and a description of how this approach developed historically. The approach involves a framework of input, output, performance, usage, effectiveness, cost-effectiveness, and outcome metrics of library services. In this approach, cost-benefit is a comparison of a service and some alternative to it using these metrics. In this article we use, as an example of these metrics, the library electronic collection services compared with print collection services. Such comparisons are considered a "benefit," if favorable, and a "cost" or "detriment," if unfavorable. Some indication of the benefits and costs of electronic over print collections are shown in Table 2.
Examined from the library perspective, it appears that the electronic collection and services will yield benefits in requiring lower prices per-title, less time of staff, and, potentially, substantial savings in space. Thus, these resources can be reallocated into additional or better services to users. Users benefit in flexibility of access; saving substantial time in searching, locating, and obtaining the articles; availability of new and useful features; and broadening the number of journals they use. Library and scientist funders benefit from better utilization of their resources (i.e., library and scientists). Thus, it appears that library electronic collections are highly beneficial to publishers, as well as libraries, readers, and their organizations (whether universities or elsewhere). Despite some turmoil, the scholarly journal system seems likely to continue its important role in research, teaching, and lifelong learning. Electronic journals will continue to grow in acceptance and strength, although some libraries may continue to purchase both electronic and print versions at minimal additional costs in order to provide current periodicals in print for readers who prefer this version.
Table 1. Proportion and Amount of Reading from Library Electronic and Print Collections by UT Scientists and Medical Staff, ORNL Scientists, Drexel University Scientists, and AAS Members: 2000 to 2002. Electronic Library Total Article Collection Reading Survey Readings per Respondents Person per Year (%) No. UT Scientists 201 9.0 17 UT Medical Staff 322 12.7 41 ORNL Scientists 113 21.3 24 Drexel Scientists 214 38.0 81 AAS Members 226 25.9 59 Print Library Collection Reading Survey Respondents (%) No. UT Scientists 23.0 46 UT Medical Staff 7.6 24 ORNL Scientists 26.7 30 Drexel Scientists 8.5 18 AAS Members 5.8 13 Source: UT, ORNL, Drexel, and AAS surveys (n = 1,474) Table 2. Comparison of Electronic vs Print Economic Metrics Benefits Collection as a Resource Input Negotiation and system Collection development Purchase price Depends Processing Lower cost ($) Inventory control Depends Subscription maintenance Output Quantities More titles Output Attributes Accessibility Proximity Availability 24-hour access Features (e.g., AAS) Many features possible Performance Lower cost/title ($) Collection Services Current Journal (Input) Collection fixed cost (allocated) Lower cost ($) Training, publicity (allocated) Variable cost Current Journal (Output) Some more use Current Journal (Performance) Lower cost/use ($) Older Journal (Input) Collection fixed cost (allocated) Lower cost ($) Training, publicity (allocated) Variable cost Older Journal (Output) Similar use Older Journal (Performance) Much lower cost/use ($) External (e.g., Office) Access (Input) Collection fixed cost (allocated) Lower cost ($) Training, publicity (allocated) Variable cost Much lower cost/use ($) External Access (Output) More use External Access (Performance) Much lower cost/use ($) Usage Reading Some more readings Purpose of Reading Similar purposes User Time Save user time User Effort Less effort needed Cost-effectiveness Lower cost/reading ($) Outcomes Importance of Information Content Similar rating Provided New Information More new information Other Outcomes Both high Comparison of Electronic vs Print Economic Metrics Costs/Detriments Collection as a Resource Input Negotiation and system High costs ($) Collection development Higher cost ($) Purchase price Depends Processing Inventory control Depends Subscription maintenance Higher cost ($) Output Quantities Output Attributes Accessibility Availability Features (e.g., AAS) Performance Collection Services Current Journal (Input) Collection fixed cost (allocated) Training, publicity (allocated) Moderate cost ($) Variable cost Slightly higher cost ($) Current Journal (Output) Current Journal (Performance) Older Journal (Input) Collection fixed cost (allocated) Training, publicity (allocated) Moderate cost ($) Variable cost Slightly higher cost ($) Older Journal (Output) Similar use Older Journal (Performance) External (e.g., Office) Access (Input) Collection fixed cost (allocated) Training, publicity (allocated) Moderate cost ($) Variable cost External Access (Output) External Access (Performance) Usage Reading Purpose of Reading Similar purposes User Time User Effort Cost-effectiveness Outcomes Importance of Information Content Similar rating Provided New Information Other Outcomes Both high
The authors would like to acknowledge the contributions to this article by the following persons: Sarah Aerni, Janine Golden, Matt Herbison, Willie J. Johnson Jr., and David Robins, University of Pittsburgh, School of Information Sciences; Katie Brady and Thomas McLaughlin, Drexel University, Hagerty Library; Randy Hoffman, Oak Ridge National Laboratory; and Rhyn Davies, Matt Grayson, Sarah Greene, and Keri-Lynn Paulson, University of Tennessee, School of Information Sciences.
Baumol, W. J., & Marcus, M. (1973). Economics of academic libraries. Washington, DC: American Council on Education.
Bickner, R. E. (1971). Concepts of economic costs. In King, D. W., Roderer, N. K., & Olsen, H. A. (Eds.), Key papers in the economics of information, 1983 (pp. 107-46). New York: Knowledge Industry Publications.
Buckland, M. K. (1983). Library services in theory and context. New York: Pergamon Press.
Charles River Associates (1978). Development of a model of the demand for scientific and technical information services. Cambridge, MA: Charles River Associates.
Clark, P. M. (1976). A study to refine and test new measures of library service and train library personnel in their use. Rutgers University: Bureau of Library and Information Science Research.
Cooper, M. D. (1979). The economics of library size: A preliminary inquiry. Library Trends 28 (Summer): 63-78.
Cummings, M. M. (1986). The economics of research libraries. Washington, DC: Council on Library Resources.
D'Elia, G., & Walsh S. (1983). User satisfaction with library service--a measure of public library performance? Library Quarterly, 53(2), 109-33.
deProspo, E. R. et al. (1973). Performance measures for public libraries. Chicago, IL: American Library Association.
Goodman, D. (2002). A year without print at Princeton, and what we plan next. Learned Publishing 15(1), 43-50.
Griffiths, J-M. (1977). The Computer Simulation of Information Retrieval Systems. Doctoral Dissertation, University College London.
Griffiths, J-M. (1998). Why the Web is not a library. In Hawkins, B. L., & Battin, P. (Eds.), The mirage of continuity: Reconfiguring academic information resources for the twenty-first century (pp. 229-246). Washington, DC: Council on Library and Information Resources and the Association of American Libraries.
Griffiths, J-M. (2002). Performance measurement in libraries. International Encyclopedia of Information and Library Science, 2nd ed. (printing in progress).
Griffiths, J-M., & King, D. W. (1986). A study of library staffing. Final report. Prepared for the California State University, Office of the Chancellor. King Research, Inc.
Griffiths, J-M., & King, D. W. (1989). Keys to success. Performance Indicators for Public Libraries. British Library, Office of Arts and Libraries. [Available from Unipub, 4611 F Assembly Drive, Lanham, MD, 20706].
Griffiths, J-M., & King, D. W. (1991). Massachusetts libraries: An alliance for the future: Final report. Prepared for the Massachusetts Board of Library Commissioners. Boston, MA: King Research, Inc.
Griffiths, J-M., & King, D. W. (1993). Special libraries: Increasing the information edge. Washington, DC: Special Libraries Association.
Guthrie, K. M. (2000). Revitalizing older published literature: Preliminary lessons from the use of JSTOR. Paper presented at the PEAK Conference. (March). Retrieved April 4, 2001, from http://www.jstor.org/about/preliminarylessons.html.
Hamburg, M. et al. (1974). Library planning and decision-making systems. Cambridge, MA: MIT Press.
Kantor, P. B. (1984). Objective performance measures for academic and research libraries. Washington, DC: Association of Research Libraries.
Kidd, T. (2002). Electronic journal usage statistics in practice. Serials, 15(1), 11-17.
King, D. W., & Bryant, E. C. (1971). The evaluation of information services and products. Washington, DC: Information Resources Press.
King, D. W., & Lancaster, E W. (1969). Costs, performance, and benefits of information systems. In Proceeedings of the American Society for Information Science, 6, 501-505.
King D. W., & Montgomery, C. H. (2002). After migration to an electronic journal collection: Impact on faculty and doctoral students. D-lib Magazine, 8(12). Retrieved from http:// www.dlib.org/dlib/december02/12king/12king.html.
King, D. W., & Palmour, V. E. (1974). User behavior. In Fenichel, C. (Ed.), Changing patterns in information retrieval (Proceedings of the 19th Annual National Information Retrieval Colloquium). Philadelphia: American Society for Information Science.
King, D. W., & Tenopir, C. (1999). Using and reading scholarly literature. In Williams, M. E. (Ed.), Annual Review of Information Science and Technology: Vol. 34. Medford, NJ: Information Today, Inc.
King, D. W., & Xu, H. (2003). Library consortia & electronic journal services. In Pearce, A., The consortium license: Is it a sustainable model? Oxford: Ingenta Institute.
King, D. W., Neel, P. N., & Wood, B. L. (1972). Comparative evaluation of retrieval effectiveness of descriptor and free-text search systems using CIRCOL (Central Information Reference and Control Online). Prepared for Rome Air Development Center, MD: Westat, Inc. AD-738299.
King, D. W. et al. (1976). Statistical indicators of scientific and technical information communication (1960-1980). Volume I: A summary report. National Science Foundation C-878. King Research Inc., Center for Quantitative Sciences. GPO 083-000-00295-3.
King, D. W. et al. (1981). Scientific journals in the United States: Their production, use, and economics. Stroudsburg, PA: Hutchinson-Ross Publishing Co.
Lancaster, E W. (1977). The measurement and evaluation of library services. Washington, DC: Information Resources Press.
Lancaster, F. W. (1993). If you want to evaluate your library ... 2nd ed. Champaign, IL: University of Illinois Graduate School of Library and Information Science.
Luther, J. (2000) White Paper on Electronic Journal Usage Statistics. Washington, DC: Council on Library and Information Resources.
Machlup, F. (1979). Uses, value, and benefits of knowledge. In Knowledge: Creation, diffusion, and utilization. Beverly Hills: Sage Publications.
McClure, C. R., & Lopata, C. L. (1996). Assessing the academic networked environment: Strategies and options. Washington, DC: Coalition for Networked Information.
McClure, C. R. et al. (1987). Planning & role setting for public libraries. Prepared for the Public Library Development Project. Chicago, IL: American Library Association.
McDonald, D. D. et al. (1981). Cost and usage study of the Educational Resources Information Center (ERIC) System. Submitted to National Institute of Education (ED 208902). Rockville, MD: King Research.
Miller, R., & Schmidt, S. (2002). E-metrics: Measures for electronic resources. Serials, 15(1), 19-25.
Montgomery, C. H. & King, D. W. (2002). Comparing library and user related costs of print and electronic journal collections: A first step towards a comprehensive analysis. D-lib Magazine, 8(10). Reprinted in New States Review of Information Networking 6 (2000). Retrieved from http://www.dlib.org/dlib/october02/montgomery/10montgomery.html.
Morse, P. M. (1968). Library effectiveness: A systems approach. Cambridge, MA: MIT Press.
Odlyzko, A. (2002). The rapid evolution of scholarly communication. Learned Publishing15(1). 7-19.
Palmour, V. E. et al. (1980a). A planning process for public libraries. Prepared for the Public Library Association of the American Library Association.
Palmour, V. E. et al. (1980b). Public library services. In Washnis, G. J. (Ed.), Productivity improvement handbook for state and local government (pp. 1446-64). New York: John Wiley & Sons.
Shim, W. et al (2001). Measures and statistics for research library networked services: Procedures and issues. ARL E-Metrics Phase II Report. Washington, DC: Association of Research Libraries.
Tenopir, C., & King, D. W. (2000). Towards electronic journals: Realities for scientists, librarians, and publishers. Washington, DC: Special Libraries Association.
Tenopir, C., & King, D. W. (2001). Lessons for the future journals. Nature, 413(October), 672674.
Van House, N. A. et al. (1987). Output measures for public libraries. Prepared for the Public Library Development Project. Chicago, IL: American Library Association.
Zweizig, D., & Rodger, E. J. (1982). Output measures for public libraries: A manual of standardized procedures. Prepared for the Public Library Association of the American Library Association.
DONALD W. KING is the Founder of King Research, an organization that pioneered studies of library performance and costs, library users, and publication processes. Currently, he is Research Professor at the University of Pittsburgh School of Information Sciences. King is author of nearly 100 monographs, a frequent contributor to the journal literature, and speaks regularly to professional audiences. He coauthored with Carol Tenopir Towards Electronic Journals: Realities for Scientists, Librarians, and Publishers (Special Libraries Association, 2000). He is a fellow of the American Statistical Association and the recipient of both the Research Award and the Award of Merit from the American Society for Information Science and Technology.
PETER B. BOYCE spent nineteen years with the American Astronomical Society (AAS), sixteen of them as Executive Officer. While at the AAS, he developed and led the Society's program in electronic publishing, and is now doing consulting on electronic publishing for the AAS and others. He has worked at Lowell Observatory, The National Science Foundation, and the U.S. Congress. He is a Fellow of the American Association for the Advancement of Science and was awarded the PAM Division Award of the Special Libraries Association for his pioneering leadership in electronic publishing.
CAROL HANSEN MONTGOMERY is Dean of Libraries at Drexel University and has been active in the information field for over twenty-five years as an administrator, teacher, researcher, and author. A frequent speaker and contributor to the professional literature, she is the author or coauthor of five books. She has held local and national positions in numerous professional organizations and recently received a grant from the Institute for Museum and Library Services to study the economic and organizational impact of a library's transition to an electronic journal collection.
CAROL TENOPIR is Professor at the School of Information Sciences, University of Tennessee, Knoxville. Tenopir is the author of four books including Towards Electronic Journals: Realities for Scientists, Librarians, and Publishers with Donald W. King (Special Libraries Association, 2000), over 200 journal articles, and is a frequent speaker at professional conferences. Since 1983, she has written the "Online Databases" column for Library Journal. She is the recipient of the 1993 Outstanding Information Science Teacher Award from the American Society for Information Science/Institute for Scientific Information, the 2000 ALISE Award for Teaching Excellence, and the 2002 Research Award from ASIST.
Donald W. King, Research Professor, University of Pittsburgh, School of Information Sciences, 135 N. Bellefield Ave., 600, Pittsburgh, PA 15260
Peter B. Boyce, Past Executive Officer, American Astronomical Society, 33 York St., Nantucket, MA 02554
Carol Hansen Montgomery, Dean of Libraries, Drexel University, 33rd and Market Sts., Philadelphia, PA 19104
Carol Tenopir, Professor, The School of Information Sciences, University of Tennessee, 1345 Circle Park Drive, Room 451, Knoxville, TN 37996-0341
|Printer friendly Cite/link Email Feedback|
|Date:||Jan 1, 2003|
|Previous Article:||Scholarly materials: paper or digital? (Academic Libraries).|
|Next Article:||Cost, statistics, measures, and standards for digital reference services: a preliminary view. (Academic Libraries).|