Printer Friendly

A journal cancellation survey and resulting impact on interlibrary loan.

The University of New Mexico (UNM) Health Sciences Center consists of the faculty, staff, and students of the School of Medicine and its health professions programs; the College of Nursing; the College of Pharmacy; and the UNM Hospital system. In 2014, the Health Sciences Library and Informatics Center (HSLIC) at the UNM experienced a shortfall in the collection budget of approximately $250,000. This shortfall was mainly due to inflation in e-journals and other e-resource subscriptions. Resource management librarians undertook a review of all collection resources to determine what could be canceled in order to stay within budget.

Research has shown that the most important factors used to make serials cancellation decisions are faculty consultations and data, such as vendor usage statistics and cost data [1]. A standard method for gathering faculty input is to compile lists of titles being considered for cancellation in each subject area and to send the lists to faculty in the relevant departments for input [2-7]. Other libraries have used a survey approach to solicit input from faculty and other stakeholders in a more quantitative manner [8-11]. Increase in interlibrary loan (ILL) requests has been relied upon in the past to determine the effect of journal cancellations [12-14], but few studies have been published since the 1990s [15, 16].

This paper describes the approach that HSLIC took to make journal cancellation decisions and to assess the impact cancellations had on ILL.


Journals categorization

To begin the evaluation process, the authors gathered cost and usage data for all of HSLIC's individually subscribed e-journal titles (n = 117) and titles in packages that allowed individual title cancellations (n = 416) from the calendar year 2013, as well as usage from January 2014 until mid-August 2014. We calculated cost per use for all journals and sorted the list for all journals that exceeded $10 cost per use. We then sorted this list into subsets consisting of those journals exceeding $20 cost per use (n = 5) and those with less than 100 total uses (n = 5) and cancelled these titles. For the remaining 36 journals with cost per use greater than $10 and less than $20, we solicited user feedback by distributing an online survey to the greater health sciences center population.


The anonymous survey asked participants to rate the usefulness of a total of thirty-six journals being considered for cancellation (Appendix, online only). To get broad feedback, we distributed the survey to all students, faculty, staff, and house staff at the UNM Health Sciences Center during September 2014. The survey was distributed using faculty and resident email lists, listing in campus announcements, and posting in the library newsletter and on the library website. Respondents had to login with their university IDs and passwords to access the survey, and responding more than once was restricted based on Internet protocol (IP) address, but it was otherwise open to anyone with the university uniform resource locator.

Respondents were asked to list their primary academic or clinical affiliation as well as their status at the university. We sorted responses from each status into 3 response groups: faculty, residents, and staff and students. Respondents were provided a list including journal titles, costs, and number of uses for 2013, as well as major subject areas in a tabular format to facilitate browsing the list. Respondents were asked to choose from one of five options for each journal title: "Never Use," "Rarely Use," "Useful," "Important," or "Essential." The default option was "Never Use," so respondents did not have to enter a response for every title. Finally, respondents could enter free text comments up to 250 characters before submitting the survey. This study received exempt status from our institution's institutional review board prior to distribution of the survey.

At the end of calendar year 2015, we collected ILL requests and direct individual requests for purchase of the cancelled journal titles for the past calendar year. We also gathered turnaway reports (COUNTER JR2), which compile failed attempts to access full-text journal content that is not licensed by the institution [17].


Survey and analysis

The survey received 354 complete responses, of which 246 were from faculty (70%), 58 from residents (16%), and 49 from both staff and students (14%), whom we grouped together due to relatively low response rates from each group. At least 1 faculty member from all colleges and academic departments was represented among respondents, except for 1 department that did not have any relevant journals in the survey.

We converted the textual responses to numerical data for ease of analysis (e.g., "Essential" = 5, "Never Used" = 1) and cleaned affiliation values as some data were entered incorrectly or in the wrong area. We tallied responses by response group and averaged them for each journal title. We counted the total number of "Essential," "Important," and "Useful" responses for each title and averaged these counts to determine a minimum number of responses that each journal had to receive to gain a "keep" score. The minimum criteria differed for each response group.

For example, the average "Essential" response from faculty across all journals was 2.56, so we set the minimum criteria for each journal as 3 for this response group. We followed the same process for "Important" and "Useful." Journals that did not meet the minimum criteria for at least 2 of these categories were assigned a "Cancel" score for that response group. We repeated this process across all response groups and then compared the results of all the groups.

Journals with a "Cancel" score from 2 or more response groups were marked for cancellation. Figure 1 provides an example of this process. Due to survey feedback, we cancelled 8 journals for a total savings of approximately $27,000.

Requests for cancelled material

During 2015, we received a total of 43 ILL requests for articles from the 18 titles that we cancelled. Only 15 of these requests were for articles published during 2015. Only 2 of the titles were requested greater than 5 times, and only 1 title surpassed the CONTU Rule of 5 [18]. Reviewing our cancelled titles, the over $20 cost-per-use group had 12 requests, the under 100 uses group had 14 requests, while the survey group had a total of 17 requests.

Turnaway reports were available for all but one of the cancelled titles. The number of turnaways for the cancelled titles ranged from zero to seventy-six. Because turnaway reports do not indicate the publication year of the content that the user tried to access, we could not be certain that all of these turnaways resulted from attempts to access content from 2015. It was possible that the turnaways could have come from attempts to access articles in a journal's backfile. We did not find any correlation after comparing the number of turnaways with the number of ILL requests. Overall, the turnaway reports did not provide much insight into the success or failure of our cancellation decisions.

To date, librarians have not received any direct requests for subscriptions to the cancelled titles, either through liaison librarians or through the library's Request a Purchase web form.


The ILL request rate for cancelled titles indicated our methodology was likely successful in making cancellation decisions, as none of the titles received a significant number of ILL requests. The request rate and resulting copyright fees were financially negligible.

While past studies have relied on ILL statistics to determine the effectiveness of journal cancellations, as have we, they might not accurately capture the need for the cancelled titles. Current research in this area indicates that crowdsourcing scholarly material from various online sources and social media is preferable to ILL, due to quick turnaround [19]. This might explain why the entire group of cancelled titles only received a total of fifteen requests for material published during 2015, a figure much lower than reported usage for the prior year. While there is no way to accurately measure or predict the need for cancelled journals, ILL data can approximate this need and inform librarian selectors about whether it is more cost effective to subscribe to journals or to borrow them.

In January 2015, our library migrated to a new integrated library system and discovery layer. One of the features of the new system was the ability to enable an ILL request button within catalog records that auto-populates the ILL request form. Previously, users had to navigate to our ILL system and manually type or copy and paste article information into the ILL form. As a result, our ILL request rate rose 137% over the previous calendar year. The overall increase in ILL demand might account for some of the ILL requests for cancelled titles and demonstrated that our users were more aware of our ILL service. However, the 43 ILL requests for articles from the cancelled journals only constituted 1.4% of the total ILL requests for the year, so the cancellations did not place a burden on our ILL service.

Use of survey responses in decision making

After reviewing the responses to the survey, we decided that setting strict, quantitative criteria for retention or cancellation would not have led to the best results for a number of reasons, among which were the unbalanced response rates from colleges and departments and a propensity to primarily use journals within one's own area of study. In fact, in three cases, we made the decision to keep or cancel a journal despite user feedback. One of these cases was the Journal of Interprofessional Care, a title directly related to the nascent interprofessional education program at UNM. In this way, rather than using the survey as an absolute method of cancellation, we instead used it to help inform our decisions to retain or cancel specific titles, while still seeking to balance the many wants and needs of all the areas that we serve.

Free-text responses

The free-text response portion of the survey provided some insight as to why certain individuals found certain journals essential to their work. In other cases, the response provided an interesting look at misconceptions that many in our community have about the library and the need to cancel journal titles, as well as general misconceptions about the journal publishing marketplace and how much academic journals can cost.

Several respondents entered comments indicating that they were fine with the library cancelling titles as long as they could get necessary articles through ILL. Respondents also noted the importance of ILL remaining a free service.

Other comments suggest that some respondents did not fully understand the purpose of the journal survey. One respondent was concerned that the journals essential to the respondent's work were not represented in the survey, not understanding that the absence of those journals meant they were not being considered for cancellation. Another noted that it was fine to cancel the print version of the title as long as we maintained electronic access, even though our library had cancelled the vast majority of our print subscriptions more than ten years ago.

There were also comments suggesting a need for training on how to access library materials. One respondent said the library did not have any of the journals the respondent needed from a particular publisher, when in fact the library had access to almost all of that publisher's journals through a "Big Deal" bundle. Another respondent requested two journal subscriptions, both of which the library already purchased.


We were unable to calculate a true response rate because we could not accurately determine the number of potential respondents. Because recruitment was primarily conducted using email lists and completion of the survey was voluntary, the results might be biased toward individuals who had an interest in the library's journal subscriptions. Some library liaisons more actively promoted the survey in their departments than others, and some department chairs encouraged all their faculty members to take the survey; therefore, some departments might have been better represented in the survey than others. We attempted to minimize the effect this had on the analysis by comparing the respondent's departmental affiliation with the subject area of the journal. However, there were limitations looking at department affiliation given the interdisciplinary nature of the health sciences and crossover between departments. For example, a pediatric endocrinologist who works in the Pediatrics Department would be interested in journals in the subject area of pediatrics and endocrinology. Our survey demographics were not this granular, and we did not have this type of information about respondents unless they chose to enter it as a free-text response.



[1.] Williamson J, Fernandez P, Dixon L. Factors in science journal cancellation projects: the roles of faculty consultations and data. Issues Sci Technol Librariansh [Internet]. 2014;54(3-4):78. [cited 24 Nov 2015]. <>.

[2.] Clement S, Gillespie G, Tusa S, Blake J. Collaboration and organization for successful serials cancellation. Ser Libr [Internet]. 2008;54(3-4):229-34. <>.

[3.] Calvert K, Fleming R. Giving them what they want: providing information for a serials review project. Proceedings of the Charleson Library Conference; 2011. p. 393-6.

[4.] Sinha R, Tucker C, Scherlen A. Finding the delicate balance: serials assessment at the University of Nevada, Las Vegas. Ser Rev. 2005;31(2):120-4.

[5.] Carey R, Elfstrand S, Hijleh R. An evidence-based approach for gaining faculty acceptance in a serials cancellation project. Collect Build. 2006;30(2):59-72.

[6.] Enoch T, Harker KR. Planning for the budget-ocalypse: the evolution of a serials/ER cancellation methodology. Ser Libr. 2015;68(1-4):282-9.

[7.] Nixon JM. A reprise, or round three: using a database management program as a decision-support system for the cancellation of serials. Ser Libr. 2010;59(3-4):302-12.

[8.] Bucknell T, Stanley T. Design and implementation of a periodicals voting exercise at Leeds University Library. Serials. 2002;15(2):153-9.

[9.] Bloom M, Marks L. Growing opportunities in the hospital library: measuring the collection needs of hospital clinicians. J Hosp Librariansh [Internet]. 2013 Apr;13(2):113-9. [cited 25 Nov 2015]. <>.

[10.] Murphy A. An evidence-based approach to engaging healthcare users in a journal review project. Insights UKSG J. 2012;25(1):44-50.

[11.] Day A, Davis H. Collection intelligence: using data driven decision making in collection management. Proceedings of the Charleston Library Conference; 2010. p. 81-97.

[12.] Crump MJ, Freund L. Serials cancellations and interlibrary loan: the link and what it reveals. Ser Rev. 1995;21(2):29.

[13.] Wilson MD, Alexander W. Automated interlibrary loan/document delivery data applications for serials collection development. Ser Rev. 1999;25(4):11-9. DOI:

[14.] Kilpatrick TL, Preece BG. Serial cuts and interlibrary loan: filling the gaps. Interlend Doc Supply [Internet]. 1996;24(1):12-20. <>.

[15.] Calvert K, Fleming R, Hill K. Impact of journal cancellations on interlibrary loan demand. Ser Rev [Internet]. 2013 Sep;39(3):184-7 [cited 21 Jun 2016]. <>.

[16.] Calvert K, Gee W, Malliet J, Fleming R. Is ILL enough? examining ILL demand after journal cancellations at three North Carolina universities. Proceedings of the Charleston Library Conference [Internet]. 2013 [cited 21 Jun 2016]. p. 416-9. < >.

[17.] COUNTER Online Metrics. The COUNTER code of practice for e-resources: release 4 [Internet]. 2012 [cited 23 Feb 2013]. <>.

[18.] Copyright Clearance Center. The campus guide to copyright compliance [Internet]. The Center; 2005 [cited 23 Mar 2016]. <>.

[19.] Gardner CC, Gardner GJ. Fast and furious (at publishers): the motivations behind crowdsourced research sharing. Coll Res Libr [Internet]. 2017 [cited 21 Jun 2016]. <>. Forthcoming.

(EC) A supplemental appendix is available with the online version of this journal.

Jacob L. Nash, MSLIS, AHIP; Karen R. McElfresh, MSLS, AHIP


Jacob L. Nash, MSLIS, AHIP,, Resource Management Librarian; Karen R. McElfresh, MSLS, AHIP,, Resource Management Librarian; Health Sciences Library & Informatics Center, 1 University of New Mexico, MSC09 5100, Albuquerque, NM 87131

Received March 2016; May 2016



HSL1C Journal Survey

Health Sciences Library and Informatics Center Journal Survey

Mr, Nasn, MSLIS from the Health Sciences Library and Informatics Center, is conducting a research study. The purpose of the study is to assess the HSC research, education, and clinical population's use of and need for various high cost subscription journals. You are being asked to participate in this study because you are a member of this population who reads the journals within the Health Sciences Literature.

Your participation will involve taking a brief survey. The survey should take about 5 minutes to complete. Your involvement in the study is voluntary, and you may choose not to participate. There are no names or identifying information associated with this survey. The survey includes questions such as what department you are from and what journals you find essential for your work. You can refuse to answer any of the questions at any time. There are no known risks in this study, but some individuals may experience discomfort when answering questions. All data will be kept for I year in a locked file In Mr Nash's office and then destroyed.

The findings from this project will provide Information on which high cost serials can be discontinued with little impact on the HSC research, education, and clinical population. If published, results will be presented in summary Form only.

If you have any questions about this research project, please feel free to call Mr. Nash at (505)272-9895.

If you have questions regarding your legal rights as a research subject, you may call the UNMHSC Office of Human Research Protections at (505} 272-1129.

By continuing on with this survey, you are consenting to participate in this research.

Health Sciences Library and Informatics Center Journal Collections Survey

It should take you less than 5 minutes to complete our survey.

The Health Sciences Library and Informatics Center needs your feedback on journal subscriptions to renew or cancel.

Due to the rising prices of subscription journals and a budget shortfall, the library will have to cancel subscriptions to some of these journals.

Your feedback is very important in making these decisions.

Thank you for completing this brief survey.


HSLIC Journal Survey

Professional Information

1. Please indicate your primary academic or clinical affiliation from the dropdown menu.

If your department or college Is NOT listed, please add it under "other" at the end of the list.

College of Nursing

College of Pharmacy

Anesthesiology & Critical Care Medicine

Biochemistry & Molecular Biology

Cell Biology and Physiology

Dental Medicine


Emergency Medicine

Family and Community Medicine

HSC Administration

Internal Medicine

Medical Group

Molecular Genetics & Microbiology




Obstetrics & Gynecology

Orthopedics & Rehabilitation



2. Please indicate your UNM status.

* Faculty

* Staff

* Resident

* Student


HSLIC Journal Survey


3. Journal subscriptions HSLIC is considering for cancellation are listed below.

Please indicate which of these 37 journals are essential to your work, important to your work, and which are useful, butnotessential to your work. For those you rarely use, select Rarely used.

For those resource you do NOT use. please leave the default option of Never Used

A space is provided below for you to tell us more about your choices.
Publication Title       2013      2013   Major Subject      Choice
                        Cost      Uses   Area

Advances in Neonatal    $1,193    304    Perinatology       Never used
American Journal of     $2,347    195    Public Health      Never used
Drug and Alcohol
Biochemical Journal     $4,340    325    Biochemistry       Never used
Bioessays               $2,043    187    Molecular          Never used
British Journal of      $4,123    323    Pharmacology       Never used
Cancer Causes and       $2,368    200    Oncology           Never used
Community Mental        $1,069    86     Psychiatry         Never used
Health Journal
Currenl Opinion in      $2,692    324    Nutritional        Never used
Clinical Nutrition &                     Sciences
Metabolic Care
Current Opinion in      $2,813    343    Neurology          Never used
Current Opinion In      $2,571    315    Psychiatry         Never used
European Journal of     $3,649    96     Pharmacology       Never used
Clinical Pharmacology
European Journal of     $4,346    218    Pediatrics         Never used
Experimental Brain      $10,643   184    Neurology          Never used
Head & Neck             $2,192    175    Surgery            Never used
Issues in Mental        $1,349    108    Nursing            Never used
Health Nursing
Journal of Behavioral   $1,196    61     Behavior           Never used
Journal of              $1.493    346    Cardiology         Never used
Journal or Clinical     $3,197    402    Gastroenterology   Never used
Journal of Clinical     $1,159    412    Pharmacology       Never used

Journal of              $2,022    174    Endocrinology      Never used
Journal of              $2,279    155    Health Services    Never used
Journal of Magnetic     $2,559    223    Radiology          Never used
Journal of Neurology    S3,547    311    Neurology          Never used
Journal of Pathology    $2,738    157    Pathology          Never used
Journal of Physiology   $6,485    325    Physiology         Never used

Journal of Visualized   $11.000   676    Lab Protocols      Never used
Experiments (JoVE)
Molecular and           $7,603    148    Biochemistry       Never used
Neurochemical           $2,530    155    Neurology          Never used
Neuroreport             $1,228    311    Neurology          Never used
Pediatric Pulmonology   $3,015    215    Pediatrics         Never used
Physics in Medicine     $3,762    198    Radiology          Never used
& Biology
Psychopharmacology      $7,051    427    Pharmacology       Never used
Research In Nursing     $1,937    146    Nursing            Never used
and Health
Southern Medical        $4,594    508    Medicine           Never used
Thyroid                 $1,761    339    Endocrinology      Never used
Transplantation         $1,807    187    Surgery            Never used

Please tell us more about yourchoices or provide additional comments.

Figure 1
Journal scoring workflow example

Response group    Number of   Score    Overall     Decision
                  responses            score for   for title
Faculty                                group

Essential > 3     1           Cancel
Important > 7     4           Cancel   Cancel
Useful > 10       14          Keep
Essential > 0.9   1           Keep                 Cancel
Important > 1     1           Cancel   Keep
Useful > 2        3           Keep
Essential > 0.9   0           Cancel
Important > 2     1           Cancel   Cancel
Useful > 4        1           Cancel
COPYRIGHT 2016 Medical Library Association
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2016 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Nash, Jacob L.; McElfresh, Karen R.
Publication:Journal of the Medical Library Association
Article Type:Report
Date:Oct 1, 2016
Previous Article:Evaluation of resources for analyzing drug interactions.
Next Article:Manual search approaches used by systematic reviewers in dermatology.

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters