Spilling Out of the Funnel: How Reliance Upon Interlibrary Loan Affects Access to Information

Steven A. Knowlton, Iulia Kristanciuk, and Matthew J. Jabaily

Steven A. Knowlton (steven.knowlton@memphis.edu) is Collection Development Librarian, University Libraries, University of Memphis. Iulia Kristanciuk (krstncuk@memphis.edu) is Electronic Resources Library Assistant, University Libraries, University of Memphis. Matthew Jabaily (jabailym@gmail.com) is Information Services Librarian at Rhodes College.

Submitted May 20, 2014; returned to authors July 13, 2014 for minor revision; revision submitted August 9, 2014; accepted for publication August 21, 2014.

The authors are grateful to Susan Wood for providing interlibrary loan usage figures, to Cody Behles, Laureen Cantwell, and Pam Dennis for their helpful critiques of our methods and drafts, and to Kenneth Ward for his review of our statistics.

Academic libraries that cancel serials titles typically offer interlibrary loan (ILL) as an alternative means to access these titles.This study examines how serials cancellations affect ILL usage and how reliance on ILL affects patrons’ access to content. By analyzing the number of ILL requests from canceled titles, the authors found that cancellations have a very small effect upon overall ILL usage. With the help of Google Analytics, the authors counted patron requests for link resolver access that were converted to ILL requests. When the link resolver was unable to generate a link to full text, it displayed a message to that effect on a link resolver landing page and presented the patron with a choice to request the title through ILL. Google Analytics recorded traffic to and from the link resolver landing page and generated a data set for this study. Analysis of collected data, including ILLiad records, shows that after patrons identify desired articles that require ILL, they only submit ILL requests 31 percent of the time. This means that for every successful ILL request, there are at least two articles desired that are never requested. Implications for collection development are discussed.

When academic libraries cancel journal subscriptions, patrons lose immediate access to the content within those journals.However, patrons whose libraries participate in interlibrary loan (ILL) programs retain the ability to discover the existence of desired content through indexes (often called “databases”) and obtain the material from partner libraries. Many librarians are confident that this model of obtaining material is an effective alternative to subscriptions because, as Mortimore writes, a well-run ILL program can “provide access to the right materials at the right time.”1

The practice of canceling subscriptions and relying on ILL presents two related but distinct concerns for library administrators, concerns that this paper addresses. First, savings from subscriptions budgets may be countered by increased ILL costs. Second, it is not established whether ILL meets patron information needs as well as direct subscriptions do.Accessing content via ILL is a very different experience from accessing it directly from a library portal.A patron seeking content than a subscribed title typically finds the material in an index and can begin reading it within seconds.A patron who uses ILL must undertake additional steps in the library interface to place the ILL request and wait hours or days to read the material. Waiting for the requested material may present a potential problem in the context of the increasing number of millennials among library patrons, who are described in library literature as impatient, “not tolerant of delays, expectant of instant service [and] instant gratification.”2 Regarding the first concern, whether turning from subscriptions to ILL will affect library costs in ILL, Beales suggests that such data can be influential in a library’s decision whether to abandon “Big Deal” subscription packages.3 In particular, the question addressed in this paper is whether increased ILL costs will consume savings from canceled subscriptions.

The second concern addressed in this paper is how reliance upon ILL affects patron access to content. The question addressed in this paper is if patrons who find citations in an index will use content offered through ILL as much as they use directly subscribed content. Even if reliance upon ILL reduces the amount of direct access to content, the interpretation of the significance of that effect varies. Nabe and Fowler, researchers who studied ILL requests after cancellations, offer one view. They found that ILL requests for the cancelled titles were significantly lower than the number of full-text downloads. They concluded that “download statistics are not an accurate indicator of demand.”4 They assert that ease of access via efficient article linking within library portals and on web search engines such as Google artificially inflates usage figures as a user may access an article “without meaning to, or after accessing it, determine it to be of no use.”5

An alternative to Nabe and Fowler’s conclusion may be that, rather than ease of access artificially inflating demand, the inconvenience of using ILL artificially depresses demand. For example, at the University of Memphis (UoM) Libraries, making an ILL request requires the user to take at least two extra steps beyond accessing an article within a subscribed journal. After identifying the desired content in an index and clicking on an “Article Linker” icon to open an article to which the library subscribes, the patron must identify and click on a link labelled “ILLiad,” which opens the ILL software interface, and then choose to submit a request within ILLiad.This chore of navigating through the funnel of web pages to access a desired article may deter users from completing an ILL request.

To address the related questions of how cancellations affect ILL usage and how relying on ILL affects patrons’ access to content, the authors conducted a study of user behavior related to ILL. To assess how serials cancellations affect ILL requests, the authors duplicated an experiment originally conducted by Calvert and Fleming to test if their results could be reproduced.6 Calvert and Fleming noticed a spike in ILL use and in response to that change in user behavior investigated whether the increase in ILL use was related to recent serials cancellations. They found no correlation between the cancellations and the increased use of ILL. In addition, the authors conducted a novel study using web analytics to examine to what extent the link resolver landing page deters patrons from completing an ILL request.

Background

The University of Memphis (UoM) is a publicly-supported research university whose library budget has not kept pace with increases in serials prices. In 2012, the UoM libraries reviewed subscriptions and identified 277 titles suitable for cancellation because of low usage, low citation rates, or both. Starting in January 2013, patrons who had previously enjoyed direct access to those titles would need to use ILL to access the content (except in cases where the material was available by a means other than direct subscription, such as an aggregated database.)

At UoM, making an ILL request from an online index is not a seamless process. During the period studied, a patron, upon discovering an article of interest, was required to click on a cryptic icon named “Article Linker” (see figure 1).The patron then navigated a jargon-laden page that offered the availability of ILL in small print hidden among other links (see figure 2). From there, the patron had to register for an ILL account (if not already registered) and finally place the request.To add to the complexity of making a request, if patrons had not previously registered for an ILL account, the citation data were not transferred into ILLiad.This series of steps is known as a “funnel.”7

UoM uses ProQuest’s Serials Solutions 360 Link (www.proquest.com/products-services/360-Link.html) to provide its link resolver service. Link resolver technology controls the linking between the website where the patron discovers an item of interest and the website where the full text resides. Figures 1 and 2 show the default display settings with minor customization. For a researcher who has not been shown the process of authorizing a link resolver request and then converting an unsuccessful link resolver request into an ILL request, these displays may be confusing. (Note: it is the authors’ intention to improve the public display of these functions; however, for the duration of this study patrons used the displays shown above.)

As a result of the cancellations of 277 subscriptions, there were 187 titles that patrons could access in 2012 but lacked new content in 2013. The remaining 90 cancelled titles had continued coverage through aggregated databases. The authors used this title list as one of the datasets for our studies.Patrons desiring to access the 2013 content of the cancelled titles had to rely upon ILL to acquire the material through UoM Libraries. The authors used ILL data to examine patron behavior around the ILL function to explore the impact of cancellations upon the libraries’ ILL service and upon the patrons’ access to content.

Literature Review

In the last two decades, literature about journal cancellations and ILL has frequently discussed the use of ILL data to evaluate prior cancellation decisions and to measure cost-effectiveness of borrowing versus owning serial titles.Although they vary in scope and approach, all studies draw on ILL transaction data obtained from document delivery software, and practically all find minimal impact of serials cancellations on overall ILL usage. Another shared feature of these studies is that they concentrate on studying cancellations of print serials. This differs from the current study and that of Calvert, Fleming, and Hill, which examine cancellation of online serials. However, because ILL is used to supplement access to journal content regardless of its format, the effects of cancellations are comparable between studies of print and online cancellations. There is also general agreement among the authors of the reviewed literature that ILL does represent a cost-effective option for providing access to materials from cancelled journals.Numerous authors, including Jaramillo and Lamborn, Nixon, Walter, Warner, and Welch, emphasize collaboration between libraries and faculty as an essential factor in making judicious cancellations.8 As Calvert, Fleming, and Hill point out, however, there is a dearth of recent literature on the subject of the impact of journal cancellations on ILL

In the 1990s, several articles followed up on library cancellation projects to compare the cost of filling ILL requests for articles from cancelled journals to the cost of maintaining subscriptions to those journals.Kilpatrick and Preece assessed the impact of a major cancellation project at their library four years after its implementation in 1990. The authors found that “articles from fewer than 5% of the cancelled serial titles were requested on interlibrary loan one or more times.”9 According to the authors, such low demand for articles from cancelled journals justified and supported the library’s cancellation decisions. Kilpatrick and Preece calculated that, during the six-month period of the study, the library spent less per article for ILL services than it would have for subscriptions. In a similar study, Wilson and Alexander examined ILL borrowing data from May 1995 to January 1999 and found that, with few exceptions, borrowing was more cost-effective than subscriptions. In most cases, a single year’s subscription cost as much or more than three years of borrowing for the same title. Transaction data showed that only 1.4 percent of articles from cancelled titles were requested five or more times.10

Hughes collected data about the cost of supplying three types of articles via commercial document delivery (CDD): articles from recently cancelled journals; articles from journals that were owned but were either missing, being bound, or in some other way unavailable to patrons; and articles from journals that were never owned or were cancelled very long ago. The number of requests for articles from recently cancelled journals was the lowest among the three types of requests studied in the pilot (8 percent), and for those articles, using CDD was considerably more cost-effective than subscribing. It cost the library $128.95 to get nine articles from previously cancelled journals, and the cost of subscriptions for the same journals would have been “at least $4,630” per year.11 Although requests for articles from owned but inaccessible journals and from journals that were never owned were more numerous, the costs associated with obtaining these articles through CDD were still considerably lower than subscriptions to the journals from which articles were requested.

In 1995 Crump and Freund singled out requests for cancelled titles and found that they constituted “just 0.2% of 16,632 interlibrary loan requests submitted by the University of Florida patrons during the research period.”12 A year later in 1996, Gossen and Kaczor compared journal title requests through ILL from academic scientists to two cancellation lists and found that patrons requested articles from only 1 percent of the titles canceled during the study period.13

In 1998, using data from an experimental pilot project focused on science and engineering journals, Duda and Meszaros found that the highest number of requests (over 40 percent) for articles from cancelled titles occurred during year one of the pilot (1991) and in 1997—the last year in the researched period. These figures are considerably higher than the number of requests for cancelled titles recorded in the studies conducted by other authors in the 1990s. The authors attribute the 1997 increase in the number of requests for cancelled literature to “the cumulative effect of the cancellation projects.”14 The authors demonstrate that article borrowing costs are substantially lower than subscription costs, and this conclusion aligns well with other studies reviewed in this section.

In 2011, Nabe and Fowler published two accounts of the impact on ILL of breaking Big Deal contracts. Nabe acknowledges that by leaving a Big Deal, his library incurred a significant loss in the overall number of titles; upon further examination, it turned out that a great number of these titles received low to zero use. A more reliable source of data for measuring the impact of leaving the Big Deal, ILL transactions showed that the impact of cancellations was minimal. A comparison of the top 25 percent of precancellation downloads to postcancellation ILL requests revealed that for Wiley titles, ILL demand was 0.9 percent of prior use, and for Elsevier titles it constituted 0.3 percent of prior use. Unlike other studies in this review, this examination of ILL data is vendor-based, and it will require independent verification before it can be compared to other studies. Nabe and Fowler’s account of the impact of downsizing from a Big Deal to a “medium” deal on ILL is cursory. Although his library experienced a 47 percent rise in ILL requests after breaking the Big Deal, Fowler believes it to be coincidental and attributes it to “the near-simultaneous implementation of WorldCat Local” at his institution.15

Calvert and Fleming conducted the most recent study of the impact of journal cancellations on interlibrary loan, which was published in 2013 by Hill. Having been alerted by the head of ILL to an 11 percent spike in requests between 2011 and 2012, the time when their journal cancellations took effect, Calvert and Fleming examined ILL transaction data from 2010 to 2012 to determine what factors occasioned the spike. They learned that about 4 percent of cancelled titles received ILL requests in 2012; requests for articles from cancelled journals made up about 2 percent of total ILL requests that year. The authors also noticed that the one cancelled journal that received the most requests had undergone a change in publisher coverage permissions after the cancellation, and that change made it unavailable through aggregators.16 According to Hill, Calvert and Fleming believe that this decision by the publisher to embargo certain titles accounts for the 11 percent spike in journal requests. Finally, Calvert and Fleming checked ILL data from 2010 and 2011 to determine how many requests were received for cancelled titles prior to being cut. Analysis of ILL data showed that “20 out of the 29 studied journals saw either their first use or an increase in use in 2012,” directly relating the 2 percent increase in ILL requests to the recent cancellation project. Hill summed up Calvert and Fleming’s study by stating that journal cancellations had minimal impact on the operations of their interlibrary loan department.

Although many papers have reported on the impact of cancellations on internal measurements of ILL, such as number of requests and cost to the library, to the authors’ knowledge there have been no studies that measure how many patrons are deterred from placing an ILL request by the additional steps required to complete the transaction. The authors’ method of using web analytics has seen some applications for studies of library user behavior.

Web analytics involves using tools that “collect, analyze, and report website traffic data.”17 These tools can be useful in tracking traffic patterns on a webpage to learn whence incoming traffic arrives and where outgoing traffic goes. In library applications, Turner recommends that librarians use web analytics to determine what users are looking for on a library webpage and optimize the page’s design.18 Numerous authors have reported on projects in which they used web analytics to understand patterns of website use and improve the visitor experience.19 The widespread use of web analytics in libraries prompted the publication of Marek’s monograph on the subject.20 Fagan posits a model by which web analytics can be used to assess progress toward a library’s strategic benchmarks.21

In library research, Taraghi et al. used web analytics to trace the patterns of user linking from article to article within the Open Journal System (OJS) database.They found that users have a recurrent pattern of navigation when searching for articles.22 Castro-Gessner, Wilcox, and Chandler of Cornell University used web analytics to trace the origins of visitors to their library’s LibGuide research assistance pages; they found that 70 percent of visitors were not affiliated with Cornell.23 The authors are unaware of any research using web analytics to measure user behavior regarding ILL requests.

To learn more about patterns in user behavior and navigation of serials online, libraries have analyzed link resolver data. Wakimoto, Walker, and Dabbour examined user experiences with the SFX link resolver; about half of their users were confused and closed the link resolver window without attempting to access full text.24 Chrzastowski, Norman, and Miller provide a helpful guide to generating reports using SFX.25 Stengel points out that these data tell librarians how users discover needed resources and also reveal the most-searched titles that, for a number of reasons, do not turn into ILL requests and thus are absent from ILL request logs.26

Other discussions in the literature about link resolvers and ILL venture beyond collection development to explore other areas of librarianship. Frank and Bothmann studied information-seeking behaviors of undergraduate students.27 In more systemic studies of the impact of adding an ILL option to the link resolver, Williams and Bailey found that implementation of Serials Solutions reduced ILL requests for materials provided by the library, while Munson and Otto found a correlation between link resolver clickthroughs and ILL requests.28 Stowers and Tucker described use of link resolver data in collection assessment processes, detailing a number of reports that they used in a comprehensive collection assessment.29

Method

This study was conducted in three parts.

Measuring ILL Requests for Cancelled Titles

Part one sought to duplicate Calvert and Fleming’s study to see how many ILL requests were made for articles from journals on the list of cancellations.A list of cancelled journals was created in a Microsoft Excel spreadsheet. Each title was checked to see if there was alternate access to full text available in the present year, and a second list was created that included only the journals for which full text for the current year was not available.The list included journal titles ISSNs and eISSNs, which were normalized by removing any dashes or spaces.

Next, OCLC’s ILLiad (www.oclc.org/illiad.en.html) was used to generate Excel-formatted reports of all ILL loan activity for the months of January through June 2013 (inclusive). These reports were combined into a single document and filtered to include only the requests that resulted in the delivery of an article to a patron. The list included the publication year of the article, journal title, volume, issue, author, article title, ISSN, and request date. The ISSN field was copied and normalized to remove any dashes or spaces; because the COUNTIF function used to analyze this dataset recognizes character strings, the same ISSN appearing with and without a space recognized as two different character strings.

The lists of cancelled journals and completed ILL requests were compared by looking for matches in ISSNs using the Excel COUNTIF function. ILL requests with no ISSN listed were manually checked against the cancellation list by journal title. Preliminary matches were double-checked to determine definitively whether the journal cancellation necessitated the ILL request. Results were discounted if the requested article was already available at the library in print but delivered anyway, for example for a distance user, or if the article was older than the range of volumes the publisher offered online.

Measuring Patron Interest in Articles that Did Not Result in ILL Requests from Cancelled Titles

In part two, the instances of patrons using link resolvers to attempt to find a full-text article but not submitting an ILL request were measured.The metric used was the number of access attempts via the link resolver to an article from a cancelled journal. Google Analytics code was added to the footer of the UoM Libraries Serials Solutions pages to count the number of hits to each page. Whenever a patron used the link resolver, Google Analytics recorded access to the Serials Solutions landing page. The recorded Uniform Resource Locator (URL) for link resolver requests uses OpenURL, so the URL included journal ISSN, date, and other key information about the request in a predictable format.

Link resolver request data from January to June 2013 were exported from Google Analytics as an Excel spreadsheet. The URLs were transferred to NotePad++, a source code editor, and the find-and-replace function and regular expressions were used to isolate and normalize the ISSNs from each request.30 The publication year of each request was similarly isolated. The link resolver request data were then transferred back into the Excel spreadsheet into the proper rows with the rest of the data. The list of ISSNs of the link resolver requests was compared to the list of cancelled journals. The number of hits for each journal was recorded. To determine how many requests were for current materials to which the UoM Libraries provide no access, the spreadsheet was filtered to include only the requests for articles published in 2013.

The method described above counts the number of unique URL requests for each journal, but many of the URLs for specific articles were accessed several times. Each of these accesses was counted by Google Analytics as an individual pageview.To count the number of total pageviews for articles from cancelled journals, the list of URLs and pageviews from Google Analytics was compared to the list of ISSNs of cancelled journals. The URLs that contained ISSNs from the cancellation list were isolated and the counts of pageviews were totaled. Only the number of pageviews for the entire set was recorded, as finding the total for each title would have been very time-intensive.

This study began in the second half of 2013 and examined data from the first six months of the year. In the data collection period, the authors avoided accessing any pages that were being monitored by Google Analytics as part of the study. During the study, however, the authors accessed these pages regularly to verify the URLs listed in the Google Analytics reports.Thus, expanding the time period of the study is not possible because the results would be artificially inflated by the authors’ own use.

Comparison Lists

The third part of the study examined two comparison groups.The first group consisted of a sample of thirty-five journals to which UoM Libraries had no access, online or in print. The titles were arbitrarily selected from the lists of ILL requests from previous years. This list of non-subscribed journals was subjected to the same analysis as the cancelled list; the number of ILL requests, unique Serials Solutions hits, and total number of Serials Solutions pageviews were tabulated. The purpose of analyzing this set of journals was to compare the number and ratio of ILL requests and Serials Solutions views to the cancelled set.

A second comparison group was formed using journals to which the UoM Libraries currently offers access online.This group involved forty-nine journals and was arbitrarily selected from the list of journals to which the library subscribes. Again, the number of ILL requests, unique Serials Solutions hits, and Serials Solutions pageviews were tabulated.For this group, an additional step was taken to record the number of clickthroughs recorded by Serials Solutions for the journals.The purpose was to compare the number of Serials Solutions hits and pageviews to the number of clickthroughs recorded by Serials Solutions. This would give some indication of how well the Google Analytics statistics from the Serials Solutions pages approximated the total potential use via Serials Solutions.

Results

ILL Requests for Cancelled Titles

In the first half of 2013, 3,845 ILL requests were placed.This is a decrease from the first half of 2012, when 5,336 requests were placed. The decrease may be due to the fact that in 2012 UoM signed on to two Big Deal journal packages and began subscribing to several titles that were expected to be heavily used. When cancelled titles were examined specifically, of the 187 titles cancelled only 6, or about 3 percent, received ILL requests in 2013 (see table 1).From the 187 titles cancelled, the library filled only eight ILL requests for articles from the 6 canceled titles, a figure which represents 0.2 percent of ILL requests (see table 2).

Patron Interest in Cancelled Titles that Did Not Result in ILL Requests

In the first half of 2013, there were forty-four instances of patrons following the link resolver from an index entry to the landing page of an article published in 2013 from a cancelled title; those forty-four instances referred to twenty-four unique articles.When extrapolated for the entire year, the figure would be eighty-eight instances of patrons following the link resolver from an index entry to the landing page. Yet, of those forty-four opportunities to make an ILL request for an article from a cancelled title, only six were converted to actual requests.Requests were counted from ILLiad logs, which measured requests submitted via the landing page’s link to ILLiad or by directly logging into ILLiad.Only 14 percent of sessions led to patrons completing the ILL request from the landing page; this may be called the “conversion rate.”For comparison, the conversion rate from a sample of titles to which UoM has never subscribed is 35 percent (see table 3).

As a control, the authors calculated the conversion rate for titles to which UoM subscribes.Ideally, there should be no ILL requests for subscribed titles. In fact, less than 1 percent of pageviews for currently subscribed titles are converted to ILL requests (see table 3). This result may be attributed to errors in our holdings data that accidentally denied patrons access to titles with active subscriptions.The conversion rate of less than 1 percent for subscribed titles confirms that using Google Analytics accurately tracks user access to library materials.

For the same list of subscribed titles, we compared the number of Google Analytics pageviews to the number of Serials Solutions click-throughs for each title. There were 698 pageviews recorded by Google Analytics and 921 Serials Solutions click-throughs for the set. This implies that our method underestimates the interest in a title; in this sample discovering only about 76 percent of use. This undercounting is likely because Serials Solutions can measure some use that our Google Analytics method cannot. For example, if a patron queries the Serials Solutions database by title and proceeds to the full text, the ISSN is not recorded as part of a URL by Google Analytics but Serials Solutions would count it as a click-through.

Conversion rate may be inverted to show a ratio of pageviews to successful ILL requests.For the cancelled titles, there are 6.4 pageviews for every completed ILL request. For the titles to which the library never subscribed, there are 1.9 pageviews for every completed ILL request. Overall, there are 2.3 pageviews for every completed ILL request (see table 4).

Although the data offered here is intriguing, it is true that the size of the sample is small and the time period studied was only six months. A more extensive study is likely to produce results that can be reported with a higher level of confidence, based on the commonly accepted principle that larger sample sizes are more likely to exhibit precision.

Discussion

The fact that ILL requests for articles from cancelled titles constituted only 0.2 percent of all ILL requests at UoM confirms Calvert and Fleming’s findings and other studies.Cancellation of titles based upon scrutiny of usage and other bibliometric measures will not produce an untenable increase in ILL activity. The cost savings of the cancellations will likely exceed any increased costs from ILL requests.

The notion advanced by many librarians that ILL is a patron-satisfying means of providing access to materials is open to question.Patrons who identify materials of interest via databases and are then directed to use ILL to access material overwhelmingly fail to complete the ILL request, either through following the landing page’s links to ILLiad or by directly logging in to ILLiad.

The authors speculate that there are three possible causes for this low conversion rate. First, patrons may have an immediate need for materials. Although the RAPID ILL service can fill requests within hours in many cases, patrons new to the service may not know that and may not complete an ILL request because of the perceived urgency of their research.Second, the awkward interface at UoM may be confusing enough that patrons are unable to identify the means by which to complete the ILL request. The chokepoint of the funnel for ILL requests may be a poorly-designed landing page.Or else the requirement to create an ILLiad username and password before submitting a request may be the point of deterrence.As yet, librarians at UoM have not conducted user studies which might further illuminate this matter. Third, patrons may be reluctant to “impose” upon library staff to make special requests.Again, user studies would help to better understand these matters.

Regardless of the reasons why patrons fail to complete ILL requests after identifying materials of interest, the low conversion rate is a matter of importance in collection development. The results of this study show that cancelled titles are viewed by patrons as similar to titles the library has never held.Cancellation of titles results in a much lower level of access for patrons.

Researchers who desire to explore this topic further may study the effects of ILL in place of subscriptions on patron access to information, and they may consider examining a larger set of titles over a longer period of time.Direct user studies may illuminate some of the obstacles to successful navigation of the index entry-into-ILL request funnel.

Conclusion

Based on these studies, the authors agree that carefully planned serials cancellations are unlikely to produce a large impact upon the level of ILL activity.However, it is probably not accurate to say that ILL is an acceptable substitute for journal subscriptions. For every ILL request, there are at least two articles for which a patron has expressed interest but has not accessed via ILL.

The barriers to access presented by ILL are substantial enough that a large majority of patrons do not convert their search to an ILL request.Libraries planning serials cancellations are advised to investigate whether patrons are comfortable making ILL requests and if the mechanisms for placing ILL requests are easily navigated and understood. Otherwise, many patron information-seeking sessions will terminate at the top of the funnel rather than working through the process to a completed ILL request.

References and Notes

1. Jeffrey M. Mortimore, “Access-Informed Collection Development and the Academic Library: Using Holdings, Circulation, and ILL Data to Develop Prescient Collections,” Collection Management 30, no. 3 (2005): 22.
2. Charles Becker, “Student Values and Research: Are Millennials Really Changing the Future of Reference and Research?” Journal of Library Administration 52, no. 6-7 (2012): 474, 497.
3. David Beales, “Modeling the Cost of Abandoning the Big Deal: A Case Study from the UK,” paper presented at the American Library Association Annual Conference, Anaheim, June 24, 2012.
4. Jonathon Nabe and David C. Fowler, “Leaving the ‘Big Deal’: Consequences and Next Steps,” Serials Librarian 62, no. 1-4 (2012): 61–62.
5. Ibid.
6. Kristen Calvert, Rachel Fleming, and Katherine Hill, “Impact of Journal Cancellations on Interlibrary Loan Demand,” Serials Review 39, no. 3 (2013): 184–87.
7. Kate Marek, “Using Web Analytics in the Library,” Library Technology Reports 47, no. 5 (2011): 22.
8. George R. Jaramillo and Joan G. Lamborn, “Document Delivery in Times of Shrinking Budgets,” Resource Sharing and Information Networks 11, no. 1-2 (1996): 5–15; Judith N. Nixon, “Using a Database Management Program as a Decision Support System for the Cancellation of Business Periodicals and Serials,” Journal of Business & Finance Librarianship 5, no. 4 (2000): 3–21; Pat L. Walter, “Doing the Unthinkable: Cancelling Journals at a Research Library,” Serials Librarian 18, no. 1-2 (1990): 141–53; Edward S. Warner, “The Impact of Interlibrary Access to Periodicals on Subscription Continuation/Cancellation Decision Making,” Journal of the American Society for Information Science 32, no. 2 (1981): 93–95; Jeanie M. Welch, “Is There Life after Serials Cancellation?” The Bottom Line 9, no. 2 (1996): 18–20.
9. Thomas L. Kilpatrick and Barbara G. Preece, “Serial Cuts and Interlibrary Loan: Filling the Gaps,” Interlending & Document Supply 24, no. 1 (1996): 16.
10. Mary Dabney Wilson and Whitney Alexander, “Automated Interlibrary Loan/Document Delivery Data Applications for Serials Collection Development,” Serials Review 25, no. 4 (1999): 11–19.
11. Janet Hughes, “Can Document Delivery Compensate for Reduced Serials Holdings? A Life Sciences Library Perspective,” College & Research Libraries (1997): 426.
12. Michele J. Crump and Leilani Freund, “Serials Cancellations and Interlibrary Loan: The Link and What It Reveals,” Serials Review 21, no. 2 (1995): 33.
13. Eleanor Gossen and Sue Kaczor, “Variation in Interlibrary Loan Use by University at Albany Science Departments,” Library Resources & Technical Services 41, no. 1 (1997): 17–28.
14. Andrea L. Duda and Rosemary L. Meszaros, “Validating Journal Cancellation Decisions in the Sciences: A Report Card,” Issues in Science & Technology Librarianship (1998), accessed July 30, 2014, www.istl.org/98-summer/article4.html.
15. Jonathan Nabe and David Fowler, “Leaving the Big Deal: Consequences and Next Steps,” paper presented at the 26th Annual Conference of the North American Serials Interest Group, St. Louis, June 3, 2011; see also Nabe and Fowler, “Leaving the ‘Big Deal.’”
16. Calvert, Fleming, and Hill, “Impact of Journal Cancellations,” 186.
17. Tabatha A. Farney and Nina McHale, “Introducing Google Analytics for Libraries,” Library Technology Reports 49, no. 4 (2013): 5.
18. Steven J. Turner, “Website Statistics 2.0: Using Google Analytics to Measure Library Website Effectiveness,” Technical Services Quarterly 27, no. 3 (2010): 261–78.
19. For example, see Wei Fang and Marjorie E. Crawford, “Measuring Law Library Catalog Web Site Usability: A Web Analytic Approach,” Journal of Web Librarianship 2, no. 2–3 (2008): 287–306; Julie Arendt and Cassie Wagner, “Beyond Description: Converting Web Site Statistics into Concrete Site Improvement Ideas,” Journal of Web Librarianship 4, no.1 (2010): 37–54; Sara Memmott and Susann deVries, “Tracking the Elusive Student: Opportunities for Connection and Assessment,” Journal of Library Administration 50, no. 7–8 (2010): 798–807; Paul Betty, “Assessing Homegrown Library Collections: Using Google Analytics to Track Use of Screencasts and Flash-Based Learning Objects,” Journal of Electronic Resources Librarianship 21, no. 1 (2009): 75–92; Elizabeth L. Black, “Web Analytics: A Picture of the Academic Library Website User,” Journal of Web Librarianship 3, no. 1 (2009): 3–14; Wayne Loftus, “Demonstrating Success: Web Analytics and Continuous Improvement,” Journal of Web Librarianship 6, no. 1 (2012): 45–55; Christopher J. Prom, “Using Web Analytics to Improve Online Access to Archival Resources,” American Archivist 74, no. 1 (2011): 158–84.
20. Marek, “Using Web Analytics in the Library.”
21. Jody Condit Fagan, “The Suitability of Web Analytics Key Performance Indicators in the Academic Library Environment,” Journal of Academic Librarianship 40, no. 1 (2014): 25–34.
22. Behnam Taraghi et al., “Web Analytics of User Path Tracing and a Novel Algorithm for Generating Recommendations in Open Journal Systems,” Online Information Review 37, no. 5 (2013): 672–91.
23. Gabriela Castro-Gessner, Wendy Wilcox, and Adam Chandler, “Hidden Patterns of LibGuides Usage: Another Facet of Usability,” paper presented at the Association of College and Research Libraries Conference, Indianapolis, April 10–13, 2013.
24. Jina Choi Wakimoto, David S. Walker, and Katherine S. Dabbour, “The Myths and Realities of SFX in Academic Libraries,” Journal of Academic Librarianship 32, no. 2 (2006): 127–36.
25. Tina Chrzastowski, Michael Norman, and Sarah Miller, “SFX Statistical Reports: A Primer for Collection Assessment Librarians” Collection Management 34, no. 4 (2009): 286–303.
26. Mark G. Stengel, “Using SFX to Identify Unexpressed User Needs,” Collection Management 29, no. 2 (2004): 7–14.
27. Polly P. Frank and Robert L. Bothmann, “Assessing Undergraduate Interlibrary Use,” Journal of Interlibrary Loan, Document Delivery & Electronic Reserve 18, no. 1 (2007): 33–48.
28. Doris M. Munson and Justin L. Otto, “Have Link Resolvers Helped or Hurt? The Relationship between ILL and OpenURL at a non-SFX Library,” OCLC Systems & Services: International Digital Libraries Perspectives 29, no. 2 (2013): 78–86; Karen Carter Williams and Timothy P. Bailey, “The Impact of Electronic Finding Aids on Interlibrary Loan: Desperately Seeking Full-Text,” Journal of Interlibrary Loan, Document Delivery & Electronic Reserve 18, no. 1 (2007): 113–22.
29. Eva Stowers and Cory Tucker, “Using Link Resolver Reports for Collection Management,” Serials Review 35, no. 1 (2009): 28–34.
30. For more information on NotePad++, see “NotePad++,” last modified July 27, 2014, accessed August 8, 2014, www.notepad-plus-plus.org.