A Comparative Analysis of Evidence-based Selection, Professional Selection, and Selection by Approval Plan

Molly Strothmann and Karen Rupp-Serrano

Molly Strothmann (molly.strothmann@okstate.edu) is an Associate Professor and Library Collections Strategist at Oklahoma State University. Karen Rupp-Serrano (krs@ou.edu) is Associate Dean, Scholarly Communications and Resources, University of Oklahoma Libraries.

Manuscript submitted August 14, 2019; returned to authors for revision October 10, 2019; revised manuscript submitted October 15, 2019; accepted for publication October 16, 2019.

This study compares three different models for selecting e-books for a research library’s collection. From 2013 to 2018, the University of Oklahoma Libraries contracted with Elsevier for an evidence-based selection (EBS) agreement. The titles in that EBS package were compared to the approval plan parameters to determine which books would have been purchased on approval during those years if Elsevier had been included among the publishers profiled. Subject librarians also made hypothetical selections as though they were placing firm orders from this collection. The approval plan selections and librarians’ selections were compared to usage data to determine how closely each selection model matched patrons’ choices.

Selection is “[t]he process of deciding which specific materials should be added to a library collection.”1 The selection process has varied and evolved over time, as libraries have employed different strategies for making selections and embraced different philosophies about who should be responsible for selection decisions. Finding the optimal model has sometimes been an object of debate within the profession, not least because it can be challenging to determine if the collection acquired meets its purpose.

Many approaches to assessing how successfully a library’s monograph collection has been built depend on counting the number of uses (print circulations or online views) that books receive. These approaches consider whether patrons have used books after they were added to the collection. Evidence-based selection (EBS, also known as evidence-based acquisition or EBA) is a selection model that reverses the steps: use precedes selection.2 First, patrons have the opportunity to use e-books from a specified collection. After enough time has elapsed to allow evidence to accumulate, librarians can use it to inform their selections, buying books whose value to patrons has been demonstrated by usage data.

The University of Oklahoma (OU) Libraries began engaging in EBS approximately seven years ago, including multiple models from different publishers. The Libraries’ EBS agreement with Elsevier began in 2013 with its Evidence-Based Model, which included books published in 2012 and 2013. This agreement focused on content in the sciences and engineering and included the following subject areas: biochemistry, genetics, and molecular biology; chemical engineering; chemistry; computer science; earth and planetary sciences; energy; engineering; environmental science; finance; immunology and microbiology; materials science; mathematics; physics and astronomy; and psychology. When this agreement was initiated, Elsevier was removed from the Libraries’ approval plan profiles. In 2014, the original agreement was replaced with the Elsevier Freedom Collection, which provided access to titles published from 2010 forward, with new content added annually until the agreement ended in 2018. This agreement added content in several health science disciplines plus the following subject areas: agricultural, biological and food science; fats and oils; forensics, security and criminal justice; plastics engineering; and social sciences. As the larger agreement included more content of relevance to their students and faculty, the university’s Health Sciences Center (HSC) was added to the agreement and gained access to all of its content at this time.

This study compares the data generated by the Elsevier EBS agreement to hypothetical purchases on the Libraries’ approval plan and by individual selectors. It explores the question: if the Libraries had not participated in this agreement and had instead selected Elsevier titles via the regular terms of the approval plans, would the same titles as those selected by users have been purchased? Likewise, if subject librarians had placed firm orders for individual titles, how closely would their selections have matched patrons’ usage?

Literature Review

Throughout the nineteenth and much of the twentieth century, libraries built their collections title by title. Initially, academic faculty were the usual selectors; later, librarians took over selection decisions.3 Regardless of who made the selections, the process was inefficient and expensive.4 Approval plans were introduced in the early 1960s, the first of which Abel developed for Washington State University at Pullman.5

Although the number of approval vendors has dwindled in recent years, variations on and complements to the traditional approval plan have multiplied. These expansions have been driven primarily by the advent of electronic books (e-books) and the development of new purchase-on-demand models called patron- or demand-driven acquisition (PDA/DDA). While modern libraries have always acquired materials at the request of users, PDA/DDA took on greater prominence as a formal selection model about ten years ago. PDA originated as part of interlibrary loan (ILL) acquisition strategies: rather than borrow books requested through ILL, some libraries found it cost-effective and collection-appropriate to buy them outright.6 The approach expanded further as e-book vendors implemented PDA/DDA programs near the beginning of the twenty-first century, offering both patron-driven purchases and short-term loans (STL) of e-books.7 More than 100 articles on PDA/DDA have been published since 2011 in peer-reviewed library-related journals, indicating the profession’s interest in this development.

A more recent variation on PDA/DDA is EBS. This model tends to be offered at the publisher level and provides libraries with unlimited access to a substantial portion of the publisher’s e-book output, usually including frontlist titles, for a set contractual term. At the beginning of the agreement, libraries commit to spending a certain amount to purchase e-books from the EBS title list. However, librarians make their purchase selections only after having had time to accrue, collect, and analyze usage data.8 Since EBS is a fairly new practice, there are currently few articles that discuss it. After conducting three EBS trials, librarians at the University of Liverpool decided to retain some collections in full based on strong usage and to select on a title-by-title basis for others.9 Levine-Clark characterized the Palgrave EBS as a success at the University of Denver.10

With multiple selection models in place, librarians have spent considerable time and effort comparing them: the literature describes comparisons of approval plans to librarian selection; approval plans to PDA/DDA; and PDA/DDA to librarian selection. Studies of PDA/DDA have included both print and e-books. Comparisons of librarian selections and approval plans have focused on numerous aspects; circulation has been examined quite frequently and, for the purposes of this study, is the most relevant metric. Two of the earliest papers indicated that titles selected by librarians demonstrated higher usage.11 More recently, Tucker found slightly higher circulation numbers among books purchased on approval at the University of Nevada–Las Vegas, although that result varied by subject area.12 Studies at the University of Nebraska–Lincoln and the University of Houston also indicated the effectiveness of librarian selection as demonstrated by circulation.13

A small number of studies have compared approval plans to PDA/DDA. The University of Iowa’s collection contained print duplicates of 166 e-books selected by PDA, 23 percent of the total PDA purchases made during an eleven-month study. Researchers found that, based on usage, users preferred e-books when available, and that when an e-book became available, print circulation dropped, particularly for books with two or more print circulations before the PDA program started. Even when a newer print edition was available in the library’s collection, older editions were used more when available online.14 A comparison at the University of Nebraska–Lincoln of the circulation performance of print books selected by librarians, sent on approval plans, and acquired via patron ILL requests found that approval plan titles did not circulate as frequently as either the ILL purchase-on-demand or librarian-selected titles.15 Kent State University used acquisition and usage data comparing DDA and print book acquisitions to determine which selection model better served library users’ needs, and which provided a better return on the library’s investment.16 Correspondence with the lead author clarified that the sample included print books acquired via both the approval plan and librarian selection, meaning that the study was not focused exclusively on comparing approval plans to PDA/DDA. The authors found that more e-books than print books were used during the study; that cost per use for the time period studied was equivalent between the formats but that e-books would likely generate a more favorable cost per use over time; and that uses of print and e-books aligned fairly well with overall acquisitions.

Finally, numerous studies in the past decade have compared DDA/PDA and librarian selection. The study most closely approximating the one described in this paper was conducted at Sam Houston State University.17 The authors compared PDA titles selected by users to those that librarians would have selected. During the sixteen-week PDA pilot, 637 titles were purchased on demand, while librarians selected 8,567 titles. Patron and librarian selections resembled each other in content level and recommended use, but overall the two groups did not tend to select the same titles, perhaps because of differing motivations (patrons satisfied immediate information needs; librarians built collections for the future). The aforementioned Kent State study compared DDA to both approval- and librarian-selected print acquisitions and found that e-books had a usage advantage over print books acquired through either model.18 The University of Nebraska–Lincoln study, also previously mentioned, found that ILL purchase-on-demand titles circulated more heavily than either librarian selections or approvals.19 Another study at the University of Nebraska–Lincoln compared ILL purchase-on-demand titles to librarian selections with regard to collecting levels and Library of Congress (LC) classification. Researchers found that purchase-on-demand and librarian selections diverged on LC class, but not substantially enough to cause concern. They also found that there were differences between how patrons and librarians spent acquisitions dollars, but again, those differences were minor.20

In 2015, two studies were conducted at the University of Florida and Iowa State University comparing DDA/PDA and librarian selection. At the University of Florida, librarian-selected e-books were less expensive on average ($88.45) than PDA titles ($123.04), but librarian-selected e-books had a higher average cost per use ($22.21) than PDA titles ($8.88).21 At Iowa State, patron selections included both titles triggered for purchase via DDA and usage in a leased collection. Breaking down patron and librarian selections by LC class, the researchers found that 26 percent of LC class ranges were selected through DDA but not by librarians. Librarians selected more titles in science and technology classes than patrons did; patron and librarian selections more closely resembled each other in social sciences and humanities classes.22 No published research could be found that directly compared books purchased through EBS to either librarians’ or approval plans’ selections, as this study undertakes.

Method

Sample

The study examined a subset of e-books from Elsevier’s Freedom Collection that met the following criteria:

• They became available online before December 31, 2016.
• They were published between 2012 and 2017 (some books with 2017 publication dates became available before the end of 2016 and were included).
• They were still available for purchase from Elsevier as of the date when the usage report was generated, and list prices could be obtained.
• They were in subject areas of interest to the university’s main campus. The Freedom Collection agreement was shared between the main campus and the HSC, with proportionate costs borne by each. Biomedical subject areas likely to be used mainly by HSC students and faculty were excluded, and all calculations related to spending were based only on the main campus’s share of the cost.

The sample contained 3,781 titles, all of which were from the 2015 and 2016 Freedom Collections. Most of the books were on engineering or science subjects; the remainder were in social sciences or business. A COUNTER Book Report 2 (BR2), which tallies successful section requests by title and month, was obtained for January 2013 to April 2017.23 Throughout this study, reported usage consists of the total number of times items were used within that reporting period.

Procedure

Elsevier’s subject areas were used to assign each of the e-books into one of four broad groups corresponding to the subject assignments of four main-campus librarians (see table 1).

The four subject librarians whose academic areas were represented by these Elsevier collections were given spending targets and asked to make selections within their subjects, identifying books that would have been firm orders if Elsevier e-books had been individually selected. This study refers to their selections as simulated firm orders. The spending targets were based on the number of e-books in each subject area—for example, since 58 percent of the titles in the sample were in engineering, the engineering librarian was given a budget equal to 58 percent of the main campus’s EBS spending with Elsevier. The allotments, as percentages of the total budget, are provided in table 1. Subject librarians were given spreadsheets that provided information comparable to what would be available during ordinary firm ordering—title, subject area, list price, ISBN, publication year, series title (if applicable), imprint, and URL on the publisher’s website. However, librarians made their selections without reference to the e-books’ usage data.

Separately, the Libraries’ principal book vendor, GOBI, identified books that would have been purchased automatically if Elsevier had been included among the Libraries’ approval publishers between 2012 and 2016. This study refers to these selections as simulated approval purchases. Books were profiled according to the stipulations of the Science and Technology and Social Sciences and Humanities approval plans, using classification number areas and all applicable non-subject parameters. The plans’ standard price caps of $200 per book on the SciTech plan and $150 per book on the SSH plan were used, but there was no limit imposed on total spending. GOBI’s report included all Elsevier titles that became available to the vendor during this time period, not just titles from the Freedom Collection; however, only the latter were considered in this analysis. Analysis compared simulated firm orders and simulated approval purchases to the reported use that these books received.

Results

Freedom Collection Use Summary

The COUNTER BR2 Report tallies the number of uses that e-books receive, and this analysis focuses on usage data. Throughout this study, however, e-books that received use by patrons are termed “accessed books” rather than “used books” to avoid confusion with the expression “used books” as in “pre-owned.”

Of the 3,781 books in the sample, 1,486 (39 percent) were not accessed during the time period examined in the study. The mean list price of those non-accessed books was $230.55. The remaining 2,295 books (61 percent) were accessed at least once. Their mean list price was $220.85, meaning that accessed books were less expensive than non-accessed books, but only slightly. Cost per use for accessed titles ranged from $0.07 to $750.00 and averaged $66.62.

The 2,295 accessed books were used a mean 31.8 times, with number of uses ranging from one to 1,714. The values of the first quartile, median, and third quartile were two uses, eleven uses, and thirty uses, respectively. Although the distribution precluded grouping the books perfectly into quartiles, accessed books were categorized as low, low-medium, medium-high, or high use relative to those values:

• Low use: 1 or 2 uses (623 books);
• Low-medium use: 3 to 11 uses (538 books);
• Medium-high use: 12 to 30 uses (573 books); and
• High use: 31 to 1,714 uses (561 books).

Within the “high use” category, the top 5 percent of titles by usage were singled out for further analysis. This group contained 115 books that were used 132 times or more.

Usage levels were consistent among the four subject categories, except that a slightly higher percentage of books in the social sciences were accessed than in the other three subject categories. However, titles in the social sciences also had the lowest mean number of uses (see table 2).

Simulated Firm Orders

The budgets given to the subject librarians enabled them to select 1,074 books as simulated firm orders. Their selections included 359 of the 1,486 non-accessed (24 percent) and 715 of the 2,295 accessed titles (31 percent). Librarians’ selections had a mean list price of $189.51, 16 percent lower than the mean list price for the sample overall ($224.66). Librarians’ selections from the books that patrons accessed received a mean 44.3 uses per book, with a mean cost per use of $41.01.

In general, as usage increased, so did the likelihood that librarians would select titles as simulated firm orders. Overall, librarians selected 22 percent of the low-use books (140 of 623) and 40 percent of the high-use books (223 of 561). The business librarian was the only selector whose choices were not consistent with that tendency; however, as there were only ninety-three business books in the sample, and the business budget was only sufficient to simulate ordering thirty-five of them, it is not surprising that this subject area did not demonstrate a clear selection pattern. The social sciences librarian selected a higher percentage of books overall because the list prices of books in that subject category were considerably lower than those in engineering or the sciences (see figure 1).

Further analysis considered the most highly used books. If very heavy usage is a sign that a resource is indispensable to a library’s collection, then the books that received the most use should all or nearly all have been selected as simulated firm orders. Librarians selected fifty-nine of the 115 books that comprised the top 5 percent by usage (51 percent). That rate substantially exceeds the percentage of books that librarians selected as simulated firm orders overall and continues the trend that they were more likely to select higher-use books than lower-use or unused ones.

Librarians were given a budget based on the cost of the subscription agreement. Ordering at list price, that budget would not have sufficed to purchase all the books in the Freedom Collection, or indeed all the books that patrons used. If orders were placed based only on usage, beginning with the most-used title and descending the list until the funds were exhausted, librarians would have been able to purchase 927 titles (40 percent of the accessed books) that were used sixteen times or more. That set of 927 hypothetical purchases would have included 539 books in engineering, 270 in science, 93 in social sciences, and 25 in business. That subject-area distribution is nearly identical to the composition of the sample as a whole.

Simulated Approval Purchases

Had Elsevier been included among the Libraries’ approval publishers during this study’s time frame, 1,617 print titles from this sample would have been purchased on approval. These selections included 634 of the 1,486 non-accessed titles (43 percent) and 983 of the 2,295 accessed titles (43 percent). Simulated approval purchases had a mean list price of $176.85.

The Libraries’ GOBI representative generated an approval report using price ceilings of $150 for books in the social sciences and $200 for books in the sciences. However, that report used GOBI’s prices, which were generally lower than Elsevier’s list prices for these titles. Therefore, although some titles were excluded based on price, the simulated approval purchases included a number of titles that exceeded those price ceilings according to the Elsevier prices reported throughout this study.

The simulated approval purchases included the largest percentage of available titles in the social sciences and the smallest percentage in business. Simulated approval purchases included:

• 960 of the 2,177 engineering books (44 percent)
• 400 of 1,015 science (39 percent)
• 228 of 496 social sciences (46 percent)
• 29 of 93 business (31 percent)

Across subject areas, the approval plan consistently captured the same percentage of accessed and non-accessed books as simulated approval purchases. Only in the smallest category, business, was there any difference, with a smaller percentage of accessed than non-accessed titles identified as simulated approval purchases (see figure 2).

Unlike the individual selectors, the approval plan was not more likely to select high-use than low-use titles (see figure 1). Approval orders would have included 43 percent of the non-accessed books and between 39 percent and 46 percent of the accessed books within each usage level:

• 634 of the 1,486 non-accessed books (43 percent)
• 246 of the 623 low-use books (39 percent)
• 224 of the 538 low-medium use books (42 percent)
• 265 of the 573 medium-high use books (46 percent)
• 248 of the 561 high-use books (44 percent)

The approval plan would have delivered 49 of the 115 books in the top 5 percent by use (43 percent), the same percentage that it would have delivered of the sample overall.

Although the simulated approval purchases were limited by price ceilings for individual titles, a total budget was not imposed on them (unlike simulated firm orders). Using Elsevier’s list prices, the amount that would have been spent via the approval plan exceeded expenditures on the main campus’s share of the Freedom Collection by 41 percent.

Textbooks

The authors’ library has a separate program to purchase reserve copies of some required textbooks, prioritizing those that are more expensive (over $90) and/or serve high-enrollment classes. Therefore, some subject librarians consider the Libraries’ textbook collecting adequate and avoid purchasing other textbooks with their firm ordering funds. Similarly, the approval plans exclude most textbooks from automatic purchasing, except for certain graduate-level textbooks in the sciences. The authors hypothesized that some of the discrepancies between simulated firm orders, simulated approval orders, and users’ behavior might be explained by the different approaches that those selectors took toward textbooks.

According to GOBI, 820 of the 3,781 titles within this collection (22 percent) were textbooks (Elsevier did not label any books in the sample as such). Not surprisingly, textbooks were more heavily used than the collection as a whole: 555 textbooks (68 percent) were used at least once, and those 555 textbooks were accessed a mean 52.8 times. Accessed textbooks had a mean list price of $179.71.

Nearly one-third (29 percent) of librarians’ simulated firm orders were textbooks. Librarians selected 1,074 of the 3,781 books on the list (28 percent). They selected 315 textbooks, 38 percent of the 820 textbooks available. The assumption that librarians would avoid titles that they recognized as textbooks was, therefore, not supported—librarians actually selected a higher proportion of textbooks than books in general. Librarians selected 223 of the 555 accessed textbooks (40 percent) and 92 of the 265 non-accessed textbooks (35 percent). As with the sample overall, librarians had a higher rate of selection among accessed than non-accessed titles, but the difference was minor.

The approval plan would have delivered 315 textbooks, or 38 percent of the 820 textbooks available. Coincidentally, the exact same number of textbooks were selected as simulated firm orders and as simulated approval purchases, although librarians chose different specific titles than the approval plan did. All the textbooks captured by the approval plan were graduate-level, and all but three were in science or engineering subjects. The simulated approval textbook purchases included 195 of the 555 accessed textbooks (35 percent) and 120 of the 265 non-accessed textbooks (45 percent). While the simulated approval purchases were as likely overall to capture accessed and non-accessed titles (43 percent of each), the approval plan captured a below-average proportion of accessed textbooks. This discrepancy is probably a consequence of the academic level and subject areas of the accessed textbooks: the Libraries’ approval plan excludes all introductory or undergraduate textbooks and nearly all textbooks at any level in the social sciences and humanities by default. Patrons, however, used them.

Selection Patterns by Subject

Selections were further analyzed within Elsevier’s subject areas. Two subject areas, fats and oils and plastics engineering, were excluded from the analysis because they contained too few titles (eight and six, respectively) to demonstrate any meaningful trends. The other categories contained between 51 and 835 titles each.

Overall, patrons accessed 2,295 of the 3,781 titles available in the EBS collection, or 61 percent. Their access rates were fairly consistent across subject areas: usage ranged from 47 percent of the available physics and astronomy books to 75 percent of the psychology books. The approval plan showed more variation by subject area. Overall, 1,617 titles were identified as simulated approval purchases, or 43 percent. Selection rates ranged from 24 percent of available titles in both materials science and agricultural, biological, and food sciences to 62 percent in social sciences. In only one subject, physics and astronomy, did the approval plan select a larger number of titles than did patrons. The largest discrepancies between patron and approval plan selection rates were in psychology (patrons accessed 75 percent of available titles; the approval plan identified just 38 percent) and in agricultural, biological, and food sciences (patrons accessed 57 percent; approval plan, 24 percent). Patrons and the approval plan were most similar in environmental science (both selected 57 percent of the available titles) and in computer science (patrons, 60 percent; approval plan, 57 percent).

Librarians showed even more variation by subject area than the approval plan. Overall, they selected 1,074 titles from the EBS collection as simulated firm orders, or 28 percent. Selection rates ranged from 8 percent of available titles in immunology and microbiology to 71 percent in psychology. Like the approval plan, librarians selected more titles than patrons in only one subject area; again, it was physics and astronomy. The largest discrepancies between patron and librarian selections were in immunology and microbiology (patrons accessed 63 percent of available titles; the subject librarian selected just 8 percent of them as simulated firm orders), and in forensics, security, and criminal justice (patrons, 64 percent; subject librarian, 9 percent). Librarians matched patron behavior most closely in the subjects of psychology (patrons accessed 75 percent of the titles in that area; the subject librarian selected 71 percent of them) and social sciences (patrons, 63 percent; subject librarian, 57 percent) (see table 3).

Discussion

Neither the simulated firm orders nor the simulated approval orders perfectly mirrored patrons’ usage. However, librarians were more successful than the approval plan at selecting the titles that patrons accessed most frequently. Within their own selections, librarians were also more likely to select high-use than low-use titles. Overall, simulated firm orders showed a steady upward trend from 22 percent of the low-use books to 40 percent of the high-use books. The approval plan, conversely, was approximately as likely to select books from all four usage levels. The ratio of accessed to non-accessed titles that librarians selected as simulated firm orders was approximately 2 to 1. Within the simulated approval purchases, the ratio was closer to 3 to 2. Finally, the accessed titles that librarians selected as simulated firm orders had higher mean usage and lower mean cost per use than the accessed titles overall. If the success of selection is measured by the usage of selected titles, then taken together, these results support the previous research noted in the literature review that implies that librarians are more effective at selection than approval plans.

Simulated approval purchases and simulated firm orders showed considerable duplication, with 580 titles appearing in both groups. In reality, no overlap would have occurred because librarians exclude approval purchases when making their selections. However, this result does imply that librarians apply some, but not all, of the same selection criteria when placing firm orders that they used to develop approval plan parameters.

Both librarians and the approval plan matched patrons’ behavior more closely in some subject areas than others. In most cases, it is easy to understand why both the approval plan and the librarians selected as they did. For example, since OU has no agriculture department, the approval plan parameters limit or exclude most agriculture titles. The science librarian, similarly, tends to exclude books on agricultural science when placing firm orders. Elsevier’s subject area “agricultural, biological, and food sciences” contained a large number of titles that both the approval plan and the subject librarian treated as out of scope for the collection. Elsevier’s subject area “immunology and microbiology” is similar: both the approval plan and the subject librarian selected from the microbiology titles but excluded the immunology books as more appropriate to the HSC’s library than to the main campus’s. Declining to collect materials in a subject area in which the university has no program is an eminently logical decision for an academic library. However, examining the usage data within this EBS collection reminds us that our understanding of our patrons’ needs is imperfect. Perhaps, despite the lack of an agriculture department, faculty appointed in another academic discipline are doing research in that area without librarians being aware of it. If so, the results from the EBS usage data might be a signal that adjustments to the Libraries’ collecting decisions and approval plan profiles are needed. Perhaps usage in this area is a consequence of the increasingly interdisciplinary nature of research, which makes a blanket omission of a topic as “out of scope” obsolete. Or perhaps some results simply signal a niche interest from an individual patron that could not have been anticipated. Based on the authors’ knowledge of the Libraries’ collection needs, both the approval plan and the subject librarian were right to decline purchase of Essential Oils in Food Preservation, Flavor and Safety, and yet patrons accessed it 202 times.

Neither the simulated approval purchases nor simulated firm orders in this study captured anywhere near the full number of titles available through the Elsevier EBS program. Similarly, neither simulated approval purchases nor simulated firm orders captured all the EBS titles that patrons accessed while they were available. The budgetary constraints under which subject librarians operated while making their selections enabled them to select only 1,074 titles, while the approval plan would have delivered 1,617. During the EBS program, patrons had access to 3,781 and used 2,295 of them.

Therein lies much of the appeal of the evidence-based selection model: it stretches budgets by giving patrons immediate access to a larger catalog of titles for the same amount of money. However, it should be emphasized that what is expanded is temporary access—in effect, rentals—not permanent ownership. While libraries participate in an EBS plan, their patrons can use all the titles in the agreement. When they discontinue participation, only the titles that they ultimately select for permanent purchase remain in their collection for future use. In that respect, EBS is not unlike the traditional acquisition model in which a library’s permanent selections are augmented through temporary ILLs.

Titles were selected for purchase twice—at the end of the first year of the EBS agreement and in March 2019 after the agreement ended. Initial credits sufficed to allow the OU Libraries to purchase 217 titles. The second purchase, made using multiple years’ credits, included 1,262 titles. Having concluded the EBS agreement, the Libraries have permanent ownership of 1,479 titles—a number roughly comparable to what would have been acquired via either the approval plan or selectors’ firm ordering and substantially smaller than the number that patrons accessed during the EBS agreement. It should be noted again that Elsevier’s list prices were higher for many titles than what the Libraries might have paid through GOBI for the same titles. EBS agreements can be an economical way to increase the number of titles immediately available to patrons in the short term, but they assuredly do not represent a revolution in libraries’ spending on scholarly books. Understanding that EBS results in approximately the same number of titles being added to the permanent collection as any other acquisition model, librarians who are considering incorporating it into their selection strategies must weigh the reduction in spending flexibility with a given publisher against the benefits of immediate but temporary access to a large e-book catalog and expanded data to inform purchase decisions.

When the selector (this paper’s second author) ultimately made purchase decisions, she gave the usage report considerable weight; however, it was not dispositive (she did not hew strictly to the list of the 1,479 most-accessed titles in selecting the Libraries’ 1,479 purchases.) She also considered price, topical redundancy, and the predicted ongoing usefulness of each title before adding it to the permanent collection. Furthermore, usage was interpreted in context: average cost per year was considered so that older titles, which had been available to patrons and accumulating views for years, were not automatically privileged over newer titles with lower totals but strong recent usage. Critical judgment, similar to that which subject librarians use to make title-by-title selections, was applied. For example, the EBS collection contained fourteen different guides to the software MATLAB, including multiple editions of some titles. Patrons accessed thirteen of them at least once. In making simulated firm order selections, the science and engineering librarians chose seven of them. Librarians made the professional judgment that purchasing half of these titles would create a satisfactory treatment of the topic within the permanent collection, taking into account the price, duplication of print holdings, and uniqueness of each title. When the real purchases were ultimately made, nine of the accessed MATLAB books were selected, a decision that balanced users’ demonstrated interest in the topic against the professional goal of building a strong collection without wasteful redundancy. The authors suggest that future research, ideally across multiple academic libraries, should examine how librarians weigh and interpret usage data in making purchase selections from their EBS packages.

Conclusion

Publishers promote EBS agreements to reduce the risk of purchasing materials that will not be used by offering librarians the opportunity to defer selections until after they have collected and reviewed usage data. However, there is generally no flexibility during the “rental” period when the EBS agreement is generating data. Libraries commit to a certain spending level and receive access to a fixed collection throughout the period of their EBS agreement, accepting the risk that it may not yield the anticipated usage.

Patrons at the OU Libraries—not unexpectedly—did not access a substantial percentage (39 percent) of books in this study when they were available through the EBS agreement. The authors recommend that vendors increase the flexibility of these agreements during the data-generating period. Libraries should be given the ability to revise the parameters of multi-year EBS agreements after an initial year or two of participation, an option that some vendors have begun to provide. With plans that are structured like the OU Libraries’ Elsevier Freedom Collection EBS agreement, those revisions could entail removing specific subject collections from an active agreement. If a customer finds after two years that patrons are not accessing titles in a given collection (the physics and astronomy collection, for example), it would be beneficial to have the option to eliminate it from the EBS agreement in exchange for a fair corresponding price reduction for its remaining years. Some customers might choose to keep all subject collections available and simply make their purchases from more heavily used subject areas. However, customers who planned their spending based on the assumption that all subject areas would prompt purchases might find themselves without enough evidence-based purchases to make if some portions of the EBS collection prove unviable. If librarians are reluctant to remove content from their catalogs entirely, they might consider pursuing other means of providing access to it that will have less impact on their budgets. As an obvious example, they could remove low-use collections from their EBS agreements and then add records for those titles to an e-book aggregator. They may not find themselves able to replace everything—publishers who offer EBS agreements on their own platforms frequently limit availability of titles on aggregators—but a substantial portion could be replaced using this approach. The library would benefit by continuing to offer some access to content in the low-use subjects and would probably spend much less money on STL fees on the aggregator platform than it would on the EBS agreement.

Vendors are never pleased with spending reductions, and from their perspective, increasing the predictability of their customers’ spending on e-books is a significant reason they offer EBS packages. However, providing customers with the ability to make changes relatively early in an evidence-based plan could garner a great deal of valuable customer goodwill. If vendors are not receptive to negotiating the contents and price of active EBS agreements, an alternative would be to permit customers to “bank” part of the funds they had committed to paying for e-book purchases and reallocate them to other purchases with the vendor if they are unable to justify purchasing enough e-books to equal the spending originally planned.

The usage reports generated by EBS agreements also allow librarians a new avenue for collection analysis. While patrons have access to an entire EBS package, they are able to use, and demonstrate their use of, resources that libraries might not otherwise have acquired. Librarians may be able to analyze that data to identify gaps in their collection practices. If, for example, an EBS package includes books in a subject area or at a content level that a library does not usually collect, but that patrons use heavily when they are available, librarians might take that as a cue to revise their approval plan parameters.

Neither approval plan selections nor professional selections by subject librarians precisely mirror or anticipate patrons’ usage of e-books. However, selection based on usage numbers alone may not create perfect collections either. Evidence-based selection agreements require librarians to commit to certain spending levels within specific packages from specific publishers. In exchange for that reduction in spending flexibility, they receive data to inform their purchases. Simultaneously, they are better able to incorporate professional judgment into their selections than a purely demand-driven acquisition model would allow.

References

1. Michael Levine-Clark and Toni M. Carter, eds., ALA Glossary of Library and Information Science, 4th ed. (Chicago: ALA Editions, 2013), s.v. “selection.”
2. Michael Levine-Clark, “Evidence-Based Selection at the University of Denver,” Against the Grain 27, no. 5 (2015): 18, 20, https://doi.org/10.7771/2380-176X.7176.
3. David O. Lane, “The Selection of Academic Library Materials, A Literature Survey,” College & Research Libraries 29, no. 5 (1968): 364–72, https://doi.org/10.5860/crl_29_05_364.
4. Ann L. O’Neill, “How the Richard Abel Co., Inc. Changed the Way We Work,” Library Acquisitions: Practice & Theory 17, no. 1 (1993): 41–46, https://doi.org/10.1016/0364-6408(93)90028-5.
5. Richard Abel, “The Origin of the Library Approval Plan,” Publishing Research Quarterly 11, no. 1 (1995): 46–56, https://doi.org/10.1007/BF02680417.
6. Marna Hostetler, “Purchase-on-Demand: An Overview of the Literature,” Against the Grain 22, no. 2 (2010): 46–47, https://doi.org/10.7771/2380-176X.5505; Judith M. Nixon, Robert S. Freeman, and Suzanne M. Ward, “Patron-Driven Acquisitions: An Introduction and Literature Review,” Collection Management 35, no. 3-4 (2010): 119–24, https://doi.org/10.1080/01462679.2010.486957; David C. Tyler, “Patron-Driven Purchase on Demand Programs for Printed Books and Similar Materials: A Chronological Review and Summary of Findings,” Library Philosophy & Practice, 2011, https://digitalcommons.unl.edu/libphilprac/635; Doug Way and Julie Garrison, “Financial Implications of Demand-Driven Acquisitions: A Case Study of the Value of Short-Term Loans,” in Patron-Driven Acquisitions: History and Best Practices, ed. David A. Swords (Berlin and Boston: De Gruyter Saur, 2011), 137–56.
7. Way and Garrison, “Financial Implications of Demand-Driven Acquisitions.”
8. Levine-Clark, “Evidence-Based Selection at the University of Denver.”
9. Terry Bucknell, “Buying By the Bucketful: A Comparative Study of E-book Acquisition Strategies,” Insights 25, no. 1 (2012): 51–60, https://doi.org/10.1629/2048-7754.25.1.51.
10. Levine-Clark, “Evidence-Based Selection at the University of Denver.”
11. G. Edward Evans, “Book Selection and Book Collection Usage in Academic Libraries,” The Library Quarterly: Information, Community, Policy 40, no. 3 (1970): 297–308; G. Edward Evans and Claudia White Argyres, “Approval Plans and Collection Development in Academic Libraries,” Library Resources & Technical Services 18, no. 1 (1974): 35–50.
12. James Cory Tucker, “Collection Assessment of Monograph Purchases at the University of Nevada, Las Vegas Libraries,” Collection Management 34, no. 3 (2009): 157–81, https://doi.org/10.1080/01462670902962959.
13. David C. Tyler et al., “Patron-Driven Acquisition and Circulation at an Academic Library: Interaction Effects and Circulation Performance of Print Books Acquired via Librarians’ Orders, Approval Plans, and Patrons’ Interlibrary Loan Requests,” Collection Management 38, no. 1 (2013): 3–32, https://doi.org/10.1080/01462679.2012.730494; Irene Ke, Wenli Gao, and Jackie Bronicki, “Does Title-By-Title Selection Make a Difference? A Usage Analysis on Print Monograph Purchasing,” Collection Management 42, no. 1 (2017): 34–47, https://doi.org/10.1080/01462679.2016.1249040.
14. Karen S. Fischer et al., “Give ’Em What They Want: A One-Year Study of Unmediated Patron-Driven Acquisition of e-Books,” College & Research Libraries 73, no. 5 (2012): 469–92, https://doi.org/10.5860/crl-297.
15. Tyler et al., “Patron-Driven Acquisition and Circulation at an Academic Library.”
16. Kay Downey et al., “A Comparative Study of Print Book and DDA Ebook Acquisition and Use,” Technical Services Quarterly 31, no. 2 (2014): 139–60, https://doi.org/10.1080/07317131.2014.875379.
17. Lisa Shen et al., “Head First into the Patron-Driven Acquisition Pool: A Comparison of Librarian Selections Versus Patron Purchases,” Journal of Electronic Resources Librarianship 23, no. 3 (2011): 203–18, https://doi.org/10.1080/1941126X.2011.601224.
18. Downey et al., “A Comparative Study of Print Book and DDA Ebook Acquisition and Use.”
19. Tyler et al., “Patron-Driven Acquisition and Circulation at an Academic Library.”
20. David C. Tyler et al., “Don’t Fear the Reader: Librarian versus Interlibrary Loan Patron-Driven Acquisition of Print Books at an Academic Library by Relative Collecting Level and by Library of Congress Classes and Subclasses,” College & Research Libraries 75, no. 5 (2014): 684–704, https://doi.org/10.5860/crl.75.5.684.
21. Steven B. Carrico et al., “What Cost and Usage Data Reveals About E-Book Acquisitions,” Library Resources & Technical Services 59, no. 3 (2015): 102–11, https://doi.org/10.5860/lrts.59n3.102.
22. Edward A. Goedeken and Karen Lawson, “The Past, Present, and Future of Demand-Driven Acquisitions in Academic Libraries,” College & Research Libraries 76, no. 2 (2015): 205–21, https://doi.org/10.5860/crl.76.2.205.
23. Mitchell Dunkley, “Friendly Guide to COUNTER Book Reports,” Project COUNTER, 2016, https://www.projectcounter.org/wp-content/uploads/2016/03/Library-pdf.pdf.