Shared records projects take many forms throughout the library world. In the United States, consortium members most commonly determine model cataloging standards and practices (AACR2/RDA, MARC21, LCSH or MeSH subjects, successive or latest entry serials cataloging, separate or merged records describing multiple versions), but they continue to add and maintain institutional records within their own ILS. MARC records are harvested from each consortium member’s ILS and merged in a separate centralized database, often maintained by the same ILS vendor individual consortium libraries use. This is the model described by Moeller at Prospector, the Colorado Unified Catalog, and is the model used by several other state consortia, most notably the University of California’s California Digital Library (CDL), the Illinois Library Computer Systems Organization (ILCSO), and OhioLink.⁵

Consortia have other record-sharing models available, like TRLN, whose members do not share a common integrated library system. The Virtual Library of Virginia (VIVA) Project allowed individual libraries with disparate ILSs to voluntarily for host, maintain, and distribute particular collections of set records to consortium participants.⁶ Unlike the Colorado, California, and Ohio model, which initially encompassed all formats, VIVA focused on electronic resources, which presumably required little local editing other than perhaps customizing the MARC 856 field to provide information for proxy server and display text.

The CDL also created a separate Shared Catalog Program (SCP) for managing e-resource metadata. In the SCP’s centralized model, University of California-San Diego library creates and maintains metadata and contributes these records to Melvyl, which then further distributes those records to nine campuses. In a 2002 article, French, Culbertson, and Hsiung delineated several factors that lead to success in shared cataloging projects, including common descriptive standards, high-quality metadata, timeliness, acceptance of records without local modification, use of holdings records for localized metadata, and good communication.⁷

Batch loading records for large collection sets is a strategy that libraries use to provide access to titles that are beyond the scale of current staffing levels. By relying on vendor metadata, libraries are able to make content discoverable much quicker than would be possible through manual copy cataloging procedures. Timeliness has its tradeoffs, however, including the potential for poor source metadata and the logistics for keeping current with vendor releases of record sets. Martin and Mundle discuss efforts to enhance the quality of vendor-supplied MARC metadata.⁸ They ask readers to consider the staff cost of these efforts and urge libraries and consortia to work with vendors up front to enhance the initial quality of metadata. In his 2009 article on batch loading records for open-access books, Beall also discussed the poor quality of third-party supplied records but described how timeliness of access to 100,000 titles can outweigh poor metadata quality.⁹

Most recently, Mugridge and Edmunds surveyed large academic libraries to assess current practices and issues associated with batch loading MARC records.¹⁰ Timeliness of record loads was an issue identified by a majority of the responding libraries. The authors also found that the eighteen libraries surveyed anticipated an increase in the importance of batch loading in the next five years, as long as the ILS continues to be identified as the database of record for a library’s holdings. Further, they identified the use of discovery-layer software as a factor that may affect batch loading workflows.

Stalberg and Cronin reported on the 2009 work of the “Task Force on Cost/Value Assessment of Bibliographic Control.”¹¹ The task group was charged by the Association for Library Collections and Technical Services (ALCTS) to “develop and articulate metrics for evaluating the cost and value of cataloging activities.”¹² Following an extensive review of the cost/benefit literature of cataloging, they identified seven operational definitions of value including discovery success, use of collections, improvements in display, interoperability of library bibliographic metadata, support for Functional Requirements for Bibliographic Records (FRBR) user tasks, throughput and timeliness, and support for library administrative goals.¹³

Stalberg and Cronin proposed that “the extent to which data-creation processes facilitate timeliness in resource availability” can be used as a measure of value.¹⁴ Furthermore, they argue that resources that are uncataloged are undiscoverable, and library patrons cannot use undiscoverable resources. These observations are consistent with studies indicating a negative correlation between cataloging backlogs and circulation of print materials.¹⁵ Howarth, Moore, and Sze identified a major cause for cataloging backlogs to be a mismatch in the quantity of cataloging work to be done and the capacity to complete it.¹⁶ They provided several suggestions to reduce backlogs including optimizing workflows, reallocating staff, using automated processes, and outsourcing cataloging tasks.

Writing in 2004, Fischer, Lugg, and Boese provided a ten-point checklist of business practices for reducing backlogs of print materials to release staff time for describing electronic resources.¹⁷ Though Fischer and her colleagues focused on cataloging backlogs of print materials, their recommendations to “control the Expert Mentality,” “automate and outsource where possible,” and “trust vendor-provided metadata” remain relevant in the context of cataloging electronic resources. Fischer, Lugg, and Boese noted that the expert mentality results in overly complex and often-manual procedures to solve problems in cataloging.¹⁸ Stalberg and Cronin echoed this sentiment when stating that “time spent on low-value activities (no matter which operational definition is used for ‘value’) is time not spent on high-value activities.”¹⁹

Stalberg and Cronin isolated several costs associated with managing bibliographic metadata including staff salaries, tools and systems, and database maintenance, which are inherent to any evaluation of library work processes.²⁰ Efforts to increase cataloging efficiency and timeliness should therefore be judged by their effect on controlling if not reducing the costs of cataloging and releasing expert staff to work on more complex issues or deferred processing projects.

In the hosted model, one TRLN institution assumes responsibility for hosting and maintaining metadata for a commonly held collection in their local ILS, and the host library shares the metadata with partner institutions through the Search TRLN indexes. Non-host institutions may also choose to maintain records for the set locally for reasons internal to that institution, but their records will be prevented from loading into Search TRLN through ingest filters in the metadata pipeline. In most cases, these record sets were removed from the local ILS of all but the responsible host library, although some federal document e-journals are still managed in serials knowledge bases.

Managing metadata in the hosted model includes three activities: local metadata preparation, indexing, and supporting display for library patrons. Some of these tasks are conducted by the host institution; others are managed by consortium staff.

The first step involves loading records into the host library’s ILS using typical batch loading processes. A good example is the Marcive Documents Without Shelves (DWS) record set acquired through a consortium license in 2011. UNC-Chapel Hill serves as the host institution for this collection. Each month, a UNC-Chapel Hill staff member downloads the monthly notification service file from the vendor and loads the records into their Innovative Millennium ILS. UNC-Chapel Hill takes responsibility for three maintenance tasks that were formerly handled by staff at each TRLN campus, including authority processing with an external vendor, setting holdings at OCLC for all four institutions using an OCLC group profile, and correcting URLs as needed.

Host institutions are responsible for adding identifying fields to each record in a given set so they can be isolated in the local ILS for global operations or extract. For instance, UNC-Chapel Hill adds a MARC 919 field including the text string “dwsgpo” to each of the GPO DWS records loaded into its ILS. UNC-Chapel Hill technical services staff can use this field to isolate these records for extract, for batch editing, and for archiving purposes. NCSU uses Sirsi-Dynix Symphony’s “Item Cat2” to identify EEBO and shared open-access records. Similar procedures are used to identify the other hosted record sets managed in the Shared Records program.

As noted above, each institution uses automated processes to send metadata extracts of their MARC, EAD, and DC records to the Search TRLN system for indexing. Similar processes are used to provide sets of shared records with Search TRLN for indexing. Again drawing on the DWS record set as an example, UNC-Chapel Hill provides regular maintenance of this dataset including URL corrections, monthly DWS bibliographic updates, and authority processing. UNC-Chapel Hill then extracts all of the DWS MARC records and sends them to the Search TRLN servers for indexing on a weekly basis.

At this point, processing is handed off to the Endeca applications, called pipelines, which prepare metadata for the indexes. The pipelines make several changes to each Shared Record set to support expected functionality in the user interfaces.

First, e-resource URLs must be made institution-neutral in the indexes. This typically involves removing a proxy server string from the URL as found in the extracted records. For instance, the record set for the NC LIVE Videos is hosted and maintained by NCSU and the record 856 field (Electronic Location and Access) for these records contains NCSU’s proxy server string, http://proxying.lib.ncsu.edu. The NC LIVE Video pipeline removes the NCSU proxy prefix from each MARC 856 field and stores an institution-neutral URL for each record in the index. Proxy information is restored later for display, if it is appropriate for the set.

Second, several new facet values are added to the records to support needed functionality in the user interfaces. A Shared Records flag is set to “true” for these records so the user interfaces can detect Shared Records and render them properly for end users. Additional facet values are added for each sharing institution. For example, facet values for institution (Duke, NCCU, NCSU, and UNC-Chapel Hill) and format (Internet resource and streaming video) are added to each of the records in the NC LIVE Video record set. An Access facet is also used to indicate whether or not the user interfaces should render the records as open access or IP-restricted resources. Additional logic removes these records from the host institution’s main pipeline, eliminating the possibility of creating duplicate records in the indexes.

The user interface code that drives Search TRLN and the local catalogs of each consortium member library needed minor modifications to display shared records. The most significant change was to restore the institution-specific URLs to support off-campus authentication for IP-restricted resources. UNC-Chapel Hill, Duke, and NCSU all use EZproxy (www.oclc.org/ezproxy). to provide remote access to IP-restricted resources. So in these cases, institution-specific proxy URLs are prepended to the institution-neutral URLs on-the-fly in the user interfaces. NCCU provides off-campus access to electronic resources through VPN access, so no additional processing is needed to render these URLs properly. The link text for IP-restricted (prepended proxy) and open access resources (no prepended proxy) is adjusted appropriately when records are rendered for patron use as shown in figures 4 and 5.

In the direct load model, metadata records are harvested from a vendor or third-party source and loaded directly into the Search TRLN indexes. This model includes three processes: harvesting metadata, indexing, and supporting display for library patrons. Metadata in DDI/XML format from ICPSR fall into this category and provide examples for discussion.

ICPSR generates metadata about its datasets using the Data Documentation Initiative (DDI) metadata specification (www.ddialiance.org) and makes it available on the ICPSR website in XML format. ICPSR currently uses the DDI Codebook 2.1 schema and Document Type Definition (DTD) to structure these documents. Once each week the entirety of the ICPSR XML corpus is downloaded to a TRLN server and prepared for indexing.

As in the hosted-record model, a specific Endeca pipeline prepares the DDI/XML for indexing. The first step transforms the codebooks into indexable documents. TRLN adapted an ICPSR-provided Extensible Stylesheet Language (XSL) stylesheet to transform each DDI/XML codebook into a format that could be indexed by Endeca.²⁴ The remainder of the pipeline adds facets appropriate for these records including the Duke, UNC-Chapel Hill, and NCSU institutional facets and the Statistical Dataset and Internet Resource format facets.

As with the hosted-record model, records loaded in the direct-load model also need institution-specific proxy URLs. This was not necessary for the ICPSR metadata because access control for these metadata is managed at the ICPSR website through individual logins associated with licensing institutions.

Before implementing the Shared Records Program, Duke, UNC-Chapel Hill, and NCSU independently loaded and maintained MARC records prepared annually by ICPSR. Once per year catalogers at each institution updated the ICPSR MARC records in each respective ILS. The ICPSR portion of the Shared Records Project generated several benefits. The first benefit is timeliness now that the ICPSR metadata are updated in the Search TRLN indexes on a weekly basis. Second, TRLN was able to access the entire codebook for each dataset. This allowed TRLN to index a greater proportion of the metadata about each dataset than could be done when relying on the MARC records. Third, all of the processing for the direct load model is automated and monitored by consortium staff. This allowed Duke, NCSU, and UNC-Chapel Hill to eliminate all ICPSR dataset records from their ILSs and eliminate three formerly duplicated workflows, allowing cataloging staff to address other projects.

Before 2010, the Duke University Libraries (DUL) managed e-resource holdings in two different systems, the ILS and a vendor-provided knowledge base. In late 2010, e-resource management functions were largely consolidated into a single knowledge base provided by a new vendor. This brought about a deepening comfort with managing resources and their associated bibliographic and administrative metadata outside of the ILS. The scale of electronic resource holdings made management of these resources through the ILS impossible. At the point of migration to the current knowledge base, the total number of unique electronic resources managed was 252,000. By the end of the fiscal year the following summer, 544,800 unique titles were being tracked in the knowledge base.²⁵ By November 2012, 1,059,795 unique titles were being tracked in the knowledge base. In addition to tracking resources, the knowledge base serves as a repository for details about the terms of access and workflow. Even discovery of these resources became mediated through the knowledge base, with the knowledge base provider supplying MARC records for tracked titles. Timeliness of access was also a feature that gave the knowledge base an advantage over the ILS. On the back-end, as soon as the platform and its associated titles could be tracked, metadata about those resources became available for reporting and documenting workflow decisions. More importantly, the gap between library access to a title and its discovery by the public shrank to twenty-four hours for the A–Z list compared to up to two weeks for the catalog. Thus the ILS ceased to be the database of record for electronic resource holdings, and the stage was set for managing discovery of resources via other means.

At the same time that a proliferation of often-transient electronic resources changed the staff’s perspective toward the back-end ILS as a collection management tool, advances in discovery interfaces led to abandoning of the traditional ILS-based OPAC as Duke’s primary discovery tool. By 2010, Endeca stood firmly as the library’s catalog. Two years later, DUL implemented a web-scale discovery tool. This meant that metadata sources were no longer confined to MARC records in the ILS, and that catalogers had to develop new, large-scale understandings of metadata and how it fuels discovery.

Changing perspectives and reconceptualization had created a willingness to test the Shared Record model with the Marcive DWS portion of the project. Actually seeing the model work in practice, while addressing known metadata and workflow needs, was critical for wholesale adoption. Over the course of various staffing and organizational changes at DUL, the workflow for the loading of MARC records for electronic government documents no longer followed Duke’s standards and practices for other electronic resources. An ILS migration in 2004 further added to the issues with this workflow, and staff had a metadata cleanup project to implement along with workflow changes. The workflow changes were easier to implement, though they still required staff time and maintenance. Untangling the metadata issues and fitting cleanup among other priorities was more complex. The proposal to share the DWS records between all TRLN libraries came at exactly the right time—just as staff members were working to shift priorities and address the outstanding metadata issues. To confirm the number of records that needed to be excluded through the ingest filters in the metadata pipeline, staff were able to prioritize this metadata cleanup project, gain a deeper understanding of the issues involved, and plan for making the metadata uniform with other metadata for electronic resources. The time saved from ongoing maintenance of DWS record loads allows staff to focus on metadata cleanup and refining the workflow for managing discovery of all US documents. The way in which the shared DWS records addressed such an immediate need hastened the transformation from a willingness to try the shared record model to adoption of it and taking advantage of all its benefits.

A later project to share EEBO records further underscored how beneficial the Shared Records model is to timeliness of access and discovery. The proposal to share EEBO records came at a time when past record loading workflows were transitioning across departments, and the budget for automated authority control was being examined and restructured. Once again, Duke received the benefits of a shared records project supporting timely discovery that also allowed for a restructuring of workflows and budgets. By the time the EEBO project was complete, shared records became an ingrained part of workflow planning for facilitating timely discovery of electronic resources. The most recent shared records project, which facilitates discovery of consortium-held Oxford University Press/University Press Scholarship Online eBooks, is managed at DUL and provides a concrete example in which changes in perspective transformed local perceptions of the ILS and management of access and discovery.

As previously noted, Stalberg and Cronin identify “the extent to which data-creation processes facilitate timeliness in resource availability” as a measure of metadata value.²⁶ Since the inception of the Shared Records project, TRLN has observed improvements in metadata timeliness for several collections. The ICPSR and OUP E-book projects provide good examples. The ICPSR metadata are currently updated in our indexes weekly. Before implementation, these records were updated once per year. The schedule for metadata processing of OUP e-books at DUL is driven by objectives of the broader e-book pilot, decreasing the time between e-resource availability and discoverability. The automation of all metadata harvesting and indexing processes also improves timeliness of metadata availability.

The TRLN Shared Records project has delivered efficiencies in throughput as well. That a single institution can take responsibility for managing metadata on behalf of two or three partner libraries delivers efficiencies immediately through the elimination of workflows. As seen in table 2, fourteen discrete local cataloging workflows have been eliminated or avoided in the TRLN libraries.

The elimination of duplicate workflows has created time and energy at each institution for other projects and new initiatives. For instance, before the implementation of the Shared Records program, cataloging staff at NCSU, Duke, and NCCU spent time maintaining URLs in bibliographic records for government documents. UNC-Chapel Hill, as regional depository and host institution for the TRLN DWS collection, has taken responsibility for these activities, freeing up staff at the other institutions for different activities. Duke, for instance, reallocated technical services staff to work on deferred metadata management activities related to government documents.

The TRLN Shared Records program allows the TRLN libraries to share the costs of licensing records and authority processing. For instance, a consortium license for DWS reduced NCSU’s annual Marcive record subscription costs by $650. Centralized processing yields other savings as well. Before implementing the Shared Records Program, the consortium’s libraries paid Marcive to “set holdings” at OCLC for DWS titles. This process has been centralized and UNC-Chapel Hill staff use OCLC batch processes and a group profile to set holdings, virtually eliminating this expense. As of September 2012, 221,131 records were in the Shared Records program, which removed 514,459 records from authority control at the four institutions eliminating associated processing costs.