This issue of Library Technology Reports (LTR) builds on previous work in this series, including the LTR issue by Coyle on the Semantic Web as well as Witt’s issue on the Open Archives Initiative Object Reuse and Exchange (OAI-ORE) standard, and it touches on topics referenced in Breeding’s 2009 issue on web services and SOA as well as Nagy’s 2011 case study analysis of library uses of next-generation discovery platforms.¹ In fact, this issue saw its first iteration in 2007, when Eden discussed metadata and information organization issues in libraries.² In that issue, Eden explored current metadata issues and asked what information organization might look like in the coming years. At the time, Library 2.0 and web services were newly emerging terms in library and information science literature, and while there was a vision for what library metadata and information systems might become, there were not many examples in the field.

Much has happened between these LTR issues, not the least of which has been the funding and creation of new national and international organizations whose goal is to bring together and publish the collections of cultural heritage and memory institutions. Since 2007, LAM (libraries, archives, and museums) communities have developed new cataloging and archival processing frameworks (e.g., RDA, DACS, and CCO) and are keenly interested in exploring the impact of new information systems on user needs. In the library world, this discussion has led to the BIBFRAME initiative and a focused effort to implement the Resource Description and Access (RDA) specification. In museum communities, the International Council of Museums has updated the CIDOC Conceptual Reference Model (CIDOC-CRM) as well as the Cataloging Cultural Objects (CCO) specification. In archives, standards like Encoded Archival Description (EAD), Describing Archives: A Content Standard (DACS), and Encoded Archival Context (EAC) have taken hold, and new information systems like ArchivesSpace are emerging to serve archival metadata and object management needs.

ArchivesSpace

www.archivesspace.org

The definition of new standards and development of systems across LAM and publishing communities are focused on defining techniques, standards, systems, and services that meet the changing information sources and needs of LAM patrons. The patron’s information requirements are grounded both in a need for physical and digital information artifacts and also in a community of practice in which making connections among information sources is as important as discovering information resources. While part of the LAM profession focuses on literacy and information engagement issues, the metadata community typically focuses on exploring how information systems and structures support these needs and enable cross-community and cross-repository access and aggregation.

As community needs change and metadata standards and systems are developed to meet these needs, LAM institutions are revisiting common questions of metadata exchange, migration, interoperability, scale, sustainability, and value. This issue of LTR focuses on these questions as we explore three continuing efforts in the LAM metadata community: the Library of Congress BIBFRAME initiative, the Europeana digital library, and the Digital Public Library of America (DPLA). The metadata systems of these efforts were chosen for exploration because they represent major initiatives in LAM metadata and because each community is interested in developing and delivering metadata-rich solutions that have a national and international impact.

In order to better explore the general direction of metadata research and practice in LAM communities, this issue begins by examining the broad direction of metadata development and research in chapter 1, continues with a deep dive into the underlying standards and building blocks of metadata systems in chapter 2, and broadens back out with case study explorations in chapter 3. In chapter 4, we explore the broad questions of metadata research and development within the context of our case study analyses.

The MARC standard that has served libraries well for the last forty years includes a robust metadata schema, an efficient exchange standard, and a detailed encoding and data storage system.³ While we often talk about MARC in bibliographic terms, it is a much wider standard that includes bibliographic description, storage of holdings data (MARC21 Format for Holdings Data—MFHD), storage of authorities and classification data, and storage of community data.⁴ These MARC formats have provided libraries with a detailed and interconnected ecosystem within which they could create, share, and validate records and authorities.

In addition to developing MARC to store bibliographic and other library-specific data, LAM institutions also explored the use of MARC in archival and museum settings. These uses included MARC Format for Archival and Manuscripts Control (MARC AMC); MARC Archives, Personal Papers, and Manuscripts (APPM); and MARC for Visual Materials (MARC VM). While MARC flourished in library settings, these additional implementations found mixed success in archival and museum contexts, and in the last decade other cataloging models and dissemination methods have superseded these efforts.⁵ The museum community, for example, has found value in a specialized cataloging protocol, Cataloging Cultural Objects (CCO),⁶ and the archival community has found value in Encoded Archival Description (EAD), Metadata Encoding and Transmission Standard (METS), and the cataloging specification Describing Archives, A Content Standard (DACS).

While these new standards emerged based on a need to accommodate new types of resources, one issue that has persistently confounded metadata work in the digital age is the complications associated with working with digital surrogates of physical resources.⁷ This is not for lack of appropriate data models and cataloging approaches but rather due to the natural tension between creating highly accurate metadata and creating metadata efficiently. In the Dublin Core (DC) community, this issue is discussed as the “one-to-one principle.” For example, in cataloging a digital surrogate in DC, it is easy to blur the line between a physical object and its digital surrogate by using the date of a historic photo in the dc:date field while providing the URL to the digital surrogate in the dc:identifer field. While the Qualified Dublin Core (QDC) standard introduces new properties that help differentiate metadata about physical objects from metadata about their digital surrogates, the issues of specificity, cost, and value in relation to metadata work means that adopters may find themselves, with the best interests of their institutions at heart, making choices out of sync with the standard.

A second important area of research and practice in metadata is that surrounding metadata provenance and version control. As metadata is increasingly shared, aggregated, repurposed, and reused, understanding where the metadata came from and how it was intended to be used is important in ensuring accurate disambiguation, deduplication, and rights tracking. As we will see in our case study exploration, being able to track metadata sources at a very detailed level is important for resolving discrepancies in description as well.

As libraries have moved through iterations of discovery platforms (faceted discovery, federated search, web-scale discovery), issues of metadata schema harmonization, system scale and performance, and metadata quality guided the developmental direction of systems. Nagy’s LTR issue used a case study approach to explore how libraries had addressed these issues.⁸ In addition to the user interactivity foundation of next-generation catalogs, one of the key features that differentiated faceted catalogs from more traditional integrated library system catalogs was the use of a faceted search platform. In recent years, open-source tools like VuFind, DSpace, and Blacklight have capitalized on the speed and flexibility of the Apache Solr indexing platform to deliver lightweight and fast discovery services.

VuFind

http://vufind.org

DSpace

www.dspace.org

Blacklight

http://projectblacklight.org

Apache Solr

http://lucene.apache.org/solr

Two key technologies facilitated the development of these tools. The first was the use of industry-standard ways of recording data. Also known as “serialization,” the shift away from MARC 21 storage formats to XML-based formats made it much easier to bring together bibliographic data and data from archives and museums, which was more likely to be in an XML-based format. MARC-to-XML converters like SolrMarc were the building blocks that made this transition possible. The second key technology that enabled widespread experimentation with new discovery platforms was the adoption of application programming interface (API) techniques in system design. APIs are an important design feature of Solr that allow modular development of metadata management and discovery systems.

SolrMarc

http://code.google.com/p/solrmarc

In the last few years, these discovery services have begun shifting from localized implementations of catalogs to cloud-based services that include not only local holdings but also journal database and full-text indexing. The deployment of discovery environments in the cloud requires new techniques for metadata management as well as new data structures and database platforms that perform well at this higher scale. Some examples of these environments include open-source platforms like Open Library and the Kuali Open Library Environment, community services like LibraryThing, and enterprise-level web-scale library service environments.

Open Library

http://openlibrary.org

Kuali Open Library Environment

www.kuali.org/ole

LibraryThing

http://librarything.com

While libraries are forging ahead with these new platforms, there is still a pressing need for a continued fundamental restructuring of the metadata models and records that serve as the foundation for these systems. While the MARC metadata standard is generally perceived as a standard whose time has passed,⁹ it has also been credited as a standard that formed the foundation of computerized resource description and fostered widespread standardization of bibliographic description.¹⁰ In rethinking approaches to bibliographic metadata, Miller and his colleagues point to three core functions of MARC that need to be replicated in new standards and systems: the capture of data about the “intellectual essence of a work,” the capture of data related to the “actual instance of the work,” and the capture of data to uniquely identify (e.g., LCCN, ISSN, ISBN) and situate (e.g., related works, series statements) a work in a larger body of knowledge.¹¹

This interest in capitalizing on rich and complex metadata while also developing new models that are naturally interoperable with other standards is a common theme in metadata discussion. For example, although there is a history of rich descriptive practice in archival institutions, the practice has also been shown to have difficulty scaling to meet the addition of new archival materials, making it more difficult for archives to serve patron needs.¹²

Greene and Meissner’s focus on the user is another common theme in metadata research. This focus was expanded from the notions in the Paris Principles during the definition of the Functional Requirements for Bibliographic Records (FRBR) and IFLA Statement of International Cataloguing Principles models.¹³ While the bibliographic community largely supported FRBR, it also has found implementing the model difficult, in part because of metadata quality issues, but also because of the inherent difficulty associated with shifting from one model for representing resources to another.¹⁴ In fact, fifteen years after its definition, the model has yet to be fully adopted, and as new specifications like BIBFRAME are emerging, the library community may be signaling an interest in moving on without ever having realized this goal. Finding this balance between what is achievable (e.g., “more product”) and the interest in preserving the granularity and specificity of metadata has been at the center of discussions of cost, value, and relevance.

The RDA community, for example, has spent considerable effort in building out a new series of cataloging rules geared toward accommodating an increasingly complex world of information objects. This complexity has been the source of debate in the library community¹⁵ and continues to be a popular topic on cataloging discussion lists. In April 2013, for example, a poster to the AUTOCAT Listserv suggested a “full stop” in the implementation of RDA given the “piecemeal” approaches to implementation that had been discussed previously.¹⁶ In ten days, this post garnered seventy-two responses that debated the core issues of RDA adoption. The interest in these topics has also been seen in the enthusiastic reception of the DPLA release and the detailed discussions of the still-developing BIBFRAME specification.

In this section, we have touched on metadata-related issues of web integration, adherence to industry standards, metadata management at web scale, the need to meet user needs, and the need to explore new solutions and develop new tools. At the same time, age-old issues of metadata quality, richness, granularity, specificity, cost, value, and sustainability are part of the critical framework with which we evaluate each new schema and technology.

The broad direction of current research in the metadata world involves work in the design and application of linked open data (LOD) and linked open vocabularies (LOV). New communities have been formed to discuss these new structures, and the LAM world has reached out to existing communities like the World Wide Web Consortium (W3C) to collaborate in the development of the underlying specifications of LOD and LOV. In the remainder of this chapter and throughout the rest of this issue, we explore LOD solutions and consider how both the building blocks of LOD and the specifications deployed on top of them answer the technology, design, application, use, value, and sustainability questions that are central in all metadata research.

In chapter 2, we will explore the building blocks of LOD systems, including the general rules for LOD, a general data model for LOD (the Resource Description Framework), and the design and application of vocabularies and ontologies. In preparation for that discussion, we need to have an understanding of the building blocks of metadata itself. Metadata is not just a definition of fields into which content is placed or a way of encoding (i.e., serializing) that information. Metadata is comprised of a data model, rules for how content is formatted, rules for how content is represented, rules for how content is stored, and rules for how content is exchanged. Regardless of the technology or philosophical foundation of a metadata platform, these building blocks play key roles in defining how a metadata system works and what it is capable of doing. These constructs are discussed in depth by Elings and Waibel as data fields and structure, data content and values, data formats, and data exchange.¹⁷

Table 1.1, adapted from Elings and Waibel’s work, shows the connection between four building blocks of metadata systems and some example standards for each component. The third column suggests alternative terminology that is commonly used in literature to discuss these types of systems. A fifth building block, labeled data model, is also listed in this table. While Elings and Waibel connected data model with data structure, data models in an LOD context deserve separate consideration. The data model that is the focus of LOD is the Resource Description Framework (RDF).

To enable us to start on common ground, table 1.2 unifies the content from table 1.1 and defines each of the concepts in this aggregated model. While a detailed exploration of these concepts is beyond the scope of this issue, the interdependencies and relationships among them should become apparent in chapters 2 and 3.

There are other ways of parsing metadata schemas, including the use of types of roles (e.g., descriptive, administrative, technical)¹⁸ or the use of metadata schema features.¹⁹ In this issue, we rely on these building blocks so that we can understand the technical components of metadata rather than the specific functions and roles of the various schemas and vocabulary.

This issue of LTR features a number of figures and tables based on data harvested from the linked data services discussed in chapters 2 and 3. Because of the complexity inherent in RDF-based data, you will find that these figures often contain a small snapshot of data from a series of RDF statements and may be difficult to read. In order to make it possible for readers to replicate these figures as needed and access entire example records from linked data services, all of the appendixes, figures, and tables and the files used to generate them have been checked into a GitHub repository. In addition, this issue mentions data dissemination services including application programming interfaces (APIs), data visualization tools including Gephi, and data querying tools (SPARQL). Rather than documenting the process for accessing and using these tools in this issue, tutorials have been created and are accessible in the GitHub repository. More information about how to access and make use of this data is available on the GitHub site.

GitHub LTR repository

https://github.com/mitcheet/ltr

Gephi

http://gephi.org

Throughout the remainder of this issue we will use these five building blocks of metadata—data model, content rules, metadata schema, data serialization, and data exchange—as a framework with which we can explore LOD as well as our case studies. In chapter 2, we focus on LOD and take a fairly technical look at how linked data works. In chapter 3, we explore the metadata aspects of the BIBFRAME initiative, the Europeana digital library, and the Digital Public Library of America using this framework and consider the similarities and differences among these services. In chapter 4, we take a broader view of the metadata questions and cross-domain issues we discussed in this chapter and explore where the LAM community is with these issues today. In doing so, we may find it difficult to maintain firm boundaries among these five components of metadata or between key issues and opportunities.