Revitalizing Scholarly Reference for Digital Research Requires a Redoubled Commitment to Quality and Community

Damon Zucca

Damon Zucca (Damon.Zucca@oup.com) is Scholarly Reference Publisher, Oxford University Press, New York, New York.

Basic research faces significant challenges due to the staggering proliferation of information and misinformation online. High-quality reference works can help to address this challenge. In this paper, the author uses the example of the scholarly reference program at Oxford University Press to show how reference works can support emerging digital research needs and to argue that close collaborative engagement from the academic community is essential for ensuring the quality and authority online scholarly reference resources.

We know the story well. Mass digitization has led to an abundance of easily accessed, potentially useful sources of information, which in turn has transformed research habits at all levels. The overall impact is positive, but there are new challenges. The research environment has gone from a state of scarcity, where it took considerable effort even to verify basic facts, to an information-rich world where the most significant challenge researchers face is sorting through and making sense of all of those sources. The truth is out there, but the complexity of the retrieval environment has, paradoxically, made it harder to find accurate information because it is more difficult to evaluate the trustworthiness of sources and claims.

Oxford University Press (OUP) has reoriented its scholarly reference program in response to these shifts so as to better address the evolving needs and expectations of our core student and faculty audience. Yet despite significant changes in approach, certain aspects of the publishing process at OUP remain consistent: namely, an unwavering commitment to the authority and reliability of our sources and a close collaborative relationship with the academic community as the means for ensuring quality.

How Has the Scholarly Reference Program at OUP Changed?

OUP’s scholarly reference program is pivoting away from quick-look-up factual references to concentrate on developing resources that provide context, insight, and interpretation. Sort-and-sift technologies, such as abstract and indexing services, help a researcher refine their options, which is crucial in a field one knows well, but this refining is less helpful in unfamiliar areas of research. This is the gap we aim to address. We believe a recognized expert’s unique synthesis of an area of scholarship they know well is the most reliable and most efficient means for providing a basic understanding of a topic. If you bring together many experts in one publication, it is possible to establish a foundational reference work that anchors knowledge in a given discipline.

Accordingly, we invite leading scholars to distill what they know in various ways so as to provide a researcher with an efficient pathway into an unfamiliar topic: whether it is an overview article as published in the Oxford Research Encyclopedias, a selective guide to the most important literature as published in Oxford Bibliographies, or a critical review essay published in an Oxford Handbook.

We publish long-form reference content so that facts and debates are provided with context. This sort of contextual guidance helps researchers to slice through an overabundance of information by providing a clear point of entry, a basic lay of the land, and a path for further research. The aim of the reference program at OUP is to create these points of entry and guidance.

While OUP continues to publish scholarly reference in multiple formats, our basic orientation is that of a digital publisher. This means we are alert to the multiple ways people might use and discover the content we publish, we recognize our authors’ expectations for speed of publication, we recognize our users’ expectations around updating and currency, and we look for ways to open up our publishing process to engage users for advice, feedback, and other forms of involvement.

How Do We Maintain Quality?

The tried-and-true methods OUP uses to ensure the quality and reliability of its scholarly reference works have not changed, even as the publishing process itself has evolved. Quality control begins during the initial planning phase. Press editors work closely with academic advisors to identify fields that would benefit from reference works, and then to recruit an editor-in-chief to direct the initial development of the work. We are looking for someone with broad knowledge of the field, a stellar reputation, and enthusiasm for the prospect of leading a large-scale collaborative research and publishing project. OUP works with the prospective editor-in-chief on a proposal that is sent to others in the field for comment and then refined.

Once the project begins, the first step by the editor-in-chief is to recruit an editorial board of subject specialists. For continuously updated, online reference works, the role of editorial boards is ongoing, and advisors sign on for terms of three years or more. As research networks are increasingly international, so too is membership on editorial boards. For instance, across the Oxford Research Encyclopedia program, we have advisors hailing from more than eighty countries. Editorial boards have three tasks: develop a list of potential articles in their area of specialty, recommend authors to write those articles, and oversee the peer-review process.

OUP’s reference publications are by invitation only. When we receive unsolicited requests to write, those potential authors are approved by an editorial board, which is now happing with greater frequency, as more of our publications are discoverable online. When a finished article is submitted for publication, it undergoes multiple rounds of peer review. The standard process is for one external reader to review the essay, and then an editorial board member will review both the essay and the external reader’s report. The author is then given an opportunity to revise their work before resubmitting it for copyediting.

For online reference works, all authors are sent an annual reminder starting the first year after the publication of their essay as a way to check if their article needs updating. Editorial board members review all articles in their area every three years to confirm whether an update is needed. For long-standing reference works, such as Grove Music, where some articles were written quite a long time ago, a co-author may be brought on to make adjustments to the bibliography or to the article itself, and this new person’s involvement is credited on the website. Small updates are made directly and more substantial changes are reviewed and copyedited.

Community Organizers

Speaking on the theme of reference publishing at the 2017 Charleston Conference, Uri Nodelman, a senior editor for the Stanford Encyclopedia of Philosophy, which is run out of the Philosophy Department at Stanford University, referred to the work of his editorial team as “community organizing.” He explained that the purpose of the Stanford Encyclopedia of Philosophy was to organize a community of professional scholars to create and maintain an up-to-date reference work for themselves, colleagues, students, and the general public. I found this to be an apt description of the publisher’s role on any major reference work, including those we produce at OUP.

Creating and maintaining any sizable, foundational reference work is a highly social undertaking. The most important contribution the reference publisher makes is, in my experience, in mobilizing a community of experts around the collective endeavor. What scholarly reference publishers bring is capital to fund the work, experience to help organize the project, and tools and infrastructure to manage the publishing process. However, what separates great reference works—those that have an enduring impact—is that, in addition, they have the support of the academic community whose scholarship and history is represented by the publication. For this reason, OUP strives to establish reference works that academic communities can rally around, argue with, contribute to, and feel proud of.

To a significant degree, quality control over reference works is dependent on the active support of a community of experts because the reliability of the content starts with the level of engagement from advisors who plan article topics and recommend authors, it carries through to the level of engagement of the authors when they write for the work, and it is what makes a peer-review process more meaningful than a simple up or down vote.

What Sets Scholarly Reference Apart from Sources without Formal Vetting?

In the current information ecosystem, it has become increasingly important that facts are presented in context so that they may be evaluated and understood. This context, because it depends on the considered judgments of experts, is difficult to produce, but it is also valuable for research. Knowing who has produced a piece of content—both its author and the advisors who direct the containing work—is essential. Because it is the source of quality authors, quality vetting, and overall direction, the active participation of a community of experts sets scholarly reference apart from the masses of unvetted material found online. Without involvement by specialists in the planning and producing of content, it is difficult—if not impossible—for a reference work to remain relevant, useful, and authoritative. This is the key element behind the trustworthiness and authority of OUP’s scholarly reference publications.