Giving Credit: How Well Do Librarians Cite and Quote Their Sources?

Peter Genzinger and Deborah Wills

Peter Genzinger (pgenzinger@wlu.ca) and Deborah Wills (dwills@wlu.ca) are Reference/Collections Librarians at Wilfrid Laurier University Library, Waterloo, Ontario, Canada.

The practice of citing references is integral to scholarship. This paper focuses on three prominent journals for library science: College and Research Libraries, Library Resources and Technical Services, and Reference and User Services Quarterly. Errors in both citations and quotations were found in all three journals, although no statistically significant differences among journals were discovered. Citation errors of less than 10 percent were found for all three journals, while in total, 30.3 percent of quotations were judged to be questionable in some way. The paper includes recommendations for authors, editors and librarians. It also recommends further study of errors in quotations, which appear more troubling than those in citations.

The practice of citing references is integral to scholarship. It allows researchers to understand the way that a discipline is constructed, building on the work of earlier scholars. In particular, cited references provide a means for tracking down the works that have informed the discipline and acknowledging the contributions of the authors of those works.

For practicing librarians, cited references provide a gold mine of possibilities. To help students find relevant resources for a paper, librarians frequently refer to these references, explaining the structure and value of the citation trail, and helping students identify and locate relevant items. At the same time, librarians have the opportunity to instil in their students a respect for the importance of accurate and appropriate citation of the works that they use for their own writing. As Rekdal points out, with respect to preventing plagiarism, a “good start is to make sure that all students are aware that academic citations are extremely important as tools for communication and documentation of knowledge, and that they need to be complete and accurate, and employed with precision, to fulfill these functions.”1

How much can we trust citations to be free of errors and quoted material to accurately portray the words and meaning of the referenced sources? As librarians, do we assume that references are accurate, and if so, is that a fair and useful assumption?

Our aims for this paper were to determine what research has been done on citation accuracy across disciplines, to perform our own study on selected journal articles in library science, to consider if current editorial policies might be having an effect on citation accuracy, and to make recommendations for authors, editors and librarians.

Literature Review

The study of reference accuracy has generally been divided into two areas: “citation accuracy,” which refers to the elements of the citations in the reference lists of articles, such as author name, article title, etc.; and “quotation accuracy,” which refers to the handling of direct quotes, paraphrases, and summaries.

For this paper, we reviewed fifty-four studies published between 1977 and 2015. Thirteen considered both citation and quotation accuracy, four focussed on quotations, and the remainder on citations alone. In addition, we consulted two systematic reviews of quotation accuracy in medical journal articles. See the appendix for a full listing of these papers.

Citation Accuracy

Since the 1970s, a considerable number of researchers have studied citation accuracy, particularly with respect to science and medical journals. Booth summarizes the findings of 36 studies of medical journals, and in their literature review, O’Connor and Kristof summarize nineteen studies, including three for library science journals, finding a combined error rate of 31 percent.2 These studies have often focused on the issue of findability: the ease with which citations could be tracked down. Some studies have also highlighted the need to give appropriate credit to authors. Booth, who surveyed the literature up to 2004, found that author errors dominated the results, while “fatal flaws,” which made the cited sources unfindable, were few in number.3

Researchers have generally organized errors by author, article title, volume, issue, year and pagination. For every journal studied, errors were found, ranging from as low as 8 percent to as high as 66.7 percent.4 Various researchers have attempted to classify citations as major—generally described as those that would hamper retrieval—or minor. However, the criteria for classification vary from study to study. For example, Lok, Chan, and Martinson consider incorrect years of publication as major errors, while Muhammed and Laskin classify them as minor.5 Several researchers count errors in author names as minor.6 In contrast, for Raja and Cooper, omitted or badly misspelled names are major errors, since they hamper the task of crediting authors for their work.7 Lopresti points out that even slight changes to authors’ names, such as the omission of a middle initial, can impede the tracing of authors through citations to their work.8

Many studies compare journals within a discipline. For example, Fenton studied four otolaryngology journals and found “the higher the impact factor for the journal, the lower the number of errors detected in its papers.”9 Wilks et al. found that articles published since 2010 in Research on Social Work Practice had significantly more accurate citations than those published earlier. In addition, articles with one author were significantly more accurate than those with multiple authors.10 A study by Asano et al. compared changes in error rates over time in the Canadian Journal of Anaesthesia. Between 1990 and 1994 there was a significant decrease in total errors from 48 percent to 22 percent. This was likely caused by a change in editorial policy requiring authors to submit the first page of each cited reference.11 Similarly Oren and Watson, who studied eight peer reviewed journals from the ophthalmic literature, found that Elsevier, the only one of the publishers to use librarians to check citation accuracy, “had the lowest averaged error rate.”12

Citation accuracy in library science journals has been studied by many researchers, beginning with Boyce and Banning in 1979.13 In a study published in 1992, Pope checked ten citations from each of ten library science journals, finding thirty errors in total.14 In a more extensive examination published in 1993 of five library science journals, Pandit found considerable differences among the journals. She reported that for both the journal with the least errors (Library Trends at 3.8 percent) and the journal with the most errors (Library Resources & Technical Services at 31.6 percent) the journal’s editorial office had a policy to check the authors’ citations.15 Benning and Speer compared library science with medical journals published in 1989, finding similar error rates between the disciplines, though they judged that errors in the library science literature were more likely to be minor.16 A study by Davies examined the 2007 citations from four high-impact information science journals. She found error rates ranging from 41.3 percent for MIS Quarterly to 49.1 percent for Information and Management.17

Quotation Accuracy

Since the 1980s, researchers have studied “quotation accuracy.” As well as tracking errors in direct quotations, they have included errors in paraphrases and summaries. Researchers have been particularly interested in errors that could mislead the reader or fail to reference a primary source. Again, research has focused mainly on science and medical journals.18

Drake et al., who studied ecology journals, chose a single statement from each selected article, for a total of 124. Of these, 54 percent were judged to be fully supported by the original source.19 Haussmann et al., studying physical geography journals, also chose single references from each selected article, for a total of 120. In 80.8 percent of the references, the cited article was found to clearly support the quotation.20 Although no studies appear to have included North American library science journals, a recent study of library science journals in Taiwan examined 622 quotations from 111 articles and found an error rate of almost 14 percent.21

Some researchers have labelled quotation errors as “failed to substantiate,” “unrelated,” and “contradicted.”22 Todd et al. and Haussmann et al. use the categories “clear support,” “no support,” “ambiguous,” and “empty” (not citing the primary source).23 De Lacey, Record, and Wade divide results into “precisely correct,” “trivial error,” “slightly misleading,” and “seriously misrepresenting.”24 Evans, Nadjari, and Burchell also include “oversimplification” and “generalization” for minor quotation errors.25 Lee and Lee, and Eichorn and Yankauer use the categories major errors (contradicted or failed to support) vs minor.26

In 2015, Jergas and Baethge published a systematic review of studies of quotation accuracy in medical journal articles. Given the differences in method between the various studies, their analysis could provide only rough estimates, but they concluded that about “one in every eight to nine references was seriously incorrect,” and approximately one fourth had errors of some kind.27

Given the relatively few previous studies of citation errors in library science journals, and the lack of quotation accuracy studies, we felt that further research was warranted. In particular, in our study we go beyond an assessment of error rates to consider the implications of such errors. We also suggest where authors, editors and librarians could place their emphasis as they check articles for accuracy and use sources in their professional work.

Method

Our approach was to first check for citation and quotation accuracy in selected library science journals, and then to consider the implications of the types of errors found and what lessons could be learned. Our guiding principles were findability of the articles, traceability of authors, and integrity of quotations.

To select the journals, we referred to a study by Judith M. Nixon which ranks core titles in library and information science. Her study identifies six top ranking journals and she also provides a combined list of twelve top journals identified by three expert opinion surveys.28 We chose the three peer reviewed journals that appear in both lists: College & Research Libraries (C&RL), Library Resources & Technical Services (LRTS) and Reference & User Services Quarterly (RUSQ).

We focused on the 2013 calendar year, selecting “Articles” from C&RL, “Articles” and “Reports” from LRTS and “Features” from RUSQ. Like many other researchers for the topic of citation accuracy, we chose to examine only references to journal articles, a source type used by all the authors concerned and one that is generally findable and searchable online. We also limited our focus to articles in English. This resulted in a sample of 426 from C&RL, 228 from LRTS and 189 from RUSQ. In addition, we contacted the editors of the three journals to learn their policies for checking citations.

For our study, one researcher started with C&RL and the other began with LRTS and RUSQ. Each then checked the other’s work, discussing discrepancies as needed.

Citation Checking

To choose our sample for citation checking, we used the online Sample Size Calculator from Creative Research Systems, selecting a confidence level of 95 percent and an interval width of 5 percentage points.29 This resulted in sample sizes of 202 for C&RL, 143 for LRTS and 127 for RUSQ. We then used a random sequence generator to select the sample citations. We limited our selection to articles available through our local library system to avoid overloading our interlibrary loan staff. When an article proved unavailable, we maintained our sample size by returning to the random sequence to select an additional article. The discarded articles totalled fourteen, representing 3 percent of the sample, and a variety of lesser-known library and archival journals.

Whenever possible, each citation was checked against the information from the actual journal article; when the information included with the article was incomplete, we checked further. For example, we looked for the journal’s table of contents or a database that included the journal. We divided errors into author, article title, journal title, publication year, volume, issue, and pages. If only an author’s initials were used, or if an author’s middle initial was missing, we did not consider these omissions as errors, so long as the information given was accurate. In cases where issue numbers were missing, but the journal used continuous pagination, we judged the element “not needed” and did not record an error.30 For pagination, citations were considered correct if they either included the full page range or one or more specific, relevant pages. If pagination was not available in the item cited, for example if the article was published only on the web, the omission was marked “not applicable.” When calculating error percentages, we adjusted the totals for each element to exclude those judged “not needed” or “not applicable.”

Given the inconsistent categorizations used in previous research, we did not attempt to label errors as major or minor, but all errors and omissions, including missing author initials, were noted. We conducted a separate test of findability after the error check. Previous researchers have often measured the findability of articles with erroneous citations by searching for them in proprietary databases. Lopresti, for example, discovered that “almost one in five of the studied journal errors had the potential of defeating a search in Web of Science.”31 Given the near ubiquitous use of Google, we decided to use Google as our finding aid. In addition, while working with library users, we have noticed their tendency to copy and paste the article title into Google, as they work to track down a citation. If they aren’t successful, they may choose to add more words from the citation into a new search. We therefore began by searching Google for the full article title, as listed in the citation. In cases where an author or journal title was incorrect, a second search was conducted, adding in this incorrect information. The search was judged successful if the correct item was easily identified in the first page of results. Research by Asher, Duke, and Wilson suggests that students, at least, rarely look beyond this page.32

Quotation Checking

Using recent studies by Drake and Haussmann as models for sample size, two references to journal articles were randomly selected from each article in the 2013 issues of our three journals, making a total of 122.33 We then obtained the cited articles and checked them against the quoted statements, dividing the work between us as described for citation checking. Following the examples of Todd and Haussmann, we classified quotations as “clear support,” meaning they provided unequivocal evidence, “no support,” meaning they did not substantiate the statement in any way, “ambiguous,” meaning they lacked a clear connection to the statement, or “empty,” meaning they cited secondary sources rather than the original source.34 We also noted any errors that were difficult to classify, and tracked the incorrect use of quotation marks. We did not attempt to judge the truth of any of the statements.

The issue of findability was also relevant for quotation checking, since we needed to pinpoint the appropriate parts of the referenced articles. We noted instances when this task was impeded.

Results

Citation Accuracy

Our total sample size was 472. Of these references, 366 (77.5 percent) were completely error free. We found a total of 122 errors, for an overall error rate of 25.8 percent. The total errors noted for the three journals, respectively, were: 58 (28.7 percent) for C&RL, 31 (21.7 percent) for LRTS, and 33 (23.4 percent) for RUSQ. The combined error rates for individual elements of the three journals were, from highest to lowest, the article title element (6.1 percent), issue (5.6 percent), pages (5.4 percent), author (5.1 percent), journal title (1.9 percent), volume (1.5 percent), and year (1.1 percent) (see figure 1).

We found error rates of less than 10 percent for all individual elements across all three journals. Error rates for individual elements were similar. The highest error rate by element was 7.7 percent for article title (LRTS) and the lowest was 0 percent in both year and volume (LRTS). Error rates for individual elements are listed by journal in table 1.

A chi-squared test was used to check for differences among the overall error rates for the three journals. No statistically significant difference was found. The chi-squared test with 2 degrees of freedom and a discrepancy measure of 1.99 gave a p value of 0.037. This p value is far from indicating statistical significance (usually taken as 0.05).

When we contacted the editors for the three journals, we learned that for LRTS, authors have responsibility for citation accuracy, but the editor checks all citations herself, and many editorial review board members also check citations when they review papers. For RUSQ, authors have sole responsibility for checking citations, though editors and copy editors note incomplete citations and ask for corrections. C&RL relies on the diligence of their mostly librarian authors to check their own citations. Given the lack of statistically significant differences among the three journals, it appears that these varied approaches to citation checking are not affecting error rates.

Findability of Citations

Searching for full article titles in Google resulted in correct identification of nearly all the items, and all were eventually found using the information provided in the citation. For example, in one case, the subtitle was missing from the citation. The resulting title, “Purchasing E-Books in Libraries,” proved too generic for a successful title search, but the item was findable through the journal title.

In some cases the item was findable in the top ten Google results, but hidden from view. For example, in a case where only the last word was wrongly cited, the only correct hit was for a prepublication version of the article. A search on the correct title led immediately to the published version via Project Muse.

In another case, words were missing from the beginning of the title. The item appeared as the first result in a Google search, but since the missing words were the only ones presented for the title, the item could have easily been overlooked. For another title, the citing author added extra words at the beginning, which did not show up in the search results. One item that took considerable time to locate was published simultaneously, with different titles, as a book chapter and a journal article. The citation paired the chapter title from the book with the title of the journal.

Article findability was also impeded when incorrectly spelled author names were included in a search. In four cases, searching by the article title alone identified the item as the top Google result. However, adding the wrongly spelled author name, even when the error was a single missing letter, obscured the correct result. In each case, the search pulled up various references to the required article, but most included the error in the author’s name.

Traceability of Authors

While the number of errors in the author element of the citations was small, we found a total of twenty-seven cases, from the three journals, in which initials, rather than full names were used, or where middle initials were missing. These omissions were not counted as errors, and did not impede findability of the article, but they might affect a reader’s ability to trace specific authors through their citations.

Quotation Accuracy

We checked a total of 122 quotations from the three journals. We judged that clear support for the original statements was provided in 69.7 percent of cases, no support in 9.8 percent of cases, and ambiguous support in 18.9 percent of cases. 1.6 percent of cases were judged as “empty.” While the three journals showed a similar level for clear support (66.7 percent to 72.4 percent), there was more disparity across journals for no support (6.9 percent to 20.7 percent) and ambiguous support (3.5 percent to 20.7 percent). In total, 30.3 percent of quotations were judged to be questionable in some way (see table 2).

A chi-squared test was used to check for differences in the overall error rates among the three journals. No statistically significant difference was found. The chi-squared test with 6 degrees of freedom and a discrepancy measure of 6.54 gave a p value of 0.45. As with the test for citation accuracy, this p value is also far from indicating statistical significance.

As we worked to check the accuracy of quotations, we were challenged to precisely categorize the errors we found. In contrast to the study of citation accuracy, quotation accuracy seems a more qualitative exercise, somewhat open to interpretation. As Luo et al. point out, “Quotation errors are usually harder to characterize and can be subjective.”35 We found a variety of cases that proved difficult to classify as either “no support” or “ambiguous”; some provided examples of overgeneralization or simplification, along with partial support, distortion, or insufficient context. In addition, there were various errors that did not affect the meaning of the quotation, including missing or inexact quotation marks, incorrect attribution, and wrong pages for direct quotations.

We found several examples where overgeneralization was used to support claims. One article claimed a high level of prestige for a certain type of journal based only on comments by two individuals in the referenced article. In another case, a single factor was highlighted to support an assertion, ignoring other factors listed by the referenced article. In some cases, a slight change of emphasis made a significant difference in meaning. For example, one article claimed that no changes need to be made to catalog records of a certain type; the referenced article called for “minimal” modification of these records. Another tendency was to read more into a statement than was warranted. For example, one article claimed that certain journals are read for current awareness; the referenced article did not state any reasons why these journals might be read.

In some cases, the quality of writing challenged us to judge the accuracy of the quotation. For example, one author cited his own previous work, but with such vague wording that we struggled to judge the accuracy of the quotation.

Direct quotes, within quotation marks, were nearly always accurate, though in many instances, quotation marks were not used when they should have been. In one case, only a partial quotation, missing a key term, was used. In another case, a quotation was attributed to the wrong person.

Concerning the issue of findability, whenever precise page numbers were lacking, it became difficult and time consuming to track down information. For example, one article claimed that a study showed a “doubling in usage” of ebooks. We read the long, referenced article very carefully, but could not find this information. Surprisingly, we did not find any studies on quotation accuracy that recommended using page numbers for quotations, though Fenton reported, “Fifty per cent of the book references could not be verified, as chapters or page numbers were not supplied.”36

Without the inclusion of page numbers, paraphrases proved more challenging to verify than direct quotes. Even when it is possible to search a source online, it is difficult to guess what terms to use. An article by Rekdal provides compelling arguments for the use of page numbers. He comments, “Leaving out the page number when it could have helped the reader track down the source text puts a roadblock in the path of the basic driving forces of scientific development: the production of cumulative knowledge and verification.”37

When studying the accuracy of quotations, it is only possible to address the information that is specifically referenced. We did find some suggestions of omitted references. For example, we noted a case where a study was described but no reference was provided.

Given all the ambiguities uncovered by this exercise, we concluded that it is highly problematic to fully judge how accurately and appropriately referenced sources are used. Our error totals should therefore be viewed as approximations rather than precise findings.

Discussion

Limitations

The findings of this study are limited to the journals and timeframe selected; it is not possible to generalize beyond this. In addition, given the variety of errors encountered when checking for quotation accuracy, the sample size may have proved too small to provide a full picture of how sources were handled.

Citation Accuracy

Compared with findings from other studies, citation error rates are reasonably low for the three journals tested. The overall rate of 25.8 percent places the results of this study toward the low end of the 36 published studies listed by Booth in 2004, who writes: “there is a clear trend for between 25% and 40% of references to be inaccurate.”38 Results are also low compared with those found by Davies. In her study of library and information science journals, she found that 45.3 percent of references had errors.39

When we consider the issues of findability and author tracking, certain elements of the citation gain importance. For findability, accurate titles are key. In addition, errors of omission are often less important than errors of commission, since adding erroneous information into a search can obscure correct results. For tracking authors, even the smallest error in the author’s name can be problematic.

Quotation Accuracy

Compared with other studies of quotation accuracy in science and medical journals, this study finds a relatively low percentage of cases providing clear support (see table 3). While it is not possible to judge the causes of errors, various possibilities can be suggested. For example, errors may represent a desire to bolster a given point of view, or may be caused by carelessness, haste, or a failure to read or understand the original article. It also seems possible that in their struggle to transform words from an original document into an appropriate paraphrase, some authors either leave out essential terms or substitute words that convey a different meaning.

Conclusion

Our findings suggest that citation errors are a continuing issue in library science journals. However, relatively few impede findability of the original source or traceability of authors. Quotation errors seem to present more complex and potentially more troubling problems. According to George and Robbins, quotation errors are “more important because they erroneously give credence to the authors’ assertions.”40 These errors are of concern because they deal with the substance of the research.

Although this study has not found that editorial policies have a significant effect on reference accuracy, editors may choose to do some selective checking. We would advise them to decide which elements they consider most essential for references and how best to use their limited time and resources to check for accuracy. For citations, given the ease of finding most articles in Google, it may be worth a quick Google search to ensure findability and correctness of author names. If only the referring article shows up in the results, this may indicate an error.

For quotations, editors may wish to spot check the accuracy of paraphrases, especially for very general statements. It seems less important to check the words within quotation marks and more important to ensure that precise page numbers are given. This eases the task of finding the information both for the editor and reader. It is also possible that the process of providing a page number will encourage authors to double check the accuracy of their work. (As writers of this paper, we benefited from rechecking our own page numbers.) Requiring authors to submit copies of the specific pages could further encourage accuracy, as well as easing the editor’s task. Davies comments that requiring authors to submit the first pages of the articles they reference would at least ensure that they had obtained the articles.41

For authors, we suggest they take time to ensure that references to author names are complete and correct, and that accurate page numbers are provided for quotations. To improve citation accuracy, they may wish to start with the citation information provided by journals. Many journals also provide a way to download citations in RIS format and import the files into citation management software packages.

Given the trend toward interdisciplinary research, it will likely become increasingly challenging, and increasingly important, to differentiate researchers with similar names. As librarian authors, we should be mindful of the principle of authority control that establishes recognized formats for authors’ names and be sure that full names, as listed on the sources, are always used. This approach is consistent with Chicago Style, used by many library science journals including the three in this study.

Authors who struggle to write an accurate and appropriate paraphrase may find that a direct quote is a safer choice. Not all statements easily lend themselves to rewording.

For practicing librarians, these findings can provide an object lesson that even professionals sometimes err. We should remember to encourage users to find original sources whenever possible. In particular, as we work with students, who are rushing to find that last source to support their arguments, we should remind them to read carefully and report accurately on what they read. Sources that disagree with their thesis are not only acceptable but essential for a complete and accurate description of a field of research.

Future Research

Most research in this area has focused on medical science and its various subdisciplines. Aside from economics, social work, and library and information science, we found no studies that investigated citation or quotation accuracy in disciplines related to the arts and humanities or the social sciences. Future research could consider these areas to provide a fuller picture across the academic spectrum. In addition, given our findings that citation errors rarely impede article retrieval, and that errors in quotations and paraphrases seem more troubling, we wonder if future research should focus more on quotation accuracy. In the discipline of library and information science, a larger-scale multiyear, multi-journal study of quotation accuracy may prove insightful.

Jergas and Baethge point to the need for future research into the extent to which false quotations affect a citing paper’s claim.42 We concur, with the additional proviso that such research be broadened in our field to investigate, for example, instances where library practice and/or policy may have been based on a false understanding of research findings.

While many of the studies we reviewed made recommendations about how editors might improve the accuracy of citations and quotations, few if any have specifically investigated the correlation between editorial policies and quotation accuracy. Future research might consider, for example, whether editorial staff conduct random quotation checks and how these might correlate with quotation accuracy rates for a particular journal.

Researchers may also wish to consider the value of comparing error rates across different publishing platforms, such as traditional scholarly journals vs the emerging open access models. Although publication practices are evolving, the need for care and accuracy in citing references will remain.

Acknowledgements

The authors extend their thanks to Professor Winston Cherry from the Department of Statistics and Actuarial Science, University of Waterloo, for his assistance with statistical calculation and interpretation.

References

1. Ole Bjørn Rekdal, “Academic Citation Practice: A Sinking Sheep?,” portal: Libraries and the Academy 14, no. 4 (2014): 571, https://doi.org/10.1353/pla.2014.0025.
2. Andrew Booth, “A Checker’s Career?,” Health Information and Libraries Journal 21, no. 4 (2004): 269–72, https://doi.org/10.1111/j.1471-1842.2004.00538.x; Lisa G. O’Connor and Cindy Kristof, “Verify Your Citations: Accuracy of Reference Citations in Twelve Business and Economics Journals,” Journal of Business & Finance Librarianship 6, no. 4 (2001): 23–40.
3. Booth, “A Checker’s Career?” 271.
4. Gerald de Lacey, Carol Record, and Jenny Wade, “How Accurate Are Quotations and References in Medical Journals?,” British Medical Journal (Clinical Research Edition) 291 (1985): 885; Vijay J. Roach, Tze K. Lau, and Warwick D. Ngan Kee, “The Quality of Citations in Major International Obstetrics and Gynecology Journals,” American Journal of Obstetrics & Gynecology 177, no. 4 (1997): 973–75.
5. Candy K. W. Lok, Matthew T. V. Chan, and Ida M. Martinson, “Risk Factors for Citation Errors in Peer-Reviewed Nursing Journals,” Journal of Advanced Nursing 34, no. 2 (April 2001): 224, http://www.ncbi.nlm.nih.gov/pubmed/11430284; Ali E. Mohammad and Daniel M. Laskin, “Citation Accuracy in the Oral and Maxillofacial Surgery Literature,” Journal of Oral and Maxillofacial Surgery 66, no. 1 (January 2008): 4, https://doi.org/10.1016/j.joms. 2007.06.682.
6. Examples include Gale Oren and Maureen Watson, “Accuracy of References in the Ophthalmic Literature,” Journal of the Medical Library Association: JMLA 97, no. 2 (April 2009): 143, https://doi.org/10.3163/1536-5050.97.2.014; A. Vargas-Origel, G. Gomez-Martinez, and M. A. Vargas-Nieto, “The Accuracy of References in Paediatric Journals,” Archives of Disease in Childhood 85, no. 6 (2001): 497, https://doi.org/10.1136/adc.85.6.497; Lok, Chan, and Martinson, “Risk Factors,” 224.
7. U. Y. Raja and J. G. Cooper, “How Accurate Are the References in Emergency Medical Journal?,” Emergency Medicine Journal: EMJ 23, no. 8 (August 2006): 625, https://doi.org/10.1136/emj.2004.022103.
8. Robert Lopresti, “Citation Accuracy in Environmental Science Journals,” Scientometrics 85, no. 3 (December 2010): 653–54, https://doi.org/10.1007/s11192-010-0293-6.
9. J. E. Fenton et al., “The Accuracy of Citation and Quotation in Otolaryngology/Head and Neck Surgery Journals,” Clinical Otolaryngology and Allied Sciences 25, no. 1 (February 2000): 41, http://www.ncbi.nlm.nih.gov/pubmed/10764235.
10. Scott E. Wilks et al., “Reference Accuracy Among Research Articles Published in Research on Social Work Practice,” Research on Social Work Practice (2016): 3, https://doi.org/10.1177/1049731515626802.
11. Migiwa Asano et al., “Improvement of the Accuracy of References in the Canadian Journal of Anaesthesia,” Canadian Journal of Anaesthesia = Journal Canadien D’anesthésie 42, no. 5 (1995): 370–72.
12. Oren and Watson, “Accuracy of References,” 144.
13. Bert R. Boyce and Carolyn Sue Banning, “Data Accuracy in Citation Studies,” RQ 18, no. 4 (1979): 349–50.
14. Nancy N. Pope, “Accuracy of References in Ten Library Science Journals,” RQ 32, no. 2 (1992): 241. This study includes the journals College & Research Libraries, Library Resources & Technical Services, and RQ, the former title for Reference & User Services Quarterly, in its journal selection.
15. Idrisa Pandit, “Citation Errors in Library Literature: A Study of Five Library Science Journals,” Library & Information Science Research 15, no. 2 (1993): 193.
16. Susan P. Benning and Susan C. Speer, “Incorrect Citations: A Comparison of Library Literature with Medical Literature,” Bulletin of the Medical Library Association 81, no. 1 (January 1993): 57.
17. Karen Davies, “Reference Accuracy in Library and Information Science Journals,” Aslib Proceedings 64, no. 4 (2012): 379, https://doi.org/10.1108/00012531211244734.
18. For a general overview of quotation accuracy studies, see Natalie Suzette Haussmann et al., “Referencing Practices in Physical Geography: How Well Do We Cite What We Write?” Progress in Physical Geography 37, no. 4 (2013): 543–49, https://doi.org/10.1177/0309133313482135; for systematic reviews that consider citation and quotation accuracy in the medical science literature, see Elizabeth Wager and Philippa Middleton, “Technical Editing of Research Reports in Biomedical Journals,” Cochrane Database of Systematic Reviews 4, MR000002 (2008), https://doi.org/10.1002/14651858.MR000002.pub3; and Hannah Jergas and Christopher Baethge, “Quotation Accuracy in Medical Journal Articles—A Systematic Review and Meta-Analysis,” PeerJ 3 (2015): e1364, https://doi.org/10.7717/peerj.1364.
19. D. C. Drake et al., “The Propagation and Dispersal of Misinformation in Ecology: Is There a Relationship between Citation Accuracy and Journal Impact Factor?” Hydrobiologia 702, no. 1 (2013): 2, https://doi.org/10.1007/s10750-012-1392-6.
20. Haussmann et al., “Referencing Practices,” 544–45.
21. Hsuan-Hung Kao and Wen-Yau Cathy Lin, “Quotation Errors in the Articles of Library and Information Science Journals in Taiwan,” Journal of Educational Media & Library Sciences 52, no. 2 (2015): 151, https://doi.org/10.6120/JoEMLS.2015.522/0009.RS.AM.
22. James T. Evans, Howard I. Nadjari, and Sherry A. Burchell, “Quotational and Reference Accuracy in Surgical Journals: A Continuing Peer Review Problem,” JAMA 263, no. 10 (1990): 1353; J. E. Fenton et al., “The Accuracy of Citation and Quotation,” 41, http://www.ncbi.nlm.nih.gov/pubmed/10764235; Sammy Al-Benna et al., “Accuracy of References in Burns Journals,” Burns 35, no. 5 (August 2009): 678, https://doi.org/10.1016/j.burns.2008.11.014.
23. Peter A. Todd et al., “One in Four Citations in Marine Biology Papers Is Inappropriate,” Marine Ecology Progress Series 408, (2010): 300, https://doi.org/10.3354/meps08587; Haussmann et al., “Referencing Practices,” 544.
24. De Lacey, Record, and Wade, “How Accurate Are Quotations,” 884.
25. Evans, Nadjari, and Burchell, “Quotational and Reference Accuracy,” 1353.
26. Sung Yul Lee and Jong Suk Lee, “A Survey of Reference Accuracy in Two Asian Dermatologic Journals (the Journal of Dermatology and the Korean Journal of Dermatology),” International Journal of Dermatology 38, no. 5 (1999): 357; Philip Eichorn and Alfred Yankauer, “Do Authors Check Their References? A Survey of Accuracy of References in Three Public Health Journals,” American Journal of Public Health 77, no. 8 (August 1987): 1011, http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1647239&tool=pmcentrez&rendertype=abstract.
27. Jergas and Baethge, “Quotation Accuracy in Medical Journal Articles,” 5.
28. Judith M. Nixon, “Core Journals in Library and Information Science: Developing a Method for Ranking LIS Journals,” College & Research Libraries 75, no. 1 (2014): 84, 71, https://doi.org/10.5860/crl12-387.
29. “Sample Size Calculator,” Creative Research Systems, accessed September 30, 2016, http://www.surveysystem.com/sscalc.htm.
30. Marilyn H. Oermann and Linda D. Ziolkowski, “Accuracy of References in Three Critical Care Nursing Journals,” Journal of PeriAnesthesia Nursing 17, no. 2 (April 2002): 78–83, https://doi.org/10.1053/jpan.2002.31658.
31. Lopresti, “Citation Accuracy,” 654.
32. Andrew D. Asher, Lynda M. Duke, and Suzanne Wilson, “Paths of Discovery: Comparing the Search Effectiveness of EBSCO Discovery Service, Summon, Google Scholar, and Conventional Library Resources,” College & Research Libraries 74, no. 5 (2013): 474, https://doi.org/10.5860/crl-374.
33. Drake et al., “The Propagation and Dispersal,” 2; Haussmann et al., “Referencing Practices,” 544.
34. Todd et al., “One in Four Citations,” 300; Haussmann et al., “Referencing Practices,” 544.
35. Ma Luo et al., “Accuracy of Citation and Quotation in Foot and Ankle Surgery Journals,” Foot & Ankle International 34, no. 7 (2013): 953.
36. J. E. Fenton et al., “The Accuracy of Citation and Quotation,” 40.
37. Rekdal, “Academic Citation Practice,” 572.
38. Booth, “A Checker’s Career,” 269.
39. Davies, “Reference Accuracy,” 379.
40. Pierre M. George and Kathryn Robbins, “Reference Accuracy in the Dermatologic Literature,” Journal of the American Academy of Dermatology 31, no. 1 (July 1994): 63, https://doi.org/10.1016/S0190-9622(94)70136-9.
41. Davies, “Reference Accuracy,” 384.
42. Jergas and Baethge, “Quotation Accuracy in Medical Journal Articles,” 13.

Appendix. Bibliography of Journal Articles

Al-Benna, Sammy, Prachi Rajgarhia, Safraz Ahmed, and Zeeshan Sheikh. “Accuracy of References in Burns Journals.” Burns 35, no. 5 (August 2009): 677–80. https://doi.org/10.1016/j.burns.2008.11.014.

Asano, Migiwa, Katsuya Mikawa, Kahoru Nishina, Nobuhiro Maekawa, and Hidefumi Obara. “Improvement of the Accuracy of References in the Canadian Journal of Anaesthesia.” Canadian Journal of Anaesthesia—Journal Canadien D’anesthésie 42, no. 5 (1995): 370–72.

Asher, Andrew D., Lynda M. Duke, and Suzanne Wilson. “Paths of Discovery: Comparing the Search Effectiveness of EBSCO Discovery Service, Summon, Google Scholar, and Conventional Library Resources.” College & Research Libraries 74, no. 5 (2013): 464–88. https://doi.org/10.5860/crl-374.

Benning, Susan P., and Susan C. Speer. “Incorrect Citations: A Comparison of Library Literature with Medical Literature.” Bulletin of the Medical Library Association 81, no. 1 January (1993): 56–58.

Booth, Andrew. “A Checker’s Career?” Health Information and Libraries Journal 21, no. 4 (2004): 269–72. https://doi.org/10.1111/j.1471-1842.2004.00538.x.

Boyce, Bert R., and Carolyn Sue Banning. “Data Accuracy in Citation Studies.” RQ 18, no. 4 (1979): 349–50.

Buchan, John C., John Norris, and Hannah Kuper. “Accuracy of Referencing in the Ophthalmic Literature.” American Journal of Ophthalmology 140, no. 6 (December 2005): 1146–48. https://doi.org/10.1016/j.ajo.2005.07.018.

Davids, Jon R., Daniel M. Weigl, Joye P. Edmonds, and Dawn W. Blackhurst. “Reference Accuracy in Peer-Reviewed Pediatric Orthopaedic Literature.” The Journal of Bone and Joint Surgery.American Volume 92, no. 5 (2010): 1155–61. https://doi.org/10.2106/JBJS.I.00063.

Davies, Karen. “Reference Accuracy in Library and Information Science Journals.” Aslib Proceedings 64, no. 4 (2012): 373–87. https://doi.org/10.1108/00012531211244734.

de Lacey, Gerald, Carol Record, and Jenny Wade. “How Accurate Are Quotations and References in Medical Journals?” British Medical Journal (Clinical Research Ed.) 291 (1985): 884–86.

Doms, C. A. “A Survey of Reference Accuracy in Five National Dental Journals.” Journal of Dental Research 68, no. 3 (March 1989): 442–44. http://www.ncbi.nlm.nih.gov/pubmed/2921384.

Drake, D. C., B. Maritz, S. M. Jacobs, C. J. Crous, A. Engelbrecht, A. Etale, M. J. Fourie, et al. “The Propagation and Dispersal of Misinformation in Ecology: Is There a Relationship between Citation Accuracy and Journal Impact Factor?” Hydrobiologia 702, no. 1 (2013): 1–4. https://doi.org/10.1007/s10750-012-1392-6.

Eichorn, Philip, and Alfred Yankauer. “Do Authors Check Their References? A Survey of Accuracy of References in Three Public Health Journals.” American Journal of Public Health 77, no. 8 (August 1987): 1011–12. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1647239&tool=pmcentrez&rendertype=abstract.

Evans, James T., Howard I. Nadjari, Sherry A. Burchell. “Quotational and Reference Accuracy in Surgical Journals: A Continuing Peer Review Problem.” JAMA 263, no. 10 (1990): 1353–54.

Fenton, J. E., H. Brazier, A. De Souza, J. P. Hughes, and D. P. McShane. “The Accuracy of Citation and Quotation in Otolaryngology/Head and Neck Surgery Journals.” Clinical Otolaryngology and Allied Sciences 25, no. 1 (February 2000): 40–44. http://www.ncbi.nlm.nih.gov/pubmed/10764235.

Foreman, Marquis D., and Karin T. Kirchhoff. “Accuracy of References in Nursing Journals.” Research in Nursing & Health 10, no. 3 (June 1987): 177–83. http://www.ncbi.nlm.nih.gov/pubmed/3296021.

George, Pierre M., and Kathryn Robbins. “Reference Accuracy in the Dermatologic Literature.” Journal of the American Academy of Dermatology 31, no. 1 (July 1994): 61–64. https://doi.org/10.1016/S0190-9622(94)70136-9.

Goldberg, Richard, Edward Newton, Julie Cameron, Raymond Jacobson, Linda Chan, W. Richard Bukata, and Amine Rakab. “Reference Accuracy in the Emergency Medicine Literature.” Annals of Emergency Medicine 22, no. 9 (September 1993): 1450–54. http://www.ncbi.nlm.nih.gov/pubmed/8363119.

Goodrich, June E., and Charles G. Roland. “Accuracy of Published Medical Reference Citations.” Journal of Technical Writing and Communication 7, no. 1 (1977): 15–19.

Gosling, M. C., Melainie Cameron, and Peter F. Gibbons. “Referencing and Quotation Accuracy in Four Manual Therapy Journals.” Manual Therapy 9, no. 1 (February 2004): 36–40. https://doi.org/10.1016/S1356-689X(03)00056-0.

Gupta, Piyush, Mukesh Yadav, Anup Mohta, and Panna Choudhury. “References in Indian Pediatrics: Authors Need to Be Accurate.” Indian Pediatratrics 42, no. 2 (2005): 140–45.

Hansen, Margaret E., and Donald D. McIntire. “Reference Citations in Radiology: Accuracy and Appropriateness of Use in Two Major Journals.” American Journal of Roentgenology 163, no. 3 (1994): 719–23.

Haussmann, Natalie Suzette, Trevor McIntyre, Adam John Bumby, and Michael John Loubser. “Referencing Practices in Physical Geography: How Well Do We Cite What We Write?” Progress in Physical Geography 37, no. 4 (2013): 543–49. https://doi.org/10.1177/0309133313482135.

Hinchcliff, K. W., N. J. Bruce, J. D. Powers, and M. L. Kipp. “Accuracy of References and Quotations in Veterinary Journals.” Journal of the American Veterinary Medical Association 202, no. 3 (1993): 397–400.

Holt, Shaun, Robert Siebers, Aneta Suder, Rachel Loan, and Oliver Jeffery. “Accuracy of References in Australian and New Zealand Medical Journals.” New Zealand Medical Journal 113, no. 1119 (2000): 416–17.

Jackson, Kent, Jack A. Porrino Jr., Virak Tan, and Aaron Daluiski. “Reference Accuracy in the Journal of Hand Surgery.” Journal of Hand Surgery 28, no. 3 (May 2003): 377–80. https://doi.org/10.1053/jhsu.2003.50085.

Jergas, Hannah, and Christopher Baethge. “Quotation Accuracy in Medical Journal Articles—A Systematic Review and Meta-Analysis.” PeerJ 3 (2015): e1364. https://doi.org/10.7717/peerj.1364.

Kao, Hsuan-Hung and Wen-Yau Cathy Lin. “Quotation Errors in the Articles of Library and Information Science Journals in Taiwan.” Journal of Educational Media & Library Sciences 52, no. 2 (2015): 127–56. https://doi.org/10.6120/JoEMLS.2015.522/0009.RS.AM.

Key, Jack D., and Charles G. Roland. “Reference Accuracy in Articles Accepted for Publication in the Archives of Physical Medicine and Rehabilitation.” Archives of Physical Medicine and Rehabilitation 58, no. 3 (1977): 136–37.

Kristof, Cindy. “Accuracy of Reference Citations in Five Entomology Journals.” American Entomologist 43, no. 4 (1997): 246–51.

Lee, Sung Yul, and Jong Suk Lee. “A Survey of Reference Accuracy in Two Asian Dermatologic Journals (the Journal of Dermatology and the Korean Journal of Dermatology).” International Journal of Dermatology 38, no. 5 (1999): 357–60.

Lok, Candy K. W., Matthew T. V. Chan, and Ida M. Martinson. “Risk Factors for Citation Errors in Peer-Reviewed Nursing Journals.” Journal of Advanced Nursing 34, no. 2 (April 2001): 223–29. http://www.ncbi.nlm.nih.gov/pubmed/11430284.

Lopresti, Robert. “Citation Accuracy in Environmental Science Journals.” Scientometrics 85, no. 3 (December 2010): 647–55. https://doi.org/10.1007/s11192-010-0293-6.

Luki, Ivan Kresimir, Anita Luki, Vicko Glunci, Vedran Katavi, Vladimira Vucenik, and Ana Marusi. “Citation and Quotation Accuracy in Three Anatomy Journals.” Clinical Anatomy 17, no. 7 (2004): 534–39.

Luo, Ma, Charles Chuan Li, Domingo Molina, Clark R. Andersen, and Vinod K. Panchbhavi. “Accuracy of Citation and Quotation in Foot and Ankle Surgery Journals.” Foot & Ankle International 34, no. 7 (2013): 949–55. https://doi.org/10.1177/1071100713475354.

McLellan, M. Faith, L. Douglas Case, and Molly C. Barnett. “Trust, but Verify: The Accuracy of References in Four Anesthesia Journals.” Anesthesiology 77, no. 1 (1992): 185–88.

Mohammad, Ali E., and Daniel M Laskin. “Citation Accuracy in the Oral and Maxillofacial Surgery Literature.” Journal of Oral and Maxillofacial Surgery 66, no. 1 (January 2008): 3–6. https://doi.org/10.1016/j.joms.2007.06.682.

Ngan Kee, Warwick D., Vijay J. Roach, and Tze K. Lau. “How Accurate Are References in the Australian and New Zealand Journal of Surgery?” Australian and New Zealand Journal of Surgery 67, no. 7 (1997): 417–19.

Ngan Kee, Warwick D., Vijay J, Roach, and Tze K. Lau. “The Accuracy of References in the Hong Kong Medical Journal.” Hong Kong Medical Journal = Xianggang Yi Xue Za Zhi / Hong Kong Academy of Medicine 3, no. 4 (December 1997): 377–80. http://www.ncbi.nlm.nih.gov/pubmed/11847389.

Nishina, K., M. Asano, K. Mikawa, N. Maekawa, and H. Obara. “The Accuracy of Reference Lists in Acta Anaesthesiologica Scandinavica.” Acta Anaesthesiologica Scandinavica 39, no. 5 (July 1995): 577–78. http://www.ncbi.nlm.nih.gov/pubmed/7572002.

Nixon, Judith. M. “Core Journals in Library and Information Science: Developing a Methodology for Ranking LIS Journals.” College & Research Libraries 75, no. 1 (2014): 66–90. https://doi.org/10.5860/crl12-387.

Nuckles, D. B., N. N. Pope, and J. D. Adams. “A Survey of the Accuracy of References in 10 Dental Journals.” Operative Dentistry 18, no. 1 (1993): 28–32.

O’Connor, Alan E. “Review of the Accuracy of References in the Journal Emergency Medicine.” Emergency Medicine Australasia 14, no. 2 (2002): 139–41.

O’Connor, Lisa G., and Cindy Kristof. “Verify Your Citations: Accuracy of Reference Citations in Twelve Business and Economics Journals.” Journal of Business & Finance Librarianship 6, no. 4 (2001): 23–40.

Oermann, Marilyn H., Sarah L. Cummings, and Nancy A. Wilmes. “Accuracy of References in Four Pediatric Nursing Journals.” Journal of Pediatric Nursing 16, no. 4 (August 2001): 263–68. https://doi.org/10.1053/jpdn.2001.25537.

Oermann, Marilyn H., Nancy A. Wilmes, and Patricia Braski. “Reference Accuracy in Neonatal-Maternal Nursing Literature.” Neonatal Network 21, no. 1 (February 2002): 23–26. https://doi.org/10.1891/0730-0832.21.1.23.

Oermann, Marilyn H., and Linda D. Ziolkowski. “Accuracy of References in Three Critical Care Nursing Journals.” Journal of PeriAnesthesia Nursing 17, no. 2 (April 2002): 78–83. https://doi.org/10.1053/jpan.-2002.31658.

Oren, Gale, and Maureen Watson. “Accuracy of References in the Ophthalmic Literature.” Journal of the Medical Library Association: JMLA 97, no. 2 (April 2009): 142–45. https://doi.org/10.3163/1536-5050.97.2.014.

Orlin, Wayne, Jason Pehling, and M. Anthony Pogrel. “Do Authors Check Their References? A Survey of 500 References from the Journal of Oral and Maxillofacial Surgery.” Journal of Oral and Maxillofacial Surgery 54, no. 2 (1996): 200–202.

Pandit, Idrisa. “Citation Errors in Library Literature: A Study of Five Library Science Journals.” Library & Information Science Research 15, no. 2 (1993): 185–98.

Pope, Nancy N. “Accuracy of References in Ten Library Science Journals.” RQ 32, no. 2 (1992): 240–43.

Poyer, Robert K. “Inaccurate References in Significant Journals of Science.” Bulletin of the Medical Library Association 67, no. 4 (1979): 396–98.

Raja, U. Y., and J. G. Cooper. “How Accurate Are the References in Emergency Medical Journal?” Emergency Medicine Journal: EMJ 23, no. 8 (August 2006): 625–26. https://doi.org/10.1136/emj. 2004.022103.

Rekdal, Ole Bjørn. “Academic Citation Practice: A Sinking Sheep?” portal: Libraries and the Academy 14, no. 4 (2014): 567–85. https://doi.org/10.1353/pla.2014.0025.

Roach, Vijay J., Tze K. Lau, and Warwick D. Ngan Kee. “The Quality of Citations in Major International Obstetrics and Gynecology Journals.” American Journal of Obstetrics & Gynecology 177, no. 4 (1997): 973–75.

Schulmeister, Lisa. “Quotation and Reference Accuracy of Three Nursing Journals.” Image: The Journal of Nursing Scholarship 30, no. 2 (1998): 143–46.

Siebers, Robert. “The Accuracy of References of Three Allergy Journals.” Journal of Allergy and Clinical Immunology 105, no. 4 (April 2000): 837–38. https://doi.org/10.1067/mai.2000.104935.

Siebers, Robert, and Shaun Holt. “Accuracy of References in Five Leading Medical Journals.” Lancet 356, no. 9239 (2000): 1445.

Spivey, Christina A., and Scott E. Wilks. “Reference List Accuracy in Social Work Journals.” Research on Social Work Practice 14, no. 4 (2004): 281–86. https://doi.org/10.1177/1049731503262131.

Stull, G. Alan, Robert W. Christina, and Sherrill A. Quinn. “Accuracy of References in Research Quarterly for Exercise and Sport.” Research Quarterly for Exercise and Sport 62, no. 3 (1991): 245–48.

Taylor, Mary K. “The Practical Effects of Errors in Reference Lists in Nursing Research Journals.” Nursing Research 47, no. 5 (1998): 300–303.

Todd, Peter A., James R. Guest, Junxiu Lu, and Loke Ming Chou. “One in Four Citations in Marine Biology Papers is Inappropriate.” Marine Ecology Progress Series 408 (June 3, 2010): 299–303. https://doi.org/10.3354/meps08587.

Todd, Peter A., Darren C. J. Yeo, Daiqin Li, and Richard J. Ladle. “Citing Practices in Ecology: Can We Believe Our Own Words?” Oikos 116, no. 9 (September 2007): 1599–1601. https://doi.org/10.1111/j.2007.0030-1299.15992.x.

Vargas-Origel, A., G. Gómez-Martinez, and M. A. Vargas-Nieto. “The Accuracy of References in Paediatric Journals.” Archives of Disease in Childhood 85, no. 6 (2001): 497–98. https://doi.org/10.1136/adc.85.6.497.

Wager, Elizabeth, and Philippa Middleton. “Technical Editing of Research Reports in Biomedical Journals.” Cochrane Database of Systematic Reviews 4, MR000002 (2008). https://doi.org/10.1002/14651858.MR000002.pub3.

Wilks, Scott E., Jennifer R. Geiger, Samantha M. Bates, and Amy L. Wright. “Reference Accuracy Among Research Articles Published in Research on Social Work Practice.” Research on Social Work Practice (2016): 1–5. https://doi.org/10.1177/1049731515626802.