Giving Government Information the Green Light: Creating and Using the Stoplight Evaluation Guide in the Information Literacy Classroom

Kathy Karn-Carmichael and Kari D. Weaver

Guiding students to use appropriate information for research can be a difficult task in the higher education classroom. As students enter the collegiate ranks, librarians and teaching faculty must work hard to move students away from their Google-centric search strategies and instead use databases of scholarly publications and other appropriate library resources.

To any librarian teaching students to evaluate information during introductory information literacy sessions, this challenge is especially poignant. Discussions in these courses frequently break resources down into groups considered “scholarly” or “non-scholarly,” where scholarly resources are acceptable and found from the library, and non-scholarly resources are materials largely found via the World Wide Web. This schema puts government information at a disadvantage for multiple reasons: it is primarily available online, often lacks authorship from a noted scholar, and looks and reads differently than the more traditional “academic” sources students are encouraged to use. Thus, a blanket ban on online or less-scholarly resources often catches government resources in its sweep.

Despite the fact that the University of South Carolina Aiken is part of the federal depository library program, our library instruction team felt that government resources were being underutilized for student research. In addition, it was observed that the current topics of papers and speeches in lower level undergraduate coursework suffered for lack of government resources. However, from the librarians’ instruction work in information literacy classes, we also knew that students were not being guided to these resources either. These gaps required the development of a tool to help students evaluate information that would provide categorization beyond the scholarly/non-scholarly paradigm, that might allow for students to identify and acknowledge the need for differing information in various situations, and that would allow for the appropriate use of government resources. This article discusses the development, implementation, and impact of just such a tool, the Stoplight Evaluation Guide, based on three years of usage at a medium-sized baccalaureate institution.

Literature Review

Today’s undergraduates have never known a world without the Internet, Google, and instant access to information. As a consequence, literature that discusses source evaluation concentrates primarily on websites.1 While much of the information content used by students is available electronically, source evaluation instruction should not be limited by mode of delivery, instead also integrating discussion of information beyond simply articles, books, or websites. The traditional focus of information evaluation instruction for undergraduates has conformed primarily to scholarly versus non-scholarly paradigms, which eliminate important sources, including newspapers, online communities, broadcast transcripts, and government publications.2 The use of either/or in other source evaluation checklists fails to adequately address these concerns since they ignore the situational requirements inherent in selecting information sources for discrete tasks.3 These narrowly defined criteria, which have been the standard of undergraduate research for previous generations, need to be reevaluated and expanded to reflect the changing world of increasingly complex and varied available information resources.

Librarians have attempted to supplement source evaluation comprehension by incorporating checklist models such as the widely used CRAAP test. The CRAAP test prompts students to look at specific characteristics of an information source and attempt to compartmentalize the task of analyzing the source’s credibility. Many scholars believe these tools are inadequate, particularly because “non-academic items . . . can be valid resources for undergraduate research,4 and [librarians] must shift the focus to whether or not the item is suitable for the purpose at hand.”5 Common evaluative criteria relying on currency, authority, and accuracy designed to assist in the evaluation between academic and non-academic sources are also insufficient because they similarly overlook the purpose for which such information may be used.6 Considering information evaluation in the larger framework of lifelong learning, it is apparent that librarians can incorporate aspects of currently used criteria, but must move toward models that can be employed in contexts beyond academia.

As previously noted, one of the limitations of favoring scholarly publications is that government resources fall outside the scope of scholarly information despite, for the most part, providing authoritative information. “Government document librarians have long lamented that their collections are underused and underappreciated.”7 The current methods used in information literacy instruction are only partially to blame for the exclusion of government information. As the Internet grew in acceptance as a source of information, Brunvald and Pashkov-Balkenhol note undergraduate use of web resources for research increased substantially.8 In 1997, an editorial appeared in the Chronicle of Higher Education deploring the increased reliance of web sources, which was seen to reduce quality of student research papers.9 This trend caused considerable concern in academia worldwide and some predicted a crisis in undergraduate research, fearing that the Internet might replace academic journals.10 Faculty responded by restricting or completely eliminating web resources for inclusion in undergraduate research.11 This knee-jerk reaction resulted in removing a treasure trove of primary sources contained within government document collections both in print and digital formats. Tragically, this “throw out the baby with the bathwater” approach did not solve the problem of students’ selection of information sources. Instead, it created a new problem by limiting students’ ability to recognize authoritative, reliable information sources.

Development of the Stoplight Evaluation Guide

Information evaluation features prominently in the new ACRL Framework for Information Literacy for Higher Education (http://bit.ly/acrlframework). Teaching source evaluation to lower level undergraduates is the first step in providing students with the skills needed to find and use suitable information in academic research. At the University of South Carolina Aiken, these skills are taught through information literacy instruction in one of two formats: either in discrete one-shot sessions of subject courses primarily taught by faculty in the disciplines, or in general education information literacy classes primarily taught by librarians. Each of these models requires some distinct pedagogical strategies. In the general education IL classes, librarians acting as faculty of record for a class are able to build knowledge over time and assign homework. On the other hand, instruction in one-shot sessions requires strategies that can be employed on a short time-scale and may be reinforced by faculty in the disciplines. With these varied needs and contexts, any evaluation schema must be robust and flexible at the same time.

One of the desired learning outcomes from these information modes of literacy instruction is to emphasize the use of authoritative information from a variety of sources, generally discouraging students’ reliance on Google and other popular search engines in academic research. As mentioned previously, source evaluation has historically focused on “scholarly versus non-scholarly” resources. However, lower-level students frequently struggled with the concepts and characteristics of scholarly sources, and, as a result, they frequently encountered difficulty in putting these principles into practice. In addition to this challenge, there were frequently occasions when other sources are justified even though scholarly information is generally preferred for most academic research. These challenges are what ultimately led to the creation of the Stoplight Evaluation Guide at the University of South Carolina Aiken.

The Stoplight Evaluation Guide, Pedagogy, and Practice

The Stoplight Evaluation Guide expands the scholarly versus non-scholarly paradigm, providing students with a framework that enables them to analyze non-scholarly information sources that are frequently overlooked. These overlooked sources include government information such as congressional hearings, judicial opinions, federal legislation, and government-funded research. The inclusion makes the Stoplight Evaluation Guide one of the only source evaluation tools to include government documents and authenticated government documents as acceptable sources for inclusion in academic research.

Utilizing the image of a stoplight, the guide ranks typical information sources frequently used by students into three separate categories that designate a source’s suitability for academic research. Red signifies sources that never or almost never should be used. Yellow signifies sources that may be used depending on permitted sources. Green signifies sources that almost always may be used. A color copy of the one-page document can be found at http://bit.ly/stoplighteval. Additionally, a black and white version of the guide is included with this article (see figure 1). The description for each color and allowed resources are the following:

RED LIGHT: Stop and Seriously Consider before Use. Generally unacceptable for academic purposes.

Common factors: No author or “lay” author. Casual language.

Resources: Wikipedia, commercial websites, Websites whose authorship cannot be verified, Popular search engine top results (Google, Ask.com, etc.)

YELLOW LIGHT: Proceed with Caution before Use. Acceptable, depending on purpose.

Common factors: Uses a combination of casual and academic language. Authorship may be “lay” author or subject expert

Resources: Television and radio broadcasts. Newspapers and popular magazines. Opinion/point of view websites. Online communities. Popular books. Top Google Scholar search results.

GREEN LIGHT: Go Ahead and Use.

Common factors: Authors who are experts. Use of academic language specific to the subject matter

Resources: Online subscription databases. Scholarly journals. Official government reports. Authenticated government documents. Edited books from scholarly publishing houses

By grouping commonly used resources, each color segment defines the general characteristics of the resource, enabling students to understand appropriateness of specific resource types as well as helping them recognize that writing style and authorship can provide clues to the suitability of a source. Furthermore, this tool incorporates elements of critical thinking and reasoning that allow it to possess relevance beyond the library.

When teaching the Stoplight Evaluation Guide, the librarian explains each color section and how students may interpret the guide to aid them in determining which sources are best for their research topic. Characteristics of each color group are discussed, as well as sources typically found in each section. To aid student understanding, an active learning exercise is frequently included in the Stoplight Guide instruction.

One of the benefits of the Stoplight Evaluation Guide is that it can be effectively paired with a variety of instructional strategies based on the amount of time available for addressing information evaluation in the classroom. For instance, librarians have incorporated the Stoplight Evaluation Guide into a short module in a one-shot instruction session immediately after students perform search tasks in a library database. It has been used by disciplinary faculty in a follow-up to one-shot information literacy instruction as a way to help bridge instruction in the library and the traditional classroom environment. In full semester-long information literacy classes, the Stoplight Evaluation Guide has been successfully paired with problem-based learning activities and document-based question activities, and it is flexible enough to work successfully in a number of disciplines. It has proven especially helpful when integrated with USCA’s public speaking courses. These classes use both informative and persuasive speech assignments as a basis for major assignments, and government information and reputable news reporting are both widely used in developing these assignments.

Reactions and Results

The Stoplight Evaluation Guide was introduced in the classroom in the 2012 fall semester and was used primarily in lower level English and Communication classes. Based on its initial success, it was introduced to a wider audience the following summer, when it was included in the Critical Inquiry Institute, a professional development workshop for faculty who will be teaching the first-year Critical Inquiry (CI) class to incoming freshmen. Along with training on various CI topics, the CI Institute provides faculty with suggested lesson plans and course materials. Since the introduction of the Stoplight Guide to CI instructors, it has been among the most requested instructional handouts ever provided by the library.

Faculty reaction outside of CI has also been extremely positive. Faculty frequently request source evaluation using the Stoplight Guide as a key topic to include when scheduling Information Literacy Instruction. Here are some comments the authors have received about the guide:

“The benefit of using the guide as a tool for source evaluation is that it provides students with an easy visual rubric for source evaluation. The stop (and consider the rhetorical situation of your project and whether these sources are appropriate for your purpose, message and audience), caution (these sources may be useful, but could be inappropriate for academic work), and go (sources are acceptable for academic work) signals are easy to recall and serve as reminders to students to think about the appropriateness of sources they might use.”

“The Stoplight Evaluation Guide is an essential tool in my courses. The simple visual image clearly communicates the essential principles of source evaluation. I encourage students to use it during the research process, and I also use it when providing feedback on presentations and written assignments. It works especially well as a tool when I need to work with a student struggling with understanding the importance of source evaluation.”

In addition to the positive response to the guide from students and faculty, we have also seen growth in the use of government information in student writing. USCA faculty regularly evaluate the portfolios for Critical Inquiry and the written work from the students’ 101 and 102 composition series (also known as the “freshman folder”). Since the Stoplight Evaluation Model has been introduced, it has been observed that students are using government information more frequently in these classes, and likely others.

Conclusions

Information literacy instruction has historically been limited to a constrained “scholarly/non-scholarly” paradigm, which has frequently caused confusion among students and excluded the use of many potentially valuable resources, including government information. The Stoplight Evaluation Guide was developed to provide students with a more nuanced and expansive tool for evaluating information. Since the focus of this model is not limited to scholarly/non-scholarly, sources that may have been previously overlooked by students are available for inclusion by using the guidelines set forth in the Stoplight Evaluation Guide. The model’s design also allows it to be used very flexibly in a variety of instructional situations. The response from faculty and students has been very positive, and the increased use of government information in student writing and oral presentations has been clearly observed. Finally, the Stoplight Evaluation Guide holds additional promise going forward as it provides classroom faculty and librarians a common and identifiable metaphor to use in discussions of source evaluation, allowing consistent reinforcement of these important concepts campus wide.

Kathy Karn-Carmichael (kathyk@usca.edu) is Instructor of Library Science and Government Information/Instruction Librarian, University of South Carolina Aiken. Kari D. Weaver (kariw@usca.edu) is Associate Professor of Library Science and Library Instruction Coordinator, University of South Carolina Aiken.

References

1. Eszter Hargittai et al., “Trust Online: Young Adults’ Evaluation of Web Content,” International Journal of Communication 4 (2010): 468–94; Jennifer Wiley et al., “Source Evaluation, Comprehension, and Learning in Internet Science Inquiry Tasks,” American Educational Research Journal 46, no. 4 (2009): 1060–106, http://dx.doi.org/10.3102/0002831209333183.
2. Susanna Calkins and Matthew R. Kelley, “Evaluating Internet and Scholarly Sources Across the Disciplines: Two Case Studies,” College Teaching 55, no. 4 (2007): 151–56, http://dx.doi.org/10.3200/CTCH.55.4.151-156.
3. Jonathan Ostenson, “Reconsidering the Checklist in Teaching Internet Source Evaluation,” portal: Libraries and the Academy 14, no. 1 (2014): 33-50, http://dx.doi.org/10.1353/pla.2013.0045.
4. Candice Dahl, “Undergraduate Research in the Public Domain: The Evaluation of Non-Academic Sources Online,” Reference Services Review 37, no. 2 (2009): 155–63, http://dx.doi.org/10.1108/00907320910957198; Ostenson, “Reconsidering the Checklist in Teaching Internet Source Evaluation,” 36.
5. Dahl, “Undergraduate Research in the Public Domain,” 160.
6. Ibid.
7. Amy Brunvand and Tatiana Pashkova-Balkenhol, “Undergraduate Use of Government Information: What Citations Tell us About Instruction Strategies,” portal: Libraries and the Academy 8, no. 2 (2008): 197–209, http://dx.doi.org/10.1353/pla.2008.0014.
8. Ibid., 197.
9. David Rothenberg, “How the Web Destroys the Quality of Students’ Research Papers,” Chronicle of Higher Education, August 15, 1997.
10. Brunvand and Pashkova-Balkenhol, “Undergraduate Use of Government Information,” 199.
11. Ibid.