In 2000, the Association of College and Research Libraries (ACRL) published the “Information Literacy Competency Standards for Higher Education.”² Intended to facilitate the development of lifelong learners, the standards outline the skills needed for students to identify an information need and then locate, evaluate, and utilize resources to fulfill that need.³ For more than a decade, the ACRL guidelines have directed the library profession’s approach to instruction, shaping the ways that librarians conceptualize, design, provide, and assess library instruction. Corresponding to the widespread adoption of these standards, there has been an increase in research investigating students’ skills (or lack thereof) in critical thinking, and more specifically, information evaluation. The majority of these research studies, however, are based on the evaluation of web and print sources as separate materials. As the numbers of online and open access publications increase and the boundaries between formats of information recede, the depiction of print and electronic resources as existing in distinct and separate categories does not accurately reflect the modern search experience.⁴ It is also misleading to students who are used to accessing a variety of media and information sources in multiple formats.

Student confusion about the format and quality of information sources is substantiated by recent research. In a 2009 report for the United Kingdom’s Joint Information Systems Council (JISC), researchers identified a dissonance between college and university students’ expectations of published research and the realities of those bodies of work.⁵ When asked what types of information a student would recognize as “research,” an overwhelming majority (97 percent) identified traditional formats such as books and articles. When confronted with less well-known formats, such as posters or dissertations, the number of students willing to identify the documents as “research” greatly decreased.⁶ Additional qualitative results describing student confusion were obtained in small focus group sessions. While the majority of students “distrusted” the Internet, they widely accepted “all published materials” as appropriate for academic use.⁷ This inaccurate distinction between the credibility of print and electronic resources was also reported in research by Biddix et al., who found that students view the information available from an academic library as “vetted” or “pre-accepted.”⁸ Students have oversimplified relationships between publication format, library resources, and credibility, a situation that has been further complicated by the increase in federated search tools. Although federated searching may simplify the research experience, it also increases the quantity of unfamiliar materials to which students are exposed, while simultaneously making distinctions between information sources less discrete.

As the information landscape undergoes radical shifts, librarians’ approaches to teaching information literacy and information evaluation have remained relatively static. Approximately ten years ago, two information evaluation methods associated with different mnemonic devices were shared in the library literature and were subsequently incorporated into many library instruction sessions. In 2004, Blakeslee described the motivation behind designing California State University Chico’s CRAAP Test as a desire to create a memorable acronym because of its “associative powers.”⁹ Intended to guide users through evaluating the Currency, Relevance, Authority, Accuracy, and Purpose of a document, the method’s accompanying checklist and questions can be applied to both print and online resources; however, its emphasis on the evaluation of electronic materials has resulted in a loose categorization of the method as a website evaluation tool.¹⁰ In contrast, the CRITIC method was incorporated into library instruction as a tool to be utilized in the evaluation of print resources.¹¹ In a presentation on the method at a 2004 conference, Matthies and Helmke describe CRITIC as a “practical system of applied critical thought”; repurposing the steps of the scientific method, it encourages users to approach evaluation as an iterative process and to interrogate the Claim, Role of the Claimant, Testing, Independent Verification, and Conclusion of a given document.¹²

Both the CRAAP Test and CRITIC method attempt to simplify the evaluation process by breaking down complex ideas into a set of accessible criteria, but little research has been conducted on the effectiveness of the methods themselves. However, one recently published study on the advantages of formative assessment in information literacy instruction includes a series of anecdotal observations that may provide insight into the effectiveness of the CRAAP Test.¹³ Following an instruction workshop in which the test was taught, many students self-reported a persisting difficulty with “determining the quality of different sources.”¹⁴ The authors found that some students continued to have trouble “distinguishing between popular magazines and scholarly journals” and “finding authoritative websites” even after follow-up consultations.¹⁵ Their findings suggest that the CRAAP Test may not effectively bridge the gap between determining easily identifiable qualities, such as date of publication, and those that require a greater level independent judgment and critical thinking, such as authority, especially if used in only a single instruction session.

Meola contends that it is problematic to use models such as CRAAP and CRITIC to teach information evaluation because of their structural dependence on linear processes and checklists.¹⁶ He describes such checklist-based models as “question-begging” and criticizes them for offering “slim guidance” as to how the questions should be answered.¹⁷ Meola also argues that a linear organization encourages students to view evaluation as a “mechanical and algorithmic” process, thereby separating “higher level judgment and intuition” from the evaluation process.¹⁸ Bowles-Terry et al. expand on Meola’s ideas, writing that the checklist approach “reduces critical thinking about the value of information to easily memorized and superficial criteria.”¹⁹ The solution, the authors suggest, is to reconceptualize the evaluation of information as a meaningful process rather than a “look up skill.”²⁰ Librarians can support this by broadening the evaluation methods they teach to include contextualizing a document within a student’s “wider social experience.”²¹

Bowles-Terry et al. also encourage information literacy instructors to enhance their teaching efforts by incorporating aspects of social constructivist theory, developed in large part by Lev Vygotsky.²² In his preeminent writings on child psychology, Vygotsky made highly influential contributions not only to sociological but also educational theory, including the concept of the “zone of proximal development,” or ZPD, which he describes as the distance between what a learner can accomplish independently and what he or she can accomplish under the “guidance of an adult or in collaboration with more capable peers.”²³ According to Vygotsky, a learner’s transition to a more advanced skill set or level of thinking is facilitated in collaboration with a person or group of people at a higher developmental level than the learner.²⁴

Related to the ZPD is the educational theory of instructional scaffolding, a process by which a tutor or instructor helps a learner successfully achieve a task that the learner would be unable to accomplish alone, thus spanning the ZPD. Scaffolding processes assist learners by building on behaviors and tasks they have already mastered to achieve those that require higher levels of thought. In a seminal work on scaffolding, Wood, Bruner, and Ross write that scaffolding begins when a tutor actively interacts with learners and controls the “elements of a task initially beyond the learner’s capacity.”²⁵ According to Bruner, responsive tutors gradually remove their support (the scaffold) as learners develop skills and need less assistance.²⁶ By working with instructors or more competent peers, learners who successfully negotiate skill development are then able to build on their accomplishments by achieving the component steps of a process individually and then progressing to skills of greater intellectual complexity.

Vygotsky theorized that learners may surpass their developmental level by working with others more capable, while Wood, Bruner, and Ross found that learners are capable of recognizing good solutions to a task or problem before they are capable of completing the steps needed to reach that solution by themselves.²⁷ These theories are useful to consider in the design of information literacy instruction and formative learning assessments. Integrating group work into instruction sessions may help learners achieve more success together than if they were to work alone. Utilizing instructional scaffolds may also assist learners in the development of new skills. Furthermore, if the scaffold helps students accomplish goals that they recognize as purposeful and relevant to their near-future success, they may be more invested in developing the skills and learning the process being taught. Based on these criteria, a useful evaluation method in library instruction would be associated with something already familiar to students and valued by course instructors to the extent that they would incorporate the method into their own classes after the library session. An evaluation method that met these ideal qualities would then have the potential to be more fully integrated into a student’s greater learning process by surpassing the limitations of one-shot instruction sessions.

At the University of Tennessee Knoxville, the first-year composition program includes two sequential courses, English 101 and 102. Although the common syllabus for English 101 includes three standardized composition assignments, only one of these, the argumentative paper, requires students to cite outside sources. Despite the applicability of library instruction to the composition curriculum, not all composition sections attend a library instruction session. In fall 2011 and 2012, an average of 24 percent of all English 101 sections requested library instruction, while 70 percent of instructors for English 118 (an Honors course that combines English 101 and 102) requested library instruction for a similar assignment.

Although the argumentative assignment does not require scholarly sources, many composition instructors encourage their students to cite sources with differing points of view. As a result, librarians dedicate a significant portion of the corresponding library instruction session to the development of information evaluation skills. To facilitate this process, an instructional services librarian and a graduate teaching assistant (both hereafter referred to as “the researchers”) sought to employ an in-class evaluation activity that could be consistently used in each 101/118 library session, and would accomplish two aims. First, the activity should effectively introduce students to an information evaluation method. Second, the evaluation method itself should be conducive to student recall and application after the library session.

The researchers first identified an evaluation method and created the in-class evaluation activity, which was completed in small groups during the instruction session and served as a formative assessment. A post-session summative assessment measured student application and recall of the evaluation method. To determine composition instructors’ responses to the session and, in particular, if those instructors found the evaluation method valuable or would consider adding it to their own teaching repertoire, the researchers also created a follow-up survey for composition instructors. With approval from the Institutional Review Board, the researchers piloted the assessments in fall 2011 and implemented them with post-pilot improvements in fall 2012.

When selecting an information evaluation method, researchers searched for a tool that would serve as an instructional scaffold.²⁸ Rather than introducing students to a new evaluation method, the researchers hypothesized that introducing students to a method based on a concept with which they were already familiar would have several benefits: It might allow students to grasp the evaluation criteria more quickly, interpret the steps involved more effectively, and reduce the number of clarifying questions necessary before launching into the activity and applying the method. If such benefits were actualized, the instructional scaffold would also facilitate an efficient use of time for library instructors, who were operating under the time constraints of either a fifty- or seventy-five-minute session.

Between CRAAP and CRITIC, the two methods popular in library instruction, only CRITIC is associated with a concept first-year university students might have encountered in previous learning experiences as its steps are based in the scientific method, a process taught in most elementary and secondary schools.²⁹ However, while the method’s guiding questions may seem familiar, terms associated with the scientific method are not mirrored in the words of the acronym, thereby making it appear new to users. To facilitate the effectiveness of the scaffold, researchers also wanted to teach a “catchy” evaluation method, that is, easily remembered and effectively recalled. Though this specific study did compare student recall of different evaluation methods, anecdotal conversations between library colleagues revealed that the CRAAP and CRITIC criteria were difficult for library instructors to remember. While many of the researchers’ colleagues had utilized the methods more than once in previous information literacy sessions, few were able to recall the components of either acronym.

Therefore, in the interests of familiarity and memorability, the researchers looked outside of library literature. They selected what is colloquially known as the “Five Ws” method of inquiry as a foundation for the activity and subsequent study. The method is composed of six guiding questions: who, what, when, where, why, and how. Frequently taught in primary schools as introduction to basic rhetoric, the Five Ws method is often associated with journalistic investigations and authorship. The likelihood that students would have been introduced to the Five Ws criteria at an early age satisfied the desire of the researchers to present a method with which students were already accustomed, while the guiding questions provided a framework of interrogation on which the researchers could build a more complex activity.

Using its six basic questions as the foundation for the in-class evaluation activity, researchers supplemented each main Five Ws question with more extensive questions to create an activity appropriate for university students. The “who” question, for example, asked students not only to identify the author, but also to investigate the author’s credentials, including where the author worked, if the author had been published more than once, and if the author had research or work experience that contributed to his or her authority. The resulting Five Ws activity served as a formative assessment that measured students’ existing abilities in comprehending and evaluating documents. Students had the opportunity to improve these skills by working through the Five Ws evaluation method in small groups, with a librarian available to direct or correct students’ progress.

During the instruction session, the Five Ws activity was presented to students as an online worksheet, managed and maintained in the UT Libraries’ SurveyMonkey account (appendix A). A link to the activity, as well as a PDF of the document that students evaluated, was available on all library computers used in instruction sessions. The evaluated document was a column by Nicholas Kristof about the 2011 Tōhoku earthquake, tsunami, and Fukushima nuclear radiation leaks in Japan, which appeared in PDF as a full-page from The New York Times opinion section.³⁰ The decisions to have all students evaluate the same document, and for them to analyze a column rather than an article, were deliberate, based on observations from and results of the pilot study. Analyzing an opinion piece challenged students without making the exercise aggravating and, consequently, presented the best opportunity for student learning.³¹

In the library session, students were directed to skim Kristof’s column, which was referred to by the researchers as neither a “column” nor an “article,” but simply the “document.” After skimming the document, students were asked to work in small groups of two to five to evaluate it using the Five Ws criteria via the online worksheet. They were also directed and encouraged to use Internet search engines to help them complete the evaluation, for example, to find more information about the author, his work, and his previous publications. After completing the activity, researchers asked each group to explain to the class how each of the Ws contributed to their group’s final decision of whether they would or would not cite the column in a college research paper.

During the fall 2011 pilot, researchers tested the Five Ws activity with an estimated 682 students.³² Results of the pilot study prompted researchers to make several minor adjustments to the Five Ws activity, including simplifying the phrasing of some questions, choosing to evaluate a single document rather than multiple types in one section, and adding links to definitions for several terms, such as methodology, with which students had struggled. After the pilot project, the improved Five Ws activity was incorporated into many 101 and 118 library instruction sessions. An estimated 391 students in small groups participated in the fall 2012 research study.³³

The pilot study also included a post-session survey, designed in SurveyMonkey and distributed to students in the last quarter of the semester. This twelve-question summative assessment was intended to determine whether several student learning outcomes had been met; namely, whether students found and used library resources after the library session and whether students recalled and used the Five Ws method for evaluating an information source for authority, credibility, and bias. Except for minor clarifications to phrasing, the post-session assessment sent to students in the fall 2012 study was nearly identical to the one distributed during the pilot project.

The post-session summative assessment was distributed to students via their respective composition instructors. During the fall 2011 pilot, sixteen composition instructors taught the thirty composition sessions in which the Five Ws activity was trialed. During the formal study in fall 2012, this number fell to eleven composition instructors for seventeen sections. In each iteration of the study, librarians sent course instructors an email containing an invitation to and directions for completing the 12-question follow-up survey, which they were asked to forward to their students. The emailed invitations were sent to instructors an average of three weeks after the library instruction session. Composition instructors were also sent at least one email reminder to forward to students before the last day of classes.

A separate, qualitative survey was distributed to the same sixteen composition instructors in fall 2011 and eleven composition instructors in fall 2012. This twenty-one-question survey was distributed two to five weeks after the library session and was intended to gather composition instructors’ feedback about the library instruction session. Among other questions, instructors were asked whether or not they found the Five Ws evaluation method valuable and if they had used it or planned to use it in their own classes. The follow-up survey sent to instructors in fall 2012 was nearly identical to the fall 2011 pilot with very minor clarifications to wording in some questions.

In both semesters, students were offered an incentive for participation in the post-session summative assessment. During the pilot project, participants were entered into a drawing for a single $30 gift certificate to the university bookstore. In fall 2012, the incentive was increased and participants were entered into a drawing for one of four $50 gift certificates to the university bookstore. Composition instructors received no incentive in either semester.

Responses are summarized below in an order that matches the question order as presented to participants in the assessments/surveys, with several responses included in table format. The results refer to responses gathered in the fall 2012 study, with comparisons to the pilot project results provided only at the end of each section.

In assigning the initial in-class, formative assessment the researchers had three intended goals: (1) to introduce students to a systematic information evaluation method that would serve as an instructional scaffold to develop evaluation skills, (2) to measure how many students could accurately characterize features of a given source (for example, determining that a given source was opinionated, popular, and written by a credible author), and (3) to examine if students would would be able to present a reasonable argument about why they would or would not cite an opinionated, popular source in a college paper, and if they would use criteria from the library method in their rationales.

On the first point, the use of the Five Ws as an instructional scaffold was successful. Students asked very few questions about the Five Ws method or how to use it. While no formal assessment measured student familiarity with the Five Ws before the library session, more than three quarters of students in each section confirmed by vocal agreement, a head nod, or raised hand that they had heard of the Five Ws before the library session. Because very few students had questions about the evaluation method itself, the scaffold was helpful in using class time efficiently. Most student groups (82, or 82.8 percent) completed at least three-quarters of the activity during class time, and 55 out of 99 student groups (55.6 percent) completed the entire in-class activity.

The effectiveness of the Five Ws as a scaffold was also supported by the summative assessment results. Students in sections where the Five Ws method was reiterated after the library session were better at recalling and applying the evaluation method than those exposed to the Five Ws only once (65.8% versus 46.5%). Scaffolds are tools put in place temporarily to help students master a skill, and learners may need to use a scaffold for some time before they develop or internalize the steps involved in a particular skill. Those students who used the Five Ws method in a class setting more than once were able to apply the skills of source evaluation more consistently than students who used the scaffold only once in a library instruction setting, suggesting that the former group had made more progress in internalizing these skills.

Regarding source evaluation, students were, overall, successful in determining the opinionated and popular nature of the source—nearly 90 percent of student groups described the piece as being written from an opinionated point of view and 70.7 percent of student groups stated that the author’s main purpose in writing was to persuade readers to agree with his opinion (table 1). Students accurately determined that the author had multiple degrees, first-hand knowledge of the topic, and was highly regarded by other journalists and authors. In fact, this particular author’s credentials made such an impression on students that, if a respondent recalled only one of the Five Ws on the follow-up summative assessment, it was most often the importance of evaluating the author (the “who” criterion) of a document. In addition, the vast majority of student groups correctly identified the document as having been written for the general public (92.9 percent) and published in a popular newspaper (96.8 percent).

Most students performed well in evaluating the “who” (authority of the author), “where” (credibility of the publication), and “why” (author’s purpose) characteristics of the document. Because of the emphasis on rhetorical analysis in composition courses, some students may have been attuned to analyzing the tone, language, and purpose of an author. As a result, the number of correct responses to questions about authorial tone and intent is, perhaps, unsurprising. However, the remaining Five Ws and one H—the “what” (document type), “how” (gathering and presentation of data), and “when” (recency/currency/timely impetus for publication)—were criteria with which many, if not most, students struggled.

Of these three criteria, the formative assessment results associated with the “what” and “how” criteria provide insights into what students do not know. These gaps in student knowledge might be classified, in general, as a lack of awareness of publication jargon and processes. In particular, this manifested in students’ inability to recognize either types of documents or types of authors. For instance, the majority of student groups (85.6 percent) claimed that the document was a popular article and not a column. This mistake persisted despite the author’s identification as a columnist in his biographical statements and student descriptions of the author’s opinionated perspective. Furthermore, in the library session wrap-up when groups presented their Five Ws answers to the class, most students were unable to describe to the researchers the differences between a column and an article, or between a columnist and a journalist. Although this provided the researchers with a built-in teachable moment during the session, it also indicates that, though students are capable of identifying an opinion piece when they read it, they do not recognize terminology typically associated with such pieces.

Additionally, student responses demonstrate a lack of understanding of elements included in scholarly publications, as well as ignorance of the jargon used to explain scholarly authors’ research processes. The researchers were, frankly, surprised at the number of student groups claiming that the newspaper column included an abstract and a methodology. In the pilot project, 44.4 percent of student groups claimed the column included a methodology and 33.3 percent of student groups claimed it included an abstract. After the pilot project, the researchers added links to definitions of “abstract” and “methodology” in the assessment, and this reduced the number of groups claiming the column included a methodology (17.9 percent) but had little to no impact on the number of student groups claiming that an abstract was presented (33.9 percent). Students were also asked about how the author gathered data for his argument. Most students claimed that the author conducted a research study (52.2 percent in the fall 2011 pilot, 57.1 percent in fall 2012). Approximately one quarter of students in either semester accurately stated that the author gathered data from personal experience.

While the responses to these “how” questions had the fewest number of respondents (probably because time constraints kept some students from reaching these penultimate questions), the findings are notable because they indicate that many students are unfamiliar with scholarly article components. Although the time-constrained library session did not include specific instruction on defining or identifying the parts of scholarly documents (including abstracts and methodologies), students were asked whether the author included such components. They were also directed to ask the library instructor if they had any questions, and the researchers were readily available to provide any necessary assistance. Additionally, students had access to both Internet search engines and links to definitions of these terms. Yet, for the most part, students did not ask for help defining these terms. This is further evidence that students can identify a popular piece in aggregate, but they are largely unaware of the defining characteristics and categories of publications that help knowledgeable readers distinguish between document types and authorial processes.

These findings point to an illiteracy that is important to address. Lower division undergraduate students can clearly distinguish between a scholarly document and a popular one with little instruction. What students are missing, however, is an awareness of distinctions among the processes by which columnists, journalists, and researchers arrive at their conclusions and an ability to correctly classify or label opinion pieces from factual ones. This is of particular concern in terms of scholarly publications, such as The Lancet, in which letters to the editor often include citations and refer to the letter writers’ employment at universities or other research institutes. To a new student, such a letter could easily look like a scholarly research article as opposed to criticism of another researcher’s study. Without knowledge of publishing jargon and processes, students may find criticism and opinion pieces, such as book reviews and letters to the editor, indistinguishable from their research-based counterparts in a list of database search results. These critically important abilities were underdeveloped in these first-year students who were at the end of their first semester at the university, and these findings were consistent over a two-semester period.

The other challenging criterion, the “when” questions, proved difficult to students for two reasons. First, one “when” question asked students whether they needed to cite something recently published for their assignment or if a historical piece was suitable for their topic. Because students were not reviewing this document in connection with a particular research assignment, the question was irrelevant and confusing in this context. Second, and more significant, were student difficulties regarding when the events discussed in the document occurred. Though published in March 2011, the same month in which the Tōhoku earthquake and tsunami and Fukushima nuclear disaster struck Japan, the columnist did not explicitly state that his writing was prompted by those disasters. At the time of publication, most readers would have been bombarded with media reports covering those terrible events, but these students were evaluating the document eighteen months after the events (six months after for the pilot group), and they approached the “when” at face value, providing only the dates of earthquakes that were specifically referenced in the column (1995 and 1923). Less than half of student groups approached the question from the angle of a past current event; only 45.3 percent made a connection between the March 2011 disasters in Japan and the March 2011 column. The value of situating a publication in its appropriate context was a discussion point at the end of the library session, after students had submitted their responses via SurveyMonkey and presented their group’s findings to the class.

The third and final purpose of the formative assessment was to examine student arguments for why they would, or would not, cite the opinionated, popular source in a college paper. Following examination of the document using the six criteria, the activity concluded by asking students to articulate their overall impressions of the document, both verbally and in the written assessment. These reflections were valuable for both students and researchers in that they not only prompted students to consider the document holistically and for a definitive purpose, but also provided researchers a glimpse into students’ decision-making processes. For example, several groups thought that the author was a scholar, but because he was published in a newspaper, addressed a popular audience, and provided his opinion, the document was more of a popular source than a scholarly one. Thus, most groups (98.6 percent) capably weighed multiple criteria to accurately describe the popular nature of the document.

This holistic processing was again demonstrated in the final question, in which students were asked to state definitively whether or not they would cite the document in a college paper. The researchers deliberately left this as a “judgment call” to see if responses would speak to their abilities as first-year students to navigate the complexities of the evaluation process. There was no right or wrong answer for whether the source was worthy of citation in a college paper, and some students may have been influenced by the fact that citing popular sources was permitted in their composition assignment. Just over 67 percent of respondents provided a reasonable explanation for their decisions, often referring to the author’s credentials or the expression of opinion in the document as reasons for why they would, or would not, cite the material. Nearly half of all respondents provided a comparatively well-synthesized or nuanced justification.

As a result, the Five Ws can be considered an effective instructional scaffold and evaluation method. Results indicate that students were familiar with the Five Ws before attending library instruction, were able to apply it successfully during class, and that instructors unanimously found value in the method. However, one limitation of this study is its lack of comparisons among different types of evaluation methods. At this point, researchers are unable to determine whether the Five Ws is more or less effective or memorable than alternative methods, such as CRITIC. It is also unknown if composition instructors would have preferred or valued a different evaluation method over the Five Ws, though it should be noted that no instructor offered an alternative approach.

Regarding recall, the memorability and application of the Five Ws method was less successful than researchers originally hoped. Summative assessment results demonstrate that few students (7.5 percent) recalled the Five Ws evaluation method by name, indicating that the method is not overly “catchy” as a mnemonic device. In describing their own evaluation processes, however, most student responses (66.0 percent) suggest an internalization of some aspects of the Five Ws activity. After the library session, more than half of students (60.3 percent) reported researching the backgrounds of the authors they cited in papers. The 13.2 percent of students who considered more than one aspect of a source (but not all Five Ws) most often evaluated the reputation and reliability of both the author (who) and the publication (where). Consequently, while students may not have replicated the method in its entirety, many applied aspects of the Five Ws and understood it as part of an evaluation process.

The findings of the summative assessment also point to the value of collaborating with course instructors. The impact of library instruction beyond the one-shot session was enhanced by creating an assessment/activity that served as a skills development scaffold needed for established assignments. At the conclusion of the study, one composition instructor requested a copy of the Five Ws activity to use in her class, with several additional instructors indicating their intentions to incorporate aspects of the activity in future classes. Due to the demonstrated increase of memorability and application among students who received additional instruction on the method outside of the library, the researchers plan to utilize this collaborative scaffold approach as a model for other library instruction. Activities based on this model will be similar to the formative Five Ws assessment in their value to instructors and students, repeatability in a later nonlibrary class session, and ease of incorporation into an existing assignment.

To effectively prepare students for a lifetime of learning, it is essential that information literacy instruction sessions develop skills, such as source evaluation, that transfer beyond classroom walls. Although navigating the complexities of the modern search experience in a one-shot session can be difficult, learners are increasingly likely to encounter information sources, such as online journals, that defy traditional relationships between content and means of access; gone are the days when scholarly output was more likely to be found in print sources than online. As a result, using an evaluation method that works well with any source, regardless of means of access and retrieval, is vital. The information evaluation methods published in the library literature over the past decade tend to privilege distinctions in access—online versus print—over distinctions in document types (e.g., articles versus editorials). In doing so, these methods fail to emphasize the unique characteristics that well-informed readers use to distinguish between information sources, such as the inclusion of a research methodology or the author’s affiliation.

In this study, the Five Ws was introduced as a means of evaluating sources found regardless of format or mode of access. The researchers were primarily concerned with testing the memorability of the Five Ws evaluation method, instructors’ perceived value of that method, and its effectiveness as a scaffold. Although less than 10% of students recalled the Five Ws in its entirety, a majority used salient evaluation points from the method in their own research later in the semester. In addition, most instructors found value in the method and, in those classes where instructors reiterated the method outside of the library session, student retention and use of the Five Ws increased. These results suggest that instructional scaffolding is an effective way to overcome some of the many limitations of one-shot library instruction—including time restrictions and an of abundance of learning outcomes to address—by integrating library instruction into course-level instruction through an information literacy activity based on a concept familiar to students, and easily incorporated into course instruction and assignments. Though these results are promising, still more research is needed in the application of instructional scaffolding to library instruction and in collaborating with academic departments to teach the type of information evaluation students most need in the current information environment.

Although this study was primarily concerned with testing the memorability and value of the Five Ws, several unanticipated findings were discovered via the formative assessment. These findings include that the majority of students lacked the background knowledge necessary to differentiate among the information gathering techniques of various types of authors (e.g., journalists versus researchers). Additionally, students’ difficulties in explaining differences between scholarly articles, popular articles, and columns may have resulted from their lack of familiarity with the jargon and function of publication components, such as abstracts. As a formative assessment, the Five Ws activity did not address such gaps in knowledge, but instead identified their existence among students who were close to completing their first semester at a university.

For many lower division undergraduate students, general education courses punctuate the first two years of their college careers. Introducing evaluation and research skills in these general, interdisciplinary, required courses may help to equip students with the critical thinking skills needed to succeed in advanced and specialized courses. Undergraduates who are able to acquire and internalize skills for evaluating information at both source and document level may be more prepared for upper division courses in which evaluation becomes deeper, involving the comparing and contrasting of methodologies and scholarly findings within a particular field. The findings from this study suggest that there is a need for increased attention to developing these skills in general education courses. In particular, there is a need for ensuring that students look at documents not only from a narrow, disciplinary view, but also contextually, in an attempt to understand the greater forces at play in their creation. There is life-long value in ensuring that students not only summarize competently, but also analyze competently; that they not only understand what they read, but also recognize that there are people and processes involved in creating what they read. If that is indeed the case, there may also be value in assessing students’ knowledge of publication processes prior to entry into upper division courses.

Despite the low recall of the Five Ws in its entirety or by name, the overall effectiveness of the method has led researchers to continue to use, revise, and improve the Five Ws formative assessment for English 101/118 instruction sessions. Researchers have also made adjustments to the activity for use with high school library instruction sessions and have forthcoming plans to adapt the method for use in an online tutorial. In the near future, the researchers plan to share results of this study with the First-Year Composition department in an effort to support lower division undergraduate student learning. In the long term, the researchers hope that these findings will encourage more studies on information evaluation instruction and the role it might play in the development of information literate citizens and scholars.