The first survey was directly distributed to students at two different ALA-accredited LIS programs. It was also distributed to faculty at four other LIS programs, with the request that they distribute it to their reference classes. The survey contained 26 questions, and the general purpose was to ask how they felt about their reference instruction and assignments. Questions included how often students went to the library for their reference assignments, how far they had to travel to get to an academic library, and whether visiting the library helped them understand sources and the nature of reference work.

The second survey, which was directed at practitioners, aimed to provide a more critical view of LIS education, questioning how their employers and colleagues in libraries perceive the skills and abilities of students who had recently completed their schoolwork. Invitations to participate in the survey were distributed at two state library association conferences, which were attended by public, academic, and special librarians. To get results from a wider representation of reference librarians, the survey was distributed to two electronic mailing lists, PUBLIB (for public librarians) and LIBREF-L (for reference librarians). We received 285 total responses. We do not have number of reference librarians in the states where the survey was distributed, subscribers to PUBLIB, or subscribers to LIBREF-L. Because we do not know how what the population is, we cannot make any sampling determinations.

The practitioner survey had two foci: their own practice of librarianship, and what they seek from or how they regard new reference hires. Regarding their own practice, questions included:

• How is their reference model structured (i.e., tiered or traditional)?
• What reference sources were utilized most frequently?
• What types of questions are they most often asked (i.e., in-depth research, or factual)?
• Do they tend to use more print sources to answer questions, or more online sources? Of the online sources, do they use free or subscription sources more frequently?

Questions regarding new hires included:

• Are new hires regarded as bringing adequate skills and abilities to use electronic and print sources?
• Do they feel that new hires are adequately trained to use both print and electronic reference sources?
• What sources were critical for new hires to master; were there certain tools that new hires need to know about?

Many questions included a comment section so practitioner respondents were able to clarify or qualify their answers.

After the surveys were analyzed for results, interviews with practitioners were performed to address some inconsistencies between practitioners’ views. Participants were from two academic libraries and three public libraries in the state of Missouri. Interviews were semistructured, with three initial questions about reference practice. However, discussions ranged widely, to allow the participants to discuss the issues that were most important to them. Questions focused on what sources the librarians most often used, what reference students ought to be taught, whether students could find those sources in the practitioner's library, and whether it was important for students to learn about print resources.

Most of the students who responded to the survey responded positively. They felt that they were learning a lot of new things and were pleased that they had been required to use both print and electronic reference sources. As shown in table 1, most felt that assignments involving print resources helped them understand the nature of reference work, understand reference sources, and how those sources are organized. Over half of all respondents were not working in libraries, and the assignments requiring the use of print sources gave them an opportunity to develop skills in the reference section of the library. Some students reported discussing the assignment with librarians, to find out how a professional might go about finding the answers. But because student respondents had not worked as reference librarians, their feelings of satisfaction or dissatisfaction with the curriculum were, in many ways, divorced from the context that reference work would otherwise provide.

The accessibility of print and online sources influenced students’ use of these sources. Students reported that it was less time consuming to find information online. Some reported that they just couldn't spend many hours in the library to do an assignment, so they would make a quick tour of the reference section and get out of it what they could after finding the answers online. Another student reported that there just weren't many good print sources available at the library, which necessitated searching online. When given the choice, students were more likely to turn to online sources than print sources to complete assignments, even when they were required to use print resources (sometimes comparing organization, etc., to their online sources). Most students began with online sources, though some began with print sources and two used only electronic sources, as shown in table 2.

Students were asked whether they were familiar with print reference resources before taking the reference class. As shown by table 3, most students indicated that they were vaguely familiar with print resources. Table 2 notes that a majority of students said that print sources should be required for reference homework assignments, despite the difficulties of accessing these sources.

To gauge whether students’ employment status affected their class engagement and ability to review print reference sources, we asked students if they worked and what their jobs were. Table 4 demonstrates the number of students employed in full-time and part-time jobs, including whether those were library-related jobs or outside jobs. Jobs for those who worked outside the library included full-time parent, auto mechanic, psychologist, writer, etc. Table 5 indicates the library departments that reference students worked in. Most of the students who worked in libraries were employed in Technical or Access Services.

Students employed full-time may have schedules that conflicted with library open hours. Students from rural areas have limited access to libraries with extensive print reference collections, and this is a particular concern when online programs are marketed as being available anywhere and at any time. Students were asked how far they had to travel to get to a library, and whether they felt they had to travel a long way (to ask them to provide both an objective and subjective measurement of distance). Table 6 shows the distances students had to travel. Further, it wasn't always the case that students who worked in libraries had easy access to reference sources. One respondent reported that she wasn't allowed to use the library for school while at work, which didn't leave her many hours there to do schoolwork.

The majority of practitioner respondents, 71 percent, worked in public libraries. Only 27 percent worked in academic libraries, and four respondents worked in another library type. Patron interactions were usually in person or synchronous, as shown in table 7. Many said that most of their patron interactions are in person, and the second most often used method of communication was telephone. Chat, video chat, and e-mail reference made up a smaller percentage of total time. Public librarians reported mostly face-to-face and telephone interaction, while academic librarians reported face-to-face and chat/instant messaging. “Other” types of reference interactions reported were through the mail, through Facebook, and through personal interactions at instructional sessions. Table 7 also shows how practitioners devoted their reference time. While most of the librarians devoted less than 25 percent of their time to in-depth reference consultations, there were some respondents who spent most of their time doing so. Most practitioners still spent a good deal of time answering factual questions, while technology issues were perceived as consuming the most time. Public librarians reported a fair number of directional and technology questions; academic librarians were more likely to report in-depth research consultations.

When asked what source format librarians preferred to use with patrons, 55 percent (112 out of 205) of respondents preferred online subscription services. Twenty-four percent (49 out of 206) of the librarians chose free Internet sources, while 22 percent (44 out of 203) chose print sources. Both academic and public librarians preferred to use fee-based databases over print or web resources. The most important print resources were general dictionaries and medical resources, while sources of least importance were almanacs, reader advisory sources, and geographic sources. The relative importance of general sources like dictionaries and encyclopedias may stem from their generality: they are useful in many situations and with many audiences. By contrast, the diversity of scores on reader advisory sources suggests a difference between public libraries, where leisure reading is emphasized, and academic libraries. Academic librarians viewed subject-specific encyclopedias, subject-specific dictionaries, and medical sources as the most important print sources. Public librarians’ preferred print sources were medical sources, legal sources, and general encyclopedias (see table 8).

When asked to rank traits and abilities they sought when hiring a new reference librarian, the most important quality was possessing excellent interpersonal skills, selected as “very important” by 97 percent of respondents. Ability to formulate effective online searches was the second-most important factor. In fact, academic librarians preferred the ability to formulate effective online searches slightly more than good interpersonal skills, while public librarians’ top two preferences were interpersonal skills and effective online searching. The third-most important was awareness and ability to use free web resources. Awareness and ability to use fee-based databases was fourth-most important, and technological ability (other than searching) was fifth-most important. Only then does awareness and ability to use print resources make its appearance in the list, and only managerial experience ranked lower than ability to use print resources (see table 9).

A majority of practicing librarians believe that new hires are adequately trained in using print sources, and more believe they are adequately trained in electronic sources (see table 10). However, most practicing reference librarians felt that new hires did not know the best sources to answer reference questions. This was viewed as a skill that would come with additional on-the-job experience and familiarity with the library's unique collection. One comment, typical of almost forty others, was, “So much of what we do and how we answer questions has to do with what is unique to a particular collection and the patron or patron group being served. In the beginning sometimes these are not apparent because this is a learned skill set.” Another respondent commented, “New hires know what their professors tell them, and this is often woefully out of date.”

Some librarians who did not think that students are adequately trained thought that it was due to the online format of the class: “Many programs have become online only. It is very hard to teach these skills virtually and often I run into LIS students who are doing research to learn to use tools in my public library stacks. They are not always equipped well through that online experience.” Some commenters noted that new hires were more comfortable with electronic sources than print. One recently employed librarian responded: “No attention was paid to print resources. And although they are not used as frequently as on-line databases, I would have liked to learn at least something about how to use them.”

Although 58 percent of practicing librarians felt that new hires were adequately trained on print sources, there were still several comments that indicated perceptions of print as either a dying source or an ignored source. Comments from practicing librarians indicated that print sources were used less frequently in libraries, and that they appeared to be taught less frequently as well. When asked what print, fee-based database, and free web sources every new hire should know, we received literally dozens of unique answers, suggesting the regional nature of reference source suitability. Nonetheless, there was some consistency among answers. Table 11 shows the most-frequently provided answers. Multiple respondents noted that some sources are easier to use in print, and many lamented that the Statistical Abstract would not be released in a print edition in the future. However, several respondents noted that they were phasing out or sharply weeding their print reference collections, and that they did not often use their print collections. Some respondents said that print was not emphasized enough in LIS education, while one respondent noted that there is “too much focus on using print in library school.” Regarding web sources, the most frequently recommended sources for new hires to learn were government web sources. Medical and legal sources took precedence here. Respondents who commented on fee-based databases emphasized that students should have experience searching the most popular database aggregators, while specific databases were less emphasized.

Practicing reference librarians had quite a bit more confidence in students’ ability to use electronic resources, including databases and the free web. Even here, though, they acknowledged that using sources on the job was necessary to developing a fuller understanding.

Much of this experience comes from on the job training; database training can show specific search strategies but much is gained by actually searching for information for a customer or for a specific purpose. Artificial scenarios don't always replace the real reference interactions.

Students did feel that what they learned in class regarding databases was invaluable:

Even though I personally have no excuse for not knowing about the wealth of sources available in proprietary databases, there was a lot I was not familiar with. Being exposed to many of those that I was not using before has really changed my approach to research.

Google received positive and negative reviews among the practicing librarians. While one commenter suggested that “the world is attached to Google,” another said that it was the first “source,” in quotation marks, that new hires always turn to.

Interviews were conducted to explain some of the inconsistencies that were in the survey data, especially as it compared to the literature regarding reference. The first interview was with two academic librarians who work in primarily commuter college campuses. They both use online resources much more frequently than print, and reported that much of their reference work does take place online. They use a variety of tools (especially Meebo) to instant-message with their patrons, even those who are in the library (one said that the students don't want to lose their place and lug all of their books to the desk to ask a question). Their online patrons want answers quickly, and they want online sources. Their in-person reference model also has evolved—they both use a consultation model for in-depth research, which they say is actually more important now than the limited interactions involved in answering questions on the reference desk. One of the academic librarians reported that they have reduced the size of their reference collection by approximately 90 percent, replacing it with couches and group study spaces. Volumes in the reference collection that were not out of date now circulate. The other librarian said that her library had not done this, but she hoped they would.

Three public librarians were interviewed, a business librarian from a large metropolitan library, the head of reference from an urban library, and the head of reference from a medium-sized public library. Public librarians agreed that new hires should be well-versed in print reference sources—especially those with enduring value. Their reported use of space was quite different: they had not significantly reduced the size of their reference collection, and they did not have plans to do so. Most of their reference work took place in person. While they reported that they often consulted online sources (especially databases) for reference questions, they also said that print resources were important. Getting rid of print resources was viewed as fiscally irresponsible because during times of financial crisis (if, for instance, they had to slash their budget allocation for databases), they could still function if their reference section was intact. One librarian did say that the library circulates more single-volume reference works (such as subject encyclopedias) than they did in the past. This allowed them to bring out volumes that had previously been in closed stacks, so their total reference space allocation was unchanged. There was not much consensus between academic and public librarians about sources or space allocation.

Both academic and public librarians agreed that there are some sources that are no longer needed in print, such as directories. A few exceptions were noted, such as print directories that provide added value, or directories with historical value (such as historic city directories). Both groups cited the invisibility of online reference sources. While the online sources offer advantages of access and ease of use over print resources, they are not used much, justifying their continued expense. When they were asked about the most important resources for new librarians to know, academic and public librarians agreed: law and business sources were both often in demand and difficult to learn. Students need to be able to use them.