rusq: Vol. 51 Issue 3: p. 235
Beyond the Search Box: Using Participatory Design to Elicit Librarians’ Preferences for Unified Discovery Search Results Pages
Stefanie Buck, Jane Nichols

Stefanie Buck is Ecampus and Instructional Design Librarian/Assistant Professor
Jane Nichols is Collection Development Librarian/Associate Professor at Oregon State University in Corvallis, Oregon

Abstract

New unified discovery tools are designed to help students easily identify and retrieve library content. Students want quick and easy discovery of quality sources in a “Google-like” environment. Librarians have often expressed frustration with federated and metasearch systems that cannot take full advantage of discipline specific database features. As new options, such as discovery systems, become available, it remains to be seen if librarians have the same concerns about these new tools. What librarians want in these systems is less well explored than our users. At a local conference, the authors used a participatory design strategy to elicit from the participating librarians their views on what a discovery system should look like and function. Groups of participants drew their idea of what a discovery system should look like. An analysis of the findings reveals what librarians think are important features for these tools. The authors also discuss the use of the participatory design process.


Afrequent complaint voiced by students and librarians about online database searching is the large number of choices that students have and, as a result, knowing where to start or which database or catalog to choose is not always easy.1 One of the early responses to this problem was federated search tools that query individual databases. While this was a step in the right direction, these tools are slow, draw on the lowest common denominator, and do not take advantage of some of the unique search features of a database.2 Now new tools are emerging. The most promising are unified discovery systems which aim to simulate the “one-stop shopping” experience of the Google search engine. These so-called next generation tools search indexed content, primarily journal content, rather than searching licensed databases, as well as local content and catalogs. They are faster than federated search tools and appear to be a step closer to students and researchers frequently expressed desire for one place to find all of a library's content with an easy to use interface.

From the literature and interactions with users, we have a good idea of users’ preferences for a single search option, but what librarians want in a discovery tool is not as clear despite their significant roles in tools selection, configuration, promotion, evaluation and use (through instruction and reference work). With the proliferation of new tools, such as Summon from Serial Solutions and EBSCOhost Discovery Service and Integrated Search, that hope to address users’ persistent concerns, librarians have many options for what their next discovery interface will be. The question of what librarians want arose when our institution and regional consortium began discussing whether to adopt a new discovery service. To begin learning about librarians’ preferences for a unified discovery search interface, we developed a short participatory design workshop held at a local conference, Online Northwest 2010, attended by regional librarians. This inquiry into librarians’ preferences for a single search tool may encourage librarians to reflect on their needs and wishes for a single search tool. Reflecting on both users and librarians’ preferences will help librarians evaluate software based on both groups’ requirements and choose software meeting each groups’ needs. In this paper, we describe the participatory design process, the conference workshop and discuss preliminary designs librarians created during the workshop to show their “ideal search interface.”


BACKGROUND ON FEDERATED AND DISCOVERY TOOLS

Federated search services such as MetaLib, WebFeat, E-Z Search, and LibraryFind are popular with students who like to search across multiple databases.3 Federated search engines typically provide access to a wide range of sources, including a local library catalog, subscription databases, and other local or national collections, but in the process lose some functionality. Federated search tools don't always have the ability to search by unique fields, or even sometimes, especially in early versions, the ability to use Boolean operators that are offered by subject research databases.

A unified discovery system, unlike a federated search tool that retrieves information from a select set of databases, searches pre-indexed metadata. A discovery system does not search “live” content but a pre-harvested set of information acquired directly from the publisher or data creator.4 Discovery systems mimic the Google-like single search box experience while giving the users easy access to paid library content, local digital content, and the library catalog. The major advantage here is speed, the ability to de-duplicate the search results, and, in some cases, more options for refining a search by author, subject, or content type than some federated search products provide. Of course, discovery systems are not flawless. Because they search an index, it is possible the content is older than what is most currently available directly from a content provider, although most discovery systems re-index overnight. These systems, however, show a great deal of promise in the quest to provide users with an easy way to search library owned or created content.


LITERATURE REVIEW
Federated Search Tools

What users desire from and how they react to a single search tool is well documented; speed, accuracy, relevant materials first, de-duplication, and an easy to search interface.5 However, the needs and desires of librarians, who are also significant users of the product, are less explored. Librarians have not always taken to federated search tools in the same way as their students.6 They often consider the unique fields or specialized search terms significant, especially for more advanced research, but also for undergraduate students who need to be able to develop the skills to extract the most relevant information from the datasets retrieved. With the loss of this functionality, students often retrieve huge amounts of information that is not necessarily relevant to their topic. For this and other reasons, many librarians are reluctant to teach the federated search tool that their library has. Lampert and Dabbour surveyed librarians nationally and held small group discussions; they found that two-thirds of the librarians, who were themselves confident in their abilities to search federated tools, did not teach federated search tools. The librarians disliked the lack of certain functionality. Lampert and Dabbour, like others, found librarians had somewhat ambivalent feelings about federated searches.7 Marian Hampton, guest contributor to the Serials Review column “The Balance Point,” noted that single interface searching across many sources is a benefit to metasearching but that librarians are “concerned that federated searching presents an illusion that information is simple to find.”8 In the same column, Pugh also commented on the tension between wanting to improve students’ information literacy skills and offering them an easy search interface.9 In their study of the MetaLib Combined Search (MCS), Tang, Hsieh-Yee, and Zhang found that librarian views were more negative of the usefulness of MCS when compared with student perceptions. The librarians found it “not useful, and hard to figure out.”10 While librarians have had both positive and negative reactions to federated search for a variety of reasons, specific concerns were the search interface and the results display. McHale pointed to the lack of date and peer-review filters, the lack of controlled vocabularies and indexes, and that most federated search tools do not include all databases.11 Lampert and Dabbour also pointed to the loss of a controlled vocabulary, the inability to limit to peer-reviewed, and the loss of individualized features of specialized search engines as reasons why librarians were not whole-heartedly embracing federated search tools. They also found that users have a hard time distinguishing between the types of resources retrieved.12

Design Guidelines and Best Practices

Designing a search interface is a lengthy, iterative process involving many stakeholders. Research on how users seek information is presented primarily in the information science and information architecture literature. Information science researchers have been studying how users seek information since before World War II, although this area of study flourished beginning in the 1950s.13 Early information seeking behavior studies were more concerned with the use of information sources and how users navigated an existing system rather than studying information needs. These studies led to a better understanding of how people seek information and how the design of a system can aid or hinder a user.14 As information sources went online, librarians began investigating how users interacted with databases, and later, webpages to locate information.15 These studies showed that an understanding of information seeking behavior can positively influence search interface design.16 New models of information seeking behavior are being developed as search engines, such as Google, change how people search and retrieve information.

Another important source for the design and layout of search interfaces comes from information technology literature. Usability testing, which software engineers have been conducting since the 1970s, has greatly improved how search interfaces are designed because they are based on actual tasks users perform. Usability studies originated with computer software design firms but were applied to web interface design in the early 1990s.17 Pioneers in usability testing of webpages include Jakob Nielsen and Jeffrey Rubin whose works are often cited in the library literature as well.18 While usability testing has focused more on webpages and web interface designs than database design, the principles of user-centered design are central to both. Librarians have taken these techniques, often developed to study consumer behavior, and applied them to library webpages and search interfaces.

A common theme among all these studies is that search behavior changes. Searchers will move between browsing and searching depending on the results of their search.19 Web interface designs also need to take into consideration that users bring their own experiences and expertise to a search.20 Researchers in information behavior emphasize that search is an iterative process so designs need to support users in refining and reformulating their search.21 The fact that different user behaviors may require different designs and layouts makes designing webpages a challenge. Over time, some de facto standards and best practices have developed to help designers create useful search interfaces. Kalbach summarizes different search behaviors and suggests design strategies that support each of these.22 Rose argues that search interfaces need to support the following information seeking behaviors:

  • Different interfaces (or at least different forms of interaction) should be available to match different search goals.
  • The interface should facilitate the selection of appropriate contexts for the search.
  • The interface should support the iterative nature of the search task. In particular, it should invite refinement and exploration.23

Layout and design of a page has a major impact on the users’ success in locating.24 In their book Search Patterns, Morville and Callendar lay out some of these de facto standards for the design of features and functionality that make for a good user interface.25 Another useful source for design guidelines is the Research-Based Web Design and Usability Guidelines from the U.S. Department of Health and Human Services, which distills much of the research on good web design into a single document.26 Russell-Rose describes the use of design patterns as a way to design a set of webpages that support different user search strategies and information seeking behaviors.

Although each website needs to be designed in context, there are a number of features and layouts that work well overall. Research supports the use of progressive disclosure so that as the user moves toward their desired target, the information provided by the system becomes more relevant and specific.27 Most design guidelines also strongly recommend the use of hierarchical faceted metadata which Morville and Callendar, corroborated by other researchers, refer to as “arguably the most significant search innovation of the past decades.”28 Other recommendations include the use of filter and branching to help users refine their search; content type and format are the most useful filters.29 This includes features like auto-complete or auto-suggest that provide immediate feedback to users. The system should also allow alternate views such as sorting by date or author.30 Hearst et al. recommend providing users with help and immediate responses. Layout too can influence user success.31 For example, placing the primary navigation menu in the left panel or along the top and placing navigation bars at the top of page layouts promote success.32

While information seeking behavior research and usability studies have had a strong influence on how search interfaces that meet different user needs are designed, the results of this research and these studies are not always systematically applied to the actual design of an interface. Search interfaces, Rose states, “often reflects the attributes of the tools used for searching, rather than about how people look for information.”33 Ferrara notes that search engines are often built for very narrow search strategies that basically support just keyword searching.34 This kind of design overlooks the basics of human search behavior.35

Participatory Design

Participatory design is a user-centered design method for involving users in a design process of an end product such as a webpage or a search engine interface. Participants act as co-designers at the design's formulation stage with a goal of creating a usable product incorporating new technology to meet their needs. Participatory design originated in Sweden as computers were being introduced into the work setting; key concepts included “democracy, power and control at the workplace.”36 Indeed, a participatory design process asks workers to voice how best to integrate technology into their work practices based on their knowledge, experience, and expertise. Bowler et al. highlight the iterative and circular aspects of participatory design and note the vital contextualization it brings to the design process—meaning that can be overlooked in usability tests.37 Bowler also notes that as “new approaches have been added to the collage,” the definition of participatory design has changed.38 Moreover, as the method has gained traction, its strengths and weaknesses have become apparent. Spinuzzi recommends it to “rapidly and incrementally improve” the product in question, to focus on “democracy in the workplace,” or to tap into users’ strengths. However, it is not the best method for securing user input when “radical change” is sought because of its focus on using an existing design, which can limit big picture thinking.39

Participatory design is a growing area of interest as evidenced by a conference devoted to the practice and numerous publications detailing practitioners’ experiences.40 Interest in and use of this method has also grown among librarians and libraries. The publication Studying Students: The Undergraduate Research Project at the University of Rochester,41 popularized this method in the library profession, generating enough interest that the editor, Foster, now offers workshops through The Council on Library and Information Resources for librarians wishing to learn more.42 Rochester is one of several libraries that has employed participatory design methods to improve library services; here we focus on libraries use to improve websites and users’ ability to access library provided content.43

University of Rochester Libraries (UR) 2007 ethnographic study of student research behaviors reported on their successful use of participatory design workshops to discover what students want from library space for research, how they research and write papers, and what they want from a library website.44 Though several methods were employed to gather student input, the participatory design workshops are most pertinent to this article. During the workshops students were asked to design a website from scratch and to redesign the libraries existing site based on paper mock ups. This study typifies their design process and the value of integrated and repeated use of participatory design. They continuously employ user centered methods to learn about their users to improve services. For example, they turned to participatory design workshops to learn about faculty and graduate students use of the library website and what their ideal site would include.45 The workshops helped “create an informational basis” for identifying key tasks when using the library website.46

To gain ground on solving seemingly intractable barriers users encounter when navigating libraries’ siloed content, Rochester partnered with research teams at Cornell University, Ohio State University and Yale University in the eXtensible Catalog project.47 The teams studied student and faculty research processes to learn more about these barriers and facilitated “a participatory, collaborative process of co-creation” to design a new software suite that will search across multiple content repositories.48 Their focus on work processes broadened their research horizon beyond a focus on tweaking features to understanding how students and faculty conduct scholarly work in an information environment where library provided resources share space with many other providers.

Nikolova-Houston used participatory design techniques to revise her websites featuring Slavic medieval manuscripts.49 She used three stages of participatory design: discovery, site evaluation, and site prototyping. Discovery included semistructured interviews, written surveys, classroom observation, and review of documents such as syllabi. Then, she asked her participants to evaluate her existing sites using common usability criteria and, last, she held one prototyping session. Participatory design “created a more intimate social atmosphere” between her and her participants and gave her confidence in her understanding of their tacit knowledge and future needs.50

Participatory design is a successful method for involving users in a design process to learn about their needs, thoughts and opinions about what they would like in a site and for demonstrating to them “that the online sources they design can be used for academic teaching and research.”51 However, additional research demonstrating the effectiveness of user created and inspired designs would be welcome.52


PRECONFERENCE SURVEY

Before the conference, we surveyed our potential participants to learn about their preferences and prompt thinking about what attendees would like in a next generation search tool. Our questions addressed their perceptions of and experiences with federated search tools currently in use and asked them to rate the usefulness and importance of various features. We also asked how often they used the tool, if they teach it, and to whom. We solicited their perception of how well the tool works and how it compares to other search engines or online databases.

Respondents (N = 36) were academic (53 percent) and public library librarians (30 percent). Just 24 percent did not have a federated search engine at their institution. Those that did have a federated search engine used MetaLib (24 percent), WebFeat (14 percent), and SIRSI Single Search (14 percent): 29 percent had their federated search tool for 1–3 years. Three of the respondents told us that their institution was in the process of moving to another system, but most were not looking to change at that time.

We also asked them about what features they considered important. They considered as essential the ability to locate the full text, restrict by peer-reviewed and content type, and narrow by date. They wanted the search results to display the title, author status, and availability of full text online. They also wanted to be able to sort by relevance, date, author, and title.

Many of the respondents were not active users of the tool: 46 percent only used it a few times a year, although another 46 percent used it on a daily or weekly basis. Several commented that they considered Google Scholar when connected to library holdings and subscriptions a better option than vended software. Others pointed out the familiar benefits of federated searching, such as searching multiple databases and having multiple facets to refine results. Weaknesses cited were also typical: slowness, confusing, or irrelevant results; poor relevancy ranking; and huge results sets that patrons have a difficult time interpreting. Some commented that they did not teach or demonstrate the federated search tool for these very reasons. One even said that the “search results with the federated searching products we had were miserable.” Another person commented, “Maybe we need to go back to the drawing board,” which is precisely what we hoped to do in our session.


SESSION STRUCTURE

At the conference, we divided our session into two parts. The session itself was one hour long. First, we presented an overview of unified discovery systems, how these differ from existing federated search engines, and introduced the concept of participatory design. This took approximately 20 minutes. The second part of the presentation was the participatory design activity; we allocated about 25 minutes for this part and another 15 minutes to allow the groups to share their work and discuss their drawings. Because our goal was to discover what librarians want in a federated or unified discovery system, we asked them to draw their ideas on a large sheet of paper. In the planning process we decided that rather than asking the librarians to draw their desired search interface, which would most likely be just a search box, we would ask them to draw the page that showed the results of the users search (results page).

The session was held in a spacious room with a dozen or so round tables. Participants formed self-selected groups of three to eight per group. We provided each table a large piece of flip chart paper, pens, double-sided tape, glue sticks and post-it notes of varying sizes. We created a large number of “widgets” of different sizes with labels based on the features respondents had rated important or very important in the survey as well as other features common to federated search tools (figure 1). Some of the widgets had text in them, such as “Link to My Homepage,” “Sort by,” “more like this,” or “page X of X” so the participants could easily paste one of these prefabricated widgets in their designs. They could also use blank widgets and write in a label. We provided screenshots of various federated search tools results screens with identifiers, such as tool and institution names, removed. Finally, we offered each table a list of common features and functions such as “ability to add a review,” “ability to tag,” “link to Google Scholar,” “tag cloud,” or “feedback or comments link.” The set of widgets, sample screenshots, and features and functions list were put into a packet and distributed with the intention of fostering creative thinking while helping to keep us within our time constraints. Then we opened up the session and let the groups get to work. After twenty-five minutes, we took a little time to let each group hold up their drawing and point out a few of their features.


LIMITATIONS

When we began this process, we knew we would be asking the participants to accomplish a great deal in a short amount of time. Designing an interface takes time and a great deal of usability testing and that was a luxury we did not have. We had only a short time to introduce them to the concept of participatory design. We had as many as eight people to a table, and group design is not always an easy process, therefore, we did not expect drawings to be fully formed. We included the prefabricated widgets and other aids so participants would be able to begin the design process relatively quickly. We did this primarily to serve as a “memory jog” but we realized that this would most likely influence participants’ designs. Ideally, we would have simply provided blank sheets of paper. Taking all this into account, we were struck by the obvious effort put into the designs.


COMPARING DESIGNS

After the conference, we took the seven posters back to our library (figures 2–4). We photographed each so that we could draw and comment on the different elements in the images and compare them side by side. After an initial review, we grouped features and functions into the following categories: search, refine, display, sort, access, help, save/print/e-mail, citation management, visuals, and miscellaneous items such as “links to other stuff” or “share this.” Some of the widgets could have multiple meanings, e.g., the “citation” widget could mean “citation displayed here,” “click here to see the citation,” or “format citation in a specific style.” We interpreted the intended meaning based on the widgets’ use and location in the context of the design. Layouts varied quite a bit in their complexity. One group simply drew a results box and with options to check each result as “like” or “dislike” similar to Pandora. Others offered a myriad of choices and often had functions, such as printing, located in multiple places so the user could easily locate the function. The groups used a combination of prefabricated widgets, sometimes with new text, and original widgets.

Though we did not methodically critique the drawings against design “standards” for webpages, many followed the de facto standards of good web design and layout.

Search

Even though we asked the groups to design the results page rather than the initial search page, all groups followed good practices by including a basic search feature on their designs.53 Some added features that reduce users’ cognitive load. For example, two groups included an additional place where the previous search displayed so users can see what was searched.54 It is possible that the other groups assumed that the search entered would remain in the search box. One group wrote the search phrase “Mark Twain” into the search box probably with this intention.

Surprisingly, only three groups offered a link to an “advanced search,” although they did not specify which fields they would have included. Our initial assumption was that this would be a very important feature since librarians have often commented on the lack of precision in federated search tools.55

Refine

All of the groups had options for refining the search; this is where we found the most creativity in the designs and it was clearly very important. The most common way of refining a search was through the use of facets usually displayed on the left hand side, a standard recommended by Morville and Callender.56 The most common facet was content type which Morville and Callender consider one of the most important refinement options.57 The most prevalent content types were book (n = 3), media (n = 3), and article (n = 2), but in some cases, the groups simply included a button labeled “content type” (n = 2) which leads us to believe that once the user clicked that button, he or she would be presented with the appropriate content types. One group added websites to the results, possibly as a way to direct users to related materials on the web. Only three groups included an option for limiting the search to peer-reviewed, which was probably related to time constraints but was a little surprising to us since it is a feature librarians have found lacking in some federated search tools. Two groups allowed users to refine searches to just full-text or what is “at my library.” Moreville and Callender consider format the other most significant way to refine a search.58

What we found most interesting was the number of groups (n = 5) that added a rating or voting system to indicate they wanted “more like this.” This widget was included in the packet, so we cannot say how many groups would have thought of this without a prompt, but it was clearly popular and is recommended by search interface designers as one way to direct users to the best content.59 One group used this feature exclusively, with no other options for refining search results. In this design, the user would refine results by marking them as “Hot” and “Not.” The other groups used terminology such as “more like this.” One group included a relatively sophisticated method to let the user rate a number of articles as “Thumbs up” or “Thumbs down” and then rerun the search using a “refine search using my likes.” One group included a “trash basket,” which we interpreted as a similar rating feature.

Sort

Permitting the user an “alternate view” is another feature web designers recommend as “the optimal view differs by user and task.”60 Most of the groups allowed a variety of ways to sort results. Five out of the seven designs used the prefabricated sort widget. In most cases, it was placed on the right or at the top of the screen, possibly indicating a need for high visibility. Another possibility is that this feature is often located at the top right in other databases and the groups followed this layout.

Access

All the groups except one devised ways for the user to access or acquire the actual article, book, media item, or whatever else might be displayed in the results list. Although it was not always explicit, we think that for most of the examples, clicking on the article title would retrieve the item, as almost all the results pages had some way to indicate if the full text was available. The participants used differing terminology, including “ours,” “full-text,” and “get it now,” to indicate availability. Some of the groups used the predesigned star icons or images of an article or a combination of image and text to indicate the content is readily available implementing two important design concepts, that of the meaningful label and the use of attention getting features.61 All groups except two had links to interlibrary loan services on the results page that indicated to us that an easy way to acquire an item was important. One group placed the item on the right side inside a menu along with items like “my lists,” “my library account,” “home to my library,” and “home to my university.” One group allowed users to share their results with others.

Display

Most groups had the results of the search, meaning in this case the citation or information about the article, book, or media item, displayed in the middle column of the page, which is consistent with most databases and other search systems. In most cases, the groups just indicated with a box—here is where the citation will display—and did not include the specific fields displayed, which we think is due to time and a general sense that we would want the standard information (title, author, journal, etc.) displayed. Only two groups specifically indicated that an abstract would be available which surprised us, but as with other features, we do not want to read too much into the absence of this feature. Certainly having “snippets” that preview the content is a useful feature.62

The groups designed a variety of navigation features to get the user to the next page, a specific set of results, to the library homepage, or to other sites. Several groups included a prefabricated “results x of x” in their design to let users know how many items their search had retrieved. The groups also found that getting the user to the university or library homepage was important. They also felt a link to “my library account” was an important navigational feature.

Help

Access to help was a commonly included and desirable feature.63 Four of the seven groups used a prefabricated question mark on their results page. We assumed the authors intended this to mean that clicking on it would take a user to search engine help pages, but we also realized it could link to a webpage featuring other help options like e-mail, text, phone, etc. Five of the seven groups indicated the ability to get immediate help was important by adding a chat widget. One chat box even had a slightly desperate “Help Me!” label attached. Another group supplied a way to report a technical problem.

Save/print/e-mail and Citation Management

Tools and techniques to help reduce cognitive load can help enhance the user experience.64 Keeping the search terms visible in a search box is an example of such a technique and one employed by the participants. Other techniques might include ways to save a search or item. Every group except one offered the user the option to print, save, and e-mail results, so this was clearly an important feature, as was an option to mark certain items. This feature was very visible on most designs. One group had a “drag and drop” where users could drag their selections to a folder rather than mark them with a checkbox.

Surprisingly, we did not see the ability to manage citations on many of the designs. Most groups allowed for printing, e-mailing and saving of items, and it is possible the designers meant to allow users to format the citations they had selected, but no group stated this explicitly. We did, however, see a number of groups who included the “citation” label widget on their designs, which could be read in different ways, so our interpretation can only be speculative.

Visuals

We were interested in how the groups would use visual cues on their designs. The Research-Based Web Design and Usability Guidelines recommend using attention getting features when appropriate to direct users to the right item.65 In the results section of the display, nearly every group included some text, image or combination of the two to help identify the content type of the citation retrieved. They also frequently included some kind of symbol, such as the star (n = 4) or an image of an article or journal (n = 3), to show that the full text was available.

When we examined the features that librarians had rated as essential or useful in our preconference survey, we discovered some interesting dichotomies (table 1). Of those features, only the ability to restrict by content type actually appeared on the drawings. The other features, such as refine by subject heading, ability to restrict by peer-review, date or location, which the survey respondents rated highly, showed up less than half the time on the drawings. This is inconclusive, but it did make us wonder what impact the act of drawing their ideal search tool rather than choosing from a predefined list had on the participants’ decision to include or not include something.


USING THE PARTICIPATORY DESIGN PROCESS

Reflecting on our experiences of the session and respondents’ feedback from the conference evaluation, we discovered a few challenges in gathering user feedback this way. Doing this within the context of a conference was risky for several reasons: we could not plan for a set number of attendees (how many widgets to make, how many people to put at one table, how to group attendees); and we did not have complete control over our environment (number and type of tables, session length).

We found that we provided the participants with too many of our prefabricated widgets. For some groups, it took too much time to sift through all of these, and it may have been easier for them to just create the widgets from scratch although at least one person asked for copies of our widgets to use in a similar exercise at her institution. As presenters at a conference, we did not expect to know the number of attendees nor was it within our purview. If we were to conduct this as a workshop, we might recommend people form groups of three to five people who work in a similar type of library. Twenty-five minutes is a short amount of time to try to do this exercise with a relatively large number of people. Using a typical workshop length of two hours would give attendees more time for the activity. It would also give attendees and presenters more time to share and reflect on designs created, as well as time to generate conclusions that all could take away.

On the whole, most of the evaluation respondents were satisfied (n = 25) or very satisfied with the session (n = 39). Eleven were neutral, two dissatisfied and one very dissatisfied. Themes in the comments showed that participants liked the audience interaction and the hands on work, “nice audience interaction” and “just needed more time for hands-on activity, but given the session time limit constraints, this was a very useful exercise.” Several liked the session but did not have enough time; a couple respondents mentioned the group work was not the best fit either for the material or due to library type differences; other respondents found the information presented either too little or too much. A couple felt the session was more for us, the presenters, than for them. Some attendees found the session a welcome change and a good opportunity to think about unified searching and seemed to appreciate that we asked for their design ideas. Although we guessed feedback would reflect strongly divergent opinions, we were heartened that the majority felt positive about the session.


WHAT DO LIBRARIANS WANT?

Though we cannot generalize our findings, we found the designs creative but also strongly influenced by designs currently in use. We also found that, while they did not address all of the current design standards, they did emphasize some of the more important components. Only one group created a very different design by doing away with all the facets and refining options except the “hot” and “not” option (relevance feedback). The rest of the groups thought the ability to refine the search using a variety of faceted options, the ability to get immediate help, and a direct link to the full-text or another way to get the item were very important. The groups also thought providing users with the option to print, save, or e-mail their results is essential. Like the minimalist group, four other groups viewed a rating system that let users rate their results as a means to refine their search, rather than or in addition to a content type, as a desirable feature.

The groups clearly valued some features more highly than others. They felt that having a search box, presumably to refine a search statement, on the results page was important. Access to help in some form appeared on almost every design. Most groups included multiple ways for refining a search in their designs so that the users could refine the search very precisely. Even so, having the ability to narrow a search by choosing like items or discarding irrelevant articles was also important to the groups, reflecting, we think, an effort on the part of the groups to think about how users may want to refine a search rather than how librarians want to refine a search. It could also be an acknowledgement that users don't always know what they are looking for so the other facets may not be as helpful to them. This topic warrants further research. The group also used visual cues (question mark icons, home icons, etc.) indicating to us that navigation and the ability to see immediately what kind of source it is or where to get help is an important topic. This was a paper prototyping exercise, so the results do not address librarians’ preferences or thoughts about the quality of the relevancy of results or the speed in which results are returned, two features important to students and librarians.66


CONCLUSION

As we write up our experience and findings, we ask, did our session work? Can we enumerate what librarians identify as the most critical features and functionality for federated search or unified discovery software? And what is the take away? We conclude that the use of participatory design to elicit librarians’ preferences for unified discovery search results pages can work. We were able to get a beginning sense of what librarians identify as the most critical features and functionality for unified discovery software. As the participants developed their designs they engaged in dialogue with their colleagues about which pieces to include, the purpose of each feature, its placement and the overall design. From the evaluations we learned that they felt the opportunity to think through and discuss each of these with their colleagues was a valuable experience. Even a year later, we have fielded comments and questions about the designs created during the session and about the participatory design process. In addition, we exposed librarians to participatory design and how a session could be structured. One librarian asked us for our “widgets” to use at her institution. Maybe one of our “take-aways” is just this: we value different kinds of opportunities for reflection, especially concerning tools key to our users.

It's clear we were quite ambitious to hold this session in the context of a one-hour conference session and would not do so again. Although most conference attendees liked having an interactive, participatory session, it didn't work for everyone. If held in a conference setting, a preconference would be a better choice (this one-day conference doesn't offer that option) because it would allow a longer time slot and more control over the number of registrants. We do suggest using this as a method to gather users’ feedback since the content we received was both informative and creative. Despite the time constraints, we felt the experience was worthwhile. We acknowledge that many of the designs are similar to those in existence, yet a few gems stand out.

In our preconference survey, one of our participants commented, “I have no idea what an acceptable library product really COULD offer, without becoming overly complex and complicated. I know we in libraries often want to be all things to all people, but maybe that's not possible.” Even if we do not know what the possibilities are, librarians clearly have an idea about what these search tools can and should do so. Not knowing the possibilities should not prevent us from participating in discussions about what we would like to see in these tools or advocating for features librarians consider important. In 2009, Carl Grant, President of ExLibris North America, posted to the Federated Search Blog some of his thoughts about federated and discovery system searching:

Libraries and searchers should seek and demand tools that maximize their flexibility to address search problems across ALL resources they've selected to meet end user needs. They should use their ability to organize on a large-scale basis to negotiate with content suppliers to provide that needed access.67

Using processes like participatory design, we can continue the conversation that will ultimately lead to better research tools for librarians and users.


References and Notes
1. John Boyd et al.,  "“The One-Box Challenge: Providing a Federated Search That Benefits the Research Process,”,"  Serials Review  (2006)   32:  247–54.Sara Randall,  "“Federated Searching and Usability Testing: Building the Perfect Beast,”,"  Serials Review  (2006)   32:  181–82.Tallant,  "“Metasearching in Boston College Libraries: A Case Study of User Reactions,”,"  New World Library  (2004)   105, no. 1/2:  69–75.Peter Jacso,  "“Thoughts about Federated Searching,”,"  Information Today  (2004)   21:  17–20.Carol Tenopir,  "“Database Use in Academic Libraries,”,"  Library Journal  (May 1999)   124:  36–37.Karen Eliason et al.,  "“Navigating Online Menus: A Quantitative Experiment,”,"  College & Research Libraries  (1997)   58:  509–16.
2. Paula J. Hanes,  "“The Truth about Federated Searching,”,"  Information Today  (2003)   20, no. 9:  24.
3. Lynn D.. Lampert and Katherine S. Dabbour,  "“Librarian Perspectives on Teaching Metasearch and Federated Search Technologies,”,"  Internet Reference Services Quarterly  (2007)   12:  253–78.Randall,  "“Federated Searching and Usability Testing,”,"    181Tallent,  "“Metasearching in Boston,”,"    69
4. Jason Vaughan,  "“Dispatches from the Field: web-Scale Discovery,”,"  American Libraries  (2011) Jan./Feb;  42, no. 1/2:  32.
5. Annie R. Armstrong,  "“Student Perceptions of Federated Searching vs. Single Database Searching,”,"  Reference Services Review  (2009)   37:  291–303.Susan Augustine and Courtney Greene,  "“Discovering How Students Search a Library Web Site: A Usability Case Study,”,"  College & Research Libraries  (2002)   63:  354–64.Jeffrey Belliston C.Jeffrey BellistonC. ,  Howland Jared L.Jeffrey BellistonC. ,  Roberts Brian C.,  "“Undergraduate Use of Federated Searching: A Survey of Preferences and Perceptions of Value-added Functionality,”,"  College & Research Libraries  (2007)   68:  472–87.Selkyung Jung et al.,  "“LibraryFind: System Design and Usability Testing of Academic Metasearch System,”,"  Journal of the American Society for Information Science & Technology  (2008)   59:  375–89.Tallent,  "“Metasearching in Boston,”,"    71
6. Nina McHale,  "“Why Reference Librarians and Instruction Librarians Hate Federated Searching and NextGen Catalogs,”"online posting, Web Services Librarian blog, Jan. 9, 2009, http://webserviceslibrarian.blogspot.com/2009/01/why-reference-and-instruction.html (accessed Jan. 23, 2011)..
7. Lampert and Dabbour, "“Librarian Perspectives,” 267; William J. Frost, “Do We Want or Need Metasearching?”,"  Library Journal  (2004)   129, no. 6:  68.Steven J Bell,  "“Submit or Resist: Librarianship in the Age of Google,”,"  American Libraries  (2005)   36, no. 9:  68–71.
8. BoydBoyd ,  et al.  "“The One-Box Challenge,”; " 250
9. Ibid., 252..
10. Rong Tang, Ingrid Hsieh-Yee,  and Shanyun Zhang,  "“User Perceptions of MetaLib Combined Search: An Investigation of How Users Make Sense of Federated Searching,”,"  Internet Reference Services Quarterly  (2007)   12:  229–30.
11. McHale,  "“Why Reference Librarians.”,"  
12. Lampert and Dabbour, "“Librarian Perspectives,”,"  :  263.
13. Wilson T. D.,  "“Human Information Behavior,”" in Informing Science: Special Issue of Information Science Research,   http://inform.nu/Articles/Vol3/v3n2p49–56.pdf (accessed May 21, 2011)..2000James Kalbach,  "“Designing for Information Foragers: A Behavioral Model for Information Seeking on the World Wide Web,”" in Internetworking,   www.internettg.org/newsletter/dec00/article_information_foragers.html (accessed May 21, 2011)..2000 Dec.
14. Wilson,  "“Human Information Behavior.”,"  
15. Marcia Bates,  "“The Design of Browsing and Berrypicking Techniques for the Online Search Interface,”,"  Online Review  (1989)   13, no. 5:  407–24.Peter Pirolli and Stuart K. Card,  Irvin R.. Katz, ed., with Robert L.. Mack;Linn Marks;Mary Beth Rosson and Jakob Nielsen, "“Information Foraging in Information Access Environments,”" in ACM Conference on Human Factors in Software (CHI ‘95) ,   (Denver, Colo.: ACM, 1995): 51–58, www2.parc.com/istl/groups/uir/publications/items/UIR-1995-07-Pirolli-CHI95-Foraging.pdf (accessed July 1, 2011)..Nicholas J.. Belkin, Pier Giorgio Marchetti,  and Colleen Cool,  "“BRAQUE: Design of an Interface to Support User Interaction in Information Retrieval,”,"  Information Processes & Management  (1993)   29, no. 3:  324–44.Gary N. Marchionini,   Information Seeking in Electronic Environments (Cambridge, Mass.:  Cambridge Univ. Pr., 1995): .David Ellis,  "“A Behavioural Model for Information Retrieval System Design,”,"  Journal of Information Science  (1989)   15, no. 4/5:  237–47.
16. Barbara M. Wildemuth,  "“Evidence-based Practice in Search Interface Design,”,"  Journal of the American Society for Information Science & Technology  (2006)   57, no. 6:  825–28.
17. Barbara J.. Cockrell and Elaine Anderson Jayne,  "“How Do I Find and Article? Insights from a Web Usability Study,”,"  Journal of Academic Librarianship  (2002)   28, no. 3:  122–32.
18. Ibid., 122.
19. John Ferrara,  "“Search Behavior Patterns,”" in Boxes and Arrows: The Design Behind the Design ,   (Jan. 30, 2008), www.boxesandarrows.com/view/search-behavior (accessed May 21, 2011).
20. Ferrara,  "“Search Behavior Patterns.”,"  
21. Kalbach,  "“Designing for Information Foragers.”,"  
22. Ibid.
23. Daniel E. Rose,  "“Reconciling Information-Seeking Behavior with Search Interfaces for the Web,”,"  Journal of the American Society for Information Science and Technology  (2006)   57, no. 6:  797–99.
24. EliasonEliason ,  et al.  "“Navigating Online Menus,”,"  :  509–16.
25. Peter Morville and Jeffery Callender,   Search Patterns (Sebastopol, Calif.:  O-Reilly, 2010): .
26. U.S. Department of Health and Human Services, Research-Based Web Design and Usability Guidelines  Washington, D.C.:  GPO, n.d., 
27. Tony Russell-Rose, "“User Interface Design Patterns for Search and Information Discovery”" Website: Thoughts on the Intersection of User Experience, Search, Text Analytics & More ,   (July 19, 2009), http://isquared.wordpress.com/2009/07/19/ui-design-patterns-for-search-information-discovery (accessed May 21, 2011).
28. MorvilleMorville ,  Callender,  Search Patterns, 95.Marti Hearst et al.,  "“Finding the Flow in Web Site Search,”,"  Communications of the ACM  (2002)   45, no. 9:  42–49.
29. MorvilleMorville ,  Callender,  Search Patterns, 95.Jody Condit Fagan,  "“Usability Studies of Faceted Browsing: A Literature Review,”,"  Information Technology and Libraries  (2010)   29, no. 2:  58–66.Max L.. Wilson, M. C.. Schraefel,  and Ryen W. White,  "“Evaluating Advanced Search Interfaces Using Established Information-Seeking Models,”,"  Journal of the American Society for Information Science & Technology 60, no   7, 2009:  1407–22.
30. Morville,  Callender,  Search Patterns, 68.
31. EliasonEliason ,  et al.  "“Navigating Online Menus,”,"  :  509–16.
32. Morville,  Callender,  Search Patterns, 97.
33. Rose,  "“Reconciling Information-Seeking Behavior with Search Interfaces for the Web,”,"  :  797.
34. Ferrara,  "“Search Behavior Patterns.”,"  
35. Wildemuth,  "“Evidence-based Practice in Search Interface Design,”,"  :  826.Eileen Abels and Marilyn Domas White, Karla Hahn,  "“A User-Based Design Process for Web Sites,”,"  OCLC Systems and Services  (1999)   15, no. 1:  35.Rose,  "“Reconciling Information-Seeking Behavior with Search Interfaces for the Web,”,"  :  797.Kalervo Järvelin and Peter Ingwersen,  "“Information Seeking Research Needs Extension towards Tasks and Technology,”,"  Information Research  (2004)   10, no. 1http://informationr.net/ir/10–1/paper212.html (accessed June 20, 2011).
36. Douglas Schuler and Aki Namioka,   Participatory Design: Principles and Practices (Hillsdale, N.J.:  Lawrence Erlbaum, 1993): .Tatiana Nikolova-Houston,  "“Using Participatory Design to Improve Web Sites,”" in Computers in Libraries,   ,   www.infotoday.com/cilmag/oct05/nikolova-houston.shtml (accessed June 20, 2011).Nancy Fried Foster, Nora Dimmock,  and Alison Bersani,  "“Participatory Design of Websites with Web Design Workshops,”" in The Code4lib Journal,   Mar. 4, http://journal.code4lib.org/articles/53 (accessed June 22, 2011).2008
37. Leanne Bowler, Sherry Koshman, Jung Sun Oh, Daqing He, Bernadette G.. Callery, Geof Bowker,  and Richard J. Cox,  "“Issues in User-Centered Design in LIS,”,"  Library Trends  (2011)   59:  734.
38. Ibid.
39. Clay Spinuzzi, SusanJones BSusanJonesB ,  Beth Weise MoellerSusanJonesB ,  Michael PriestleySusanJonesB ,  Bernadette Long,  "“Investigating the Technology-Work Relationship: A Critical Comparison of Three Qualitative Field Methods,”,"  IPCC/SIGDOC ’00 Proceedings of IEEE Professional Communication Society International Professional Communication Conference and Proceedings of the 18th Annual ACM International Conference on Computer Documentation: Technology & Teamwork  (2000) New York:  ACM, :  419–32.
40. Finn Kensing and Jeanette Blomberg,  "“Participatory Design: Issues and Concerns,”"; in Computer Supported Cooperative Work,   www.pdc2010.org (accessed June 15, 2011).1998 167-85
41. Nancy Fried Foster and Susan Gibbons,   Studying Students: The Undergraduate Research Project at the University of Rochester (Chicago:  Association of College and Research Libraries, 2007): .
42. Council on Library and Information Resources, "“CLIR Workshops on Participatory Design in Academic Libraries”"CLIR Activities, www.clir.org/activities/details/faculty.html (accessed July 2, 2011).
43. Libraries have used participatory design and similar techniques for other purposes: MIT Libraries sought to understand how graduate and undergraduate students access information via a 2006 Photo Diary Study where students used cameras or took screenshots to capture their work processes.Nicole Hennig et al.,   User Needs Assessment of Information Seeking Activities of MIT Students ,   (MIT Libraries, July 2006), http://dspace.mit.edu/handle/1721.1/33456 (accessed Aug, 1, 2011).Rochester used a work practice study to learn about the disconnect between libraries quick adoption of institutional repositories and faculty comparatively slow rate of depositing materials.Nancy Fried Foster and Susan Gibbons,  "“Understanding Faculty to Improve Content Recruitment for Institutional Repositories,”" in D-Lib Magazine,   http://webdoc.sub.gwdg.de/edoc/aw/d-lib/dlib/january05/foster/01foster.html (accessed July 6, 2011).Jan. 2005Participatory design methods were used in the design of digital libraries in Pauline Ngimwa, Anne Adams, and Josh Underwood, "“Collaborative Ownership in Cross-Cultural Educational Digital Library Design,”,"  Research & Advanced Technology for Digital Libraries Lecture Notes in Computer Science  (2009)   5714:  239–49.The design and test of a digital image library was described in Claudia Roda, "“Digital Image Library Development in Academic Environment: Designing and Testing Usability,”,"  OCLC Systems & Services: International Digital Library Perspectives  (2005)   21, no. 4San Jose State University Library used collaborative design (co-design) user-centric investigations in their information commons redesign; see Mary M. Somerville and Lydia Collins, "“Collaborative Design: A Learner-Centered Library Planning Approach,”" Electronic Library postprint available online at SJSU ScholarWorks, http://hdl.handle.net/10211/71 (accessed July 6, 2011).2008
44. Foster,  Gibbons,  Studying Students.
45. FosterFoster ,  DimmockFoster ,  Bersani,  "“Participatory Design of Websites.”,"  
46. Ibid.
47. eXtensible Catalog Organization University of Rochester, "“Take Control eXtensible Catalog,”"www.extensiblecatalog.org (accessed Aug 1, 2011).
48. Nancy Fried Foster et al.,   Scholarly Practice, Participatory Design and the eXtensible Catalog (Chicago:  ALA, 2011): , vii, 18..
49. Tatiana Nikolova-Houston,  "“Using Participatory Design to Improve Web Sites,”" in Computers in Libraries,   www.infotoday.com/cilmag/oct05/nikolova-houston.shtml (accessed July 6, 2011).2005
50. Ibid.
51. Ibid.
52. Only one study was found that measured participatory design's effectiveness to improve an interface design: Fabio Brunoa and Maurizio Muzzupappa, "“Product Interface Design: A Participatory Approach Based on Virtual Reality,”,"  International Journal of Human-Computer Studies  (2010) May;  68, no. 5:  254–69.
53. HearstHearst ,  et al.  "“Finding the Flow in Web Site Search,”. "47; U.S. Department of Health and Human Services, Research-Based Web Design and Usability Guidelines, 182.
54. MorvilleMorville ,  Callender,  Search Patterns, 57.
55. LampertLampert ,  Dabbour,  "“Librarian Perspectives”,"  Peg Marshall, Shawn Herman,  and Sri Rajan,  "“In Search of More Meaningful Search,”,"  Serials Review  (2006)   32:  175.Xiaotian Chen,  "“MetaLib, WebFeat, and Google: The Strengths and Weaknesses of Federated Search Engines Compared with Google,”,"  Online Information Review   30, 2006:  413–27.
56. MorvilleMorville ,  Callendar,  Search Patterns, 97.
57. Ibid., 88.
58. Ibid.
59. Hearst,  "“Finding the Flow in Web Site Search,”,"  :  43.Morville,  Callender,  Search Patterns, 58–59.
60. MorevilleMoreville ,  Callender,  Search Patterns, 72.
61. U.S. Department of Health and Human Services, Research-Based Web Design and Usability Guideline, 64.
62. Moreville,  Search Patterns, 112.
63. U.S. Department of Health and Human Services, Research-Based Web Design and Usability Guidelines, 21.
64. MorevilleMoreville ,  Callender,  Search Patterns, 76.
65. U.S. Department of Health and Human Services, Research-Based Web Design and Usability Guidelines, 11.
66. Jung,  "“LibraryFind,” 387,"  
67. Carl Grant,  "“Beyond Federated Search—Winning the Battle and Losing the War?”" "online posting" in Federated Search Blog http://federatedsearchblog.com/2009/03/30/beyond-federated-search-%E2%80%93-winning-the-battle-and-losing-the-war (accessed Feb. 14, 2011).Mar. 30, 2009

Figures

Figure 1

Examples of Widgets



Figure 2

Group Drawing 1



Figure 3

Group Drawing 2



Figure 4

Group Drawing 3



Tables
Table 1

Important features


Answer Options Preconference Survey Responses (N = 15) Drawings Used Feature in Design (n = 7)
Essential/Useful Neutral Marginally Useful Unnecessary
Citation management 12 (80%) 0 1 0 4 (57%)
Ability to locate only full text 13 (87%) 0 1 0 3 (43%)
Restrict to peer-reviewed/scholarly publications 15 (100%) 0 0 0 3 (43%)
Restrict by content type 12 (80%) 3 0 0 6 (86%)
Restrict by language 12 (80%) 3 0 0 2 (29%)
Refine by subject headings 14 (93%) 1 0 0 3 (43%)
Narrow by date 15 (100%) 0 0 0 2 (29%)
Restrict by format 12 (80%) 2 0 0 3 (43%)
Restrict to a certain location (branch, collection) 12 (80%) 2 1 0 0
Features not on initial survey but considered important
Link to interlibrary loan 5 (71%)
Help 5 (71%)
Voting/rating system 5 (71%)


Article Categories:
  • Library Reference and User Services
    • Features

Refbacks

  • There are currently no refbacks.


© 2017 RUSA