Designing technology solutions to be “user friendly” for a wide array of people with different technology skills is a difficult task. Interest in designing computers around the needs and abilities of their human users became an important topic with Licklider’s (1960) paper “Man-Computer Symbiosis,” which called for heightened awareness of the relationship between design and user.² During the late 1970s and early 1980s, the creation of IBM’s first personal computer ushered in a new era in which novice users of technology had access to computers.³ Many users had problems using this new technology, and companies started designing their products to be easier to use; designing technology to be more user friendly became a priority, but companies quickly realized that most programmers and engineers were not very effective at understanding how to design technology for the novice user. Thus, the field of human-computer interaction (HCI) emerged.

According to Eason, HCI specifically seeks to address six factors in the human-computer interaction: safety, utility, effectiveness, efficiency, usability, and appeal.⁴ A central tenet in designing technology interfaces within an HCI context is that the people who will be using it must be consulted from the very beginning. This is referred to as User Centered Design (UCD), which is “the practice of creating engaging, efficient user experiences” and places the human user as the starting point for designing effective technology solutions.⁵ UCD requires a systematic process of analysis, design, and development that involves iterative testing with representative users at each phase. Although many developers tend to think of effective web design in aesthetic terms, the functionality of a website’s interface design and information architecture are equally important and need to be specifically designed for a targeted group of users.⁶

The degree to which users seeking information find a website relevant and easy to use reflects the site’s general usability; the International Organization for Standardization (ISO) formally defines usability as the “extent to which the product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”⁷ Usability applied to web design treats it as a software development project rather than a mere “intuitive determination of user friendliness.” ⁸ As Nielsen points out, “Usability allows us to make everyday life more satisfying by empowering people to control their destiny and their technology rather than be subjugated by computers.”⁹

According to Nielsen, methods for user testing were well established in 1983 by John Gould and Clayton Lewis, who held that there were three primary principles of usability:

• 1. Establish an early focus on users and run field studies before starting any design work.
• 2. Conduct empirical usability studies throughout development.
• 3. Use an iterative design process.¹⁰

Theories of information seeking behavior serve as an appropriate window into understanding the information needs of users on the web. Taylor defined the information need of users as a “vague sense of dissatisfaction” wherein their active cognitive state is troubled by a “certain incompleteness in his picture of the world.”¹¹ Belkin, Oddy, and Brooks refer to this information need as an Anomalous State of Knowledge (ASK), which begins a negotiation process between user and information system; users actively begin seeking information in what Dervin refers to as a “sense making” process where “knowledge is the sense made at a particular point in time-space by someone.”¹²

Pirolli and Card developed an information foraging theory to describe information seeking using a hunting analogy—information needs create a hunger that starts a cognitive hunt. Their behavioural approach to information seeking is referred to as the adaptive control of thought in information foraging (ACT-IF) model in which humans are informavores constantly on the hunt for the information they seek.¹³ Morville and Rosenfeld believe the information architecture of a website should be developed with clear and strong information scents so users can quickly identify the right pathways to find what they are looking for.¹⁴

According to Lazar, usability is much more important for websites because they are different (than traditional information systems). Because the user may access a Web site infrequently, the site must be easy to use each time it’s accessed. Web users must be able to figure out immediately how to use an interface. If previous knowledge is required, the interface is confusing, information is not easy to find, or the user has to ask for outside assistance, he or she might visit another Web site because there is virtually no cost involved in switching.¹⁵

The web is designed for user control and autonomy. Nielsen states, “The original ideology of hypertext and the World Wide Web, as expressed by Vannevar Bush (1945), Ted Nelson (1960), and Tim Berners-Lee (1991) makes individual users the masters of the content and lets them access and manipulate it in any way they please.”¹⁶

Nielsen argues that site designers must always remember what he calls the Jakob Law of the Internet User Experience: “users spend most of their time on other websites.” His research has found patterns in user behavior in specific domains (e.g., investors and financial analysts) and he feels it is possible to “derive high-level design patterns for other domains as well. Such patterns must both retain sufficient flexibility and give users a sense of consistency and mastery in the things that matter.”¹⁷

Burnett and Erdelez found that researchers in the field continue to make an impassioned plea that “we remember the centrality of the user in information provision”; it is important to recognize the relationship between physical and architectural design of a system and its impact on how users seek information. The visual cues offered to a user will explicitly influence what users perceive to be available and a match to what information they are looking for.¹⁸ Richardson suggests that there are actually three interrelated elements necessary for successful provision of reference services to users: information resources, information technology, and users.¹⁹

While usability as a concept can be extremely complex to understand and consider in terms of application and implementation, it is actually quite elegantly simple to apply three basic processes:

• 1. Identify and engage representative users as design partners from the very start of the project.
• 2. Iteratively test at all design and development stages, including paper copies detailing only the information architecture and mockup of a projected digital environment.
• 3. Continuously improve, refine, and collect representative user feedback.²⁰

Jordan defined two categories of usability evaluation and testing—empirical (with representative users) and nonempirical (without representative users). Empirical usability testing methods include the use of focus groups, surveys, interviews, and usability tests with well-defined metrics (qualitative and quantitative data) and performance tasks (e.g., task analysis, task completion, time to complete tasks, etc.). Nonempirical methods include creating feature checklists (what features are most important?), task analysis (what tasks are most important?), and cognitive walkthroughs (what are the most efficient pathways to information?).²¹ Hornbaek approaches this dichotomy as the difference between usability evaluations that represent user perceptions, which are subjective, and those that do not, which are considered objective.²²

Alshamari and Mayhew classified usability evaluations into four types—model/metrics-based (use model to general usability measure), inquiry (communicate with users to derive insights into usability problems), inspection (experts use the interface to try and find problems), and testing (collect data by having users attempt to complete tasks).²³ Furthermore, they contextualize general usability evaluation as a complex interaction between six interrelated factors—evaluator, users, tasks, reporting, test environment, and prioritizing problems found.²⁴

Nielsen’s five user principle suggests that 85 percent of usability problems can be found by testing with only five users, and this has been supported by multiple research studies.²⁵ This principle based on a combination of probability theory and empirical evidence has been found to have significant caveats based on the unique contexts and differences of individual users.²⁶ According to Alshamari and Mayhew, “it can now be concluded that if the website has different types of users, it is vital to consider user numbers and their characteristics seriously.”²⁷

Hornbaek examined 180 usability studies published in the literature and classified their usability measures using the ISO 9241 definition of usability: effectiveness, efficiency, and satisfaction.²⁸ He found six major issues with current usability evaluations:

• 1. Measures of the quality of interaction (assessed by experts) are used only in a few studies.
• 2. Approximately one quarter of the studies do not assess the outcome of the users’ interaction with the system they are testing.
• 3. Measures of learning and retention are hardly employed.
• 4. Some studies treat measures of how users interact with interfaces as being synonymous with quality-in-use.
• 5. Measures of user satisfaction with interfaces are in disarray and most studies ignored validated questionnaires available.
• 6. Some studies mix together users’ perceptions of phenomena with objective measures of the phenomena.²⁹

In reviewing the literature, there have been many studies describing usability testing and research done on singular institutional websites for one individual library; however, broad studies focusing on public and academic libraries are not common. A study done by Liu examined 111 Association of Research Library websites and found that the sites tended to have search, resources by subject or subject guides, about section, library services, site search, ask-a-librarian, news or events, and contact us. Some common design patterns were columns by category and mouse-over links with sidebars.³⁰ Solomon conducted a survey of public library websites in Ohio, using a checklist of 61 usability guidelines, features, and content. Solomon found that only 35 of the 211 websites surveyed met 80 percent of her criteria, and she noted that important features were missing such as privacy policies, site searches, and feedback mechanisms.³¹

Additionally, usability studies have shown that creating websites with usability guidelines are important as “patrons who cannot successfully complete specific tasks may not revisit the site.”³² In a survey of web developers in academic libraries, Connell found that only 46.8 percent of them had conducted usability testing of any kind on their websites.³³ Chen, Germain, and Yang found that 49 percent of the Association of Research Libraries’ libraries had web usability policies, standards, or guidelines, and 85 percent of those libraries had conducted usability testing on some part of their website.³⁴

Shieh and Liu noted that the quality of the information architecture greatly influences the user’s experience and satisfaction with a library website.³⁵ King suggests that one must first envision a site as a business with information being the product: “Usability studies play a vital role in making sure library users can find information on your Web site quickly and accurately.”³⁶

Further studies gave recommendations for what a library website should look like. Liu found that “the universe of information presented on academic library homepages still focuses on library functions, requires numerous pathways for access, has overwhelming options, and takes a ‘one-design-for-all’ approach that fails to recognize users as individuals.”³⁷ Liu further recommended changing this to reduce the intimidating appearance of library homepages by “employing an appealing graphical design that accommodates usability and accessibility requirements.”³⁸ Other researchers, such as Poll, created lists for the main topics users expect to find on academic and library websites and summarized the main checklist points for a library website of high quality: adequate language (to the population), clear structure, options for different user groups, up-to-date information, and short, concise information.³⁹

A comprehensive review of the literature revealed, however, that no large-scale study had been conducted to determine the current design and usability of academic and public library websites. Unanswered questions such as, “did sites follow a consistent design,” “did they conform to website design guidelines,” “what content did they possess,” and “who designed and maintained them?,” helped inform the study’s five research questions:

• • RQ1: What is a standard design layout for academic and public library websites?
• • RQ2: What are the common features and content academic and public library websites include?
• • RQ3: Who designs and maintains academic and public library websites?
• • RQ4: To what extent do academic and public library websites adhere to recommended design guidelines?
• • RQ5: What is the general usability of library websites?

The Library Website Usability Checklist (LWUC) is a website evaluation tool designed specifically for the study containing 67 questions divided into five discrete sections—site information, recommended website features, content, feature placement, and recommended information architecture and usability factors. Section 1 collected general information about the site being evaluated: library name, URL, and webmaster email address. Section 2 comprised 19 recommended website features from the extant literature (e.g., navigation, search tools, and graphic design elements) adopted from three studies: Raward, Solomon, and Neal and Herzig.⁴¹ Section three was a checklist of 28 types of library web content recommended by previous research (e.g., location information, a link to the online public access catalog [OPAC], and describing circulation information) adapted from five studies: Raward, Solomon, Neal and Herzig, Poll, and Duncan and Holliday.⁴²

Section 4 sought information about the location and placement of five standard web features and content—library name and logo, search box, main navigation tools, library location information, and library contact information. Section 5 addressed factors of information architecture and usability identified by Morville and Rosenfeld, which involves nine essential questions they believe comprise a quality homepage.⁴³ Usability was assessed for each site by attempting to measure the degree of effectiveness, efficiency, and satisfaction based on whether each member of the research team was able to answer the nine questions.⁴⁴ Effectiveness was further broken down into task completion and quality of output, while efficiency was more specifically defined as deviations from the critical path, error rate, time-on-task, and mental effort. Each of these factors was rated on a scale from 1 to 10.

To increase inter-rater reliability and protect against threats to face and construct validity, three members of the research team tested the checklist on the same website and then compared responses for each question. By comparing ratings for each question, each member was able to clarify a common understanding of each question and increase precision in terms of what specific website features were appropriate to count for each checklist item. The process was repeated with four additional library websites until inter-rater ratings were 95 percent similar for subjective questions and within 1 point for scale items.

The researchers randomly selected four library websites from each state and the District of Columbia. Websites were stratified into four categories: one rural public library (a population of 25,000–50,000), one urban public library (a population of 50,000 or more), one private academic library, and one public academic library. See table 1.

The public library website for each site was identified using the Public Libraries (www.publiclibraries.com) and Library Sites (www.librarysites.info) websites. If the city chosen did not meet the needed stratification of either urban or rural, the process was repeated. The selection process continued in this manner until one urban library and one rural library were selected for each state.

Academic libraries were randomly selected from a Microsoft Excel spreadsheet downloaded from the Carnegie Foundation website that lists every private and academic university in each state in the United States and Washington, DC. For the purposes of viability, only four-year institutions that awarded bachelors, masters, or doctoral degrees were included in the sample. This process was repeated until one private and one public academic library were selected for each state plus Washington, DC.

Across a two-month period, 203 library websites (4 libraries for each state plus 3 libraries from Washington, DC) were evaluated using the LWUC evaluation tool by a member of the research team.

The Library Website Survey (LWS) was emailed to 9,000 academic and public libraries across the United States with a response rate of 14.1 percent or 1,266 respondents. The email distribution list was collected through a combination of each state’s state library website and the American Library Directory. This represents a valid sample of the nation’s libraries at a 99 percent confidence level with a sampling error of 3.4 percent.⁴⁵

The survey represented a revised version of the LWUC evaluation tool with most of the sections being condensed. The instrument comprised 44 questions broken down into 5 sections: general information (4 questions), web design and management (5 questions), feature checklist (5 questions), content (22 questions), and page location and placement (8 questions). Additional questions asked the libraries for general demographic information such as library type, size of user population and categorization of library (urban or rural, and public or private), and information about how their website was designed and managed.

There was a high degree of agreement in homepage design elements found between library responses and researcher evaluations. Main navigation was located at the top center (38.4 percent) or left side (36.3 percent) of the page. Surprisingly 37.8 percent reported not having a search feature available on the homepage, and 30 percent of search tools were located at the top right of the page. Library names and logos were located either at the top center (45.4 percent) or top left (43.3 percent) of the page. Locations of library contact information was the most varied as the majority were found at the bottom center of the homepage (21.6 percent), followed by not at all (16.2 percent), top center, side left, top right, and the center of the page. Library location information, such as address and directions, were either not present (21.4 percent) or located predominately at the bottom center (18.7 percent) or top center (15.3 percent) of the page.

Our randomly selected, independent evaluations aligned well with the survey results. Table 2 indicates the most frequently selected web element for both the surveys and evaluations and overall average of the two.

Figure 2 shows the homepage design trends graphically.

The results suggest that the majority of library websites have four common design features: Main navigation that tends to be horizontal and located at the top center of the page or vertical on the left side of the page; library logos that are located at the top of the page, which are either centered or at the left top corner; contact and location information that is centered on the bottom of the page footer or on a left side bar; and a search feature, when available, tends to be usually found at the top right of the page.

Both academic and public library websites were found mostly to be uncluttered with clean graphics and no splash pages or unnecessary graphics; they tended to be organized in a logical, hierarchical fashion. The use of color was also effective as most sites had high contrasts between site backgrounds and texts and consistent use of font styles and text formatting for increased readability. They also were compatible with multiple browsers and had user-friendly headings, so it was easy to know where one was when navigating library websites.

Based on the results using the library website usability checklist, the majority of library websites, similarly to the findings of Liu, readily provided contact information, directions, hours of operation, and access to their OPAC.⁴⁷ Other frequently found content and features included access to patron circulation information; library policies; the ability to renew, reserve, or checkout books; information on children/youth services; and access to electronic content. Information about the library, other branches, library news, and general library services were also typically available.

Content and features not as readily available included the date of page creation or updates, copyright notices, opportunities to provide feedback, frequently asked questions (FAQs), information about special collections, ready access to Web 2.0 tools and to social networking such as Facebook, Twitter, blogs, photos, RSS feeds, and libguides. Information or availability of virtual reference services through email, instant messenger, and video were also often hard to find.

Based on our representative sample, the majority of library websites are designed and managed by librarians who work on the website part-time as part of their regular librarian duties (50 percent). The second highest category, “other” (18.4 percent), represented a combination of volunteers, the state library, or entire web committees. Given the complexity and rapid changes involved in designing and maintaining effective websites, libraries might consider more dedicated resources to meet the growing online demands of patrons. Most websites were, however, recently designed or redesigned in the past two years (54.3 percent), which suggests that continued improvement and refinement of library websites is occurring for over half the participating libraries.

The library websites evaluated in this study followed a typical homepage layout and were somewhat aligned with design guidelines as proposed by Raward, Solomon, and Neal and Herzig.⁴⁸ Navigation was located either at the top center or left side and the name of the library or logo was also above the navigation top center or top left of the page. Information that was missing, however, included basic yet important information for users—library contact and location information.

HCI guidelines call for organizations to study how their users are interacting with and impacted by the technology that they are using, and user-centered design (UCD) principles call for representative users to be included in all phases of the design and development process, so that their user experiences are “engaging and efficient.”⁴⁹ The finding that more than 70 percent of libraries who responded to our survey have not conducted usability testing supports Connell’s findings that it is not a priority in the majority of libraries and suggests that most libraries are not practicing HCI guidelines or UCD principles, which require direct user input and testing.⁵⁰ This also indicates that other facets of usability design standards and heuristics, along with recommended web design stages, may be unknown and unused, reflecting what Liu referred to as the “one-design-for-all” approach.⁵¹

Despite this finding, preliminary analysis of library website homepages, both through survey self-reports and evaluations of a random sample of library websites in every state across the United States using the library website usability checklist derived from the literature and developed for this study, suggest that there are some common design conventions being followed. For example Morville and Rosenfeld recommended nine homepage questions should be easily answered with a high level of success on most library websites and six of the nine were consistently answered on over 80 percent of library websites.⁵² Preliminary usability evaluation suggests that library homepages are relatively high in effectiveness, efficiency, and satisfaction, at least in terms of being able to answer questions around the information architecture guidelines of well-designed homepages from the perspective of the researchers.

It is essential to emphasize though that this does not reflect the perspective of library users. Only 30 percent of participants reported that their website had been tested for usability, which suggests that the perspective of users does not appear to have been systematically taken into account. The usability for general users remains unknown, although the principles of HCI and UCD suggest that it is difficult to achieve high levels of usability with any level of specificity around unique users without close collaboration with the users a site seeks to serve.