Usability Evaluation

Usability evaluation was conducted for sixty two digital libraries recommended as candidates of well-designed digital libraries by the content quality evaluation. The Usability Evaluation was completed from approximately March 2010 to October 2010 with the Usability Evaluation Criteria:

Accessibility: whether users can access information of a digital library with no or at least limitations without errors;

For Interface Usability:

Convenience/ ease of use: whether it is easy for users to learn and use the digital library in browsing and scanning (how much easy it is to navigate to from most pages).
Interfaces’ consistency: whether it uses the same words, color, font, graphics, and layout among web pages.
Visible design and Aesthetic Appeal of a digital library: whether it communicates clearly and visibly the purpose and value of the interface’s components with unique and descriptive headings and visibility, and whether it has attractive aesthetic appeal.

What is Usability?

ISO defines usability as “the effectiveness, efficiency, and satisfaction with which specified users achieve specified goals in particular environments” (ISO), where:

Effectiveness means “the accuracy and completeness with which specified users can achieve specified goals in particular environments;
Efficiency means “the resources expended in relation to the accuracy and completeness of goals achieved”;
Satisfaction means “the comfort and acceptability of the work system to its users and other people affected by its use” (ISO).

What is Usability Evaluation?

In Usability.gov website, usability evaluation is generally referred to “how well users can learn and use a product (e.g. a website, a software application, or a digital library) to achieve their goals and how satisfied they are with that process” (Usability.gov). Hilbert and Redmiles define usability evaluation as “the act of measuring (or identifying potential issues affecting) usability attributes of a system or device with respect to particular users, performing particular tasks, in particular contexts” (Hilbert & Redmiles, 2000).

The usability testing goal is to identify any usability problems on participants’ performance such as time on task and error rates. The methods to evaluate usability are Heuristic Evaluation, Card Sorting, Contextual Interview, Focus Groups, Survey, Prototyping, and Use Cases (Usability.gov).

What is Purposes of Usability Evaluation?

The main purpose of usability evaluation is to evaluate whether well-designed digital libraries (WDDLs) is accessible to users readily. For the purpose, the Usability Evaluation Criteria have two main criteria: Accessibility and Interface Usability. First, accessibility criteria are essential for the International Open Public Digital Library (IOPDL), because “Open” in the IOPDL means that anyone enables to access to WDDLs with no or few limitations. Thus, accessibility is one of the most important criteria to evaluate the candidate digital libraries. Also, for ease of use purpose, interfaces’ usability should be considered, because “Public” in the IOPDL means that all ages over the world can use easily WDDLs. Thus, Interface Usability Criteria have three sub criteria to evaluate their easiness: Convenience (ease of use), Interfaces’ consistency, and Visual design and Aesthetic Appeal. These criteria are chosen, because they are commonly used in the usability evaluation, and because they are suitable to evaluate many digital libraries’ usability, and related closely to the purpose of the IOPDL.

What does Usability measure?

Usability.gov explains common five factors are measured in usability testing: “Ease of learning (how fast can a first user learn it satisfactorily well), Efficiency of use (once an experienced user has learned to use the system, how fast can he or she accomplish tasks?), Memorability (“if a user has used the system before, can he or she remember enough to use it effectively the next time or does the user have to start over again learning everything?), Error frequency and severity (errors rate, and how do users recover from these errors?), Subjective satisfaction” (Usability.gov).

In detail, Saracevic (2000) lists usability criteria applied in various studies:

For content of a portal or site: “accessibility, availability, clarity, complexity of organization or structure, adequacy, coverage, quality, accuracy, authority and reliability.
For process carrying out tasks (e.g. search, navigation, browsing, finding, evaluation or obtaining a resource): learnability, convenience, ease of use, confusion, completion, interpretation difficulty, and error rate.
For format: attractiveness, consistency, representation of labels and messages’ communicativeness.
For overall assessment: satisfaction, relevance, usefulness of results, barriers, quality of experience, learning, and preferences” (Saracevic, 2004, p. 6).

Among above criteria, we should choose several important criteria, because we cannot evaluate sixty three candidate digital libraries by many criteria.

1. Accessibility Evaluation

Accessibility evaluation in Usability Evaluation Criteria investigates limitations and error degrees in accessing a digital library. W3C Web Accessibility initiative provides many tools that have been developed to evaluate accessibility automatically. Seven web accessibility evaluation tools are selected to evaluate many digital libraries in a short time. Read More… about methodology, results, and analyses of accessibility evaluations for sixty three digital libraries.

2. Interface Usability Evaluation

Interface Usability Evaluation is emphasized as a door to the content contained in the digital collection (Nielsen J. , Usability engineering, 1993). As much as interfaces play important roles for digital libraries, interface usability evaluations are important for users to use them with facility. Because we cannot evaluate sixty three candidate digital libraries by many criteria, we should choose several important criteria. Three interface usability criteria are chosen emphasizing importance of ease of use, consistency, and visual design, because they are main factors to engage users to navigate readily websites. These three criteria will check whether a digital library can engage users without troubles or difficulties. Read More…. about methodology, results, and analyses of interface usability evaluations with three main criteria for 63 candidate digital libraries.

Usability Evaluation