MLA 2012: Through the User's Eyes

Through the User’s Eyes: Discovery Tool Usability

Lucy Holman – Director, Langsdale Library, University of Baltimore
Elias Darraj – DCD Candidate, Interaction Design and Information Architecture, University of Baltimore
Jonathan (Yoni) Glaser – MS Candidate, Interaction Design and Information Architecture, University of Baltimore


  • Users prefer Google-like interface, one-stop shopping.
  • Students driven away by library websites & databases due to lack of intuitiveness
  • Federated search using Z39.50 – search across platforms. But very slow – literally searching each one each time.
  • Discovery tool will “pre-harvest” into a centralized index – much faster.
  • Products started debuting in late 2007 into early 2009
  • USMAI: 16 libraries with one ILS, consortial purchases, cross-functional groups
  • 2010/2011 decided to look at discovery tools for the system
  • Fall semester – webinars & demos for 6 of the tools. Narrowed to EDS, Summon, Encore and Primo.
  • Lucy was asked to do a major usability test on them (she teaches in an info architecture & interactive design program) – one track is User Research Design  – user-centered design, conducting focus groups, task-based usability testing. Design with user in mind.
  • Focused on general search behaviors & use of particular features.
  • Audiences: undergrad, grad, faculty. 6 groups of graduate researchers who each worked with 3-4 participants. Each participant worked with 2 tools. Recruited throughout the system, though most were from UB as that’s where they did the testing.  21 participants – 8 UG, 7 grad, 6 faculty. (In the task-based usability world, you can generally ID major issues with 5-8 users.)
  • ID’d 6 tasks and built scenarios around them. 2 known item, 2 topic sarches, item save & retrieval.
  • Observations took place in April 2011 in UB’s usability lab. Students used Morae screen capture software and Toby eye tracking software.
  • Summon & Primo gave access to real campuses (Summon: U of AZ, Primo: Boston College). EDS & Encore had a generic trial database. EDS branded as UGA. Encore’s set up created a lot of problems for users as it forced them to go outside of the system to actually find stuff. Not included in results.


  • EDS: Intuitive, minimalist. Facets on left. Consistent, detailed information. “Right-side blindness” for available tools (aka “ad blindness”). Search results were consistent. Inconsistent terminology (“refine search” vs “search options”). Screens gray out as they update, which threw some people off. Have found the item but can’t figure out if it’s in the library. Look at accession number, look at ISBN – but library holdings info isn’t on the record view. Easier to do a known item search – they want the ISBN, author or title. Easier to *not* use the advanced search screen.
  • Summon: Overwhelming main interface but results page was pretty clear. Inconsistent detailed record. Video of a user who had no idea where to start on that homepage. Detailed record view changes depending on where the record is coming from – need to wrap it into a unified interface. Folder for saved items is only at bottom of page – people missed it.
  • Primo: Advanced option right form the start. Very clean. “Didn’t expect it to have so few options on the advanced search page.”


  • Faceted search very important for discovery tools. Much easier for libraries than for Google – leverages metadata.
  • Most relevant filter should be nera top of the page. Provide many filtering options, but not all at once. Display on the left side (“right side blindness”). Allow multiple filter selections before updating results. Strong visual cues for which filters are applied. Avoid jargon in category names.
  • We’ll seem very critical but they are doing different things well.
  • EDS: Contextual facets with important stuff at the top, but limited filtering options. Required to actively select “update” to apply filters. Weak visual cue for what’s selected. User: “Where is the faceted search, I know this is supposed to be helpful.”
  • Primo: Hyperlinked – only one option at a time. Date ranges are limited – pre-set in strange increments. Exhaustive list of facets, users had to scroll down. Show the most relevant and have a “see more” hyperlink to unfold the full list. Unfamiliar terminology. User who had a result set and picked a filter that expanded it.
  • Summon: Strong visual cues for selected filters, but no author filter. Updates automatically based on filter selection (but this can be hard on your web services). But the results page blacks out with a note while it reloads.
  • None are perfect. Need more user-friendly categorization. Consistency between advanced search & facets. Remove unnecessary “fail points” like Summon’s reload screen. Some did have a good order on facets though – most used at top, least used at bottom.
  • Discovery tools are an attempt to mimic what Google has done – now there’s a need for collaboration. Education is a collaborative effort. You want to be fast but maybe not the first in the market.
  • Save & Share functionality. Many different ways to do this – flexibility is a must. Multiple save options, exporting to a variety of tools, confirmation of save. Simple retrieval. This is not the make or break on your decision making, though.
  • EDS: Visual cues go unnoticed, but good confirmation.
  • Primo: Was on the right hand side, not good. Multiple options & good confirmation.
  • Summon: Strong visual for save, multiple options to save/export.
  • Used infrequently and with a wide variety of different uses. Placement and look of save icon is important.


  • We focused on what we thought was the most important functionality, but others are focused on volume and type of content that the tool has ingested, compatibility with existing systems.
  • We feel usability should be a big driver though.


  • Differences among the populations you tested with? If you had to choose, what would you choose? Lucy: Regarding audience, faculty tended to use the advanced options more often & quickly. Very few UGs ever did. You have to think about what the most novice user will be able to handle – the more experienced will find the advanced functionality. USMAI went with EDS but hasn’t implemented it yet. A lot driven by another consideration – cost. Did feel that the usability was good.
  • Did you share this with the vendors? Can you use it as a bargaining chip? Lucy: Interesting. Some of the vendors were a little too interested in getting the info. Others not so much. Some were very proprietary about it. Chapter in a book coming out in June.
  • Homepages are busy – better to take this off the homepage? Lucy: Provide strong visual cues – they will go to what stands out most on the page. Clearly above the fold, avoid the right side, prominent visual presence.
  • Disconnect between library products & web design best practices. Communication problem with getting end-user voices up through the chain of command?  Lucy: Excellent observation. Vendors consider that their users are librarians – so they cater to us. We need to know our users and advocate for their needs. Personally feels that it’s up to us to test, get feedback, and relay to vendors.