Usability tests galore!

In an effort to improve users' experiences with the Duke Libraries website, library staff have conducted a number of usability tests over the last several.  The Digital Projects Department paved the way in testing interfaces and making changes based on users' experiences, but in the last six months, public services librarians in Reference and Instruction & Outreach have completed a number of studies.

We started with a test of the Search Resources tabbed interface, which we found is the primary reason that users visit the Libraries homepage.  Four librarians interviewed ten undergraduates and ten graduate students and proposed a number of fairly simple fixes that we hope will improve the overall design:  Eliminating the Google Scholar tab and including text in the search boxes to help users understand what they're actually searching are two changes that are coming soon... 

Math and Physics Librarian Michael Peper and I then headed to the Bryan Center (Duke's student center) to poll users about their experience with the Research by Subject page, which Michael and I helped redesign this past summer.  This study was significantly simpler than the Search Resources test -- we wrote the script, conducted the tests, analyzed our data, wrote our report and shared our results in about three weeks.  Changes that resulted?  Not many (our work with students affirmed that the page works fairly well) -- we added a chat widget and rewrote text to clarify the purpose of the search box. 

Next up, SILS field experience student Alice Whiteside and I tested the LibGuides interface early this semester -- you may recall that I blogged about our work last month.

Finally, Head of Reference Jean Ferguson, Divinity Librarian Luba Zakharov and I decided to follow up on our Search Resources tabbed interface study by exploring further how users interact with the tab labeled "Articles" on the Libraries homepage.  We observed 18 students and staff use the Articles search tool to find resources on topics of their choice and are in the process of analyzing our data.  Stay tuned for the results.

So what have we learned from all of this testing and analyzing?  Here are my top five lessons learned:

1. Usability tests take considerable time and effort -- each requires a lead person who is committed to making the test happen in a timely fashion.
2. Specialized software (we use Morae at Duke) is not always necessary -- a test can be conducted with nothing more than a laptop for participants and a laptop for taking notes (candy for participants doesn't hurt!).
3. Guerrilla tests seem to work best at Duke -- we go to high-traffic areas (the student center, Perkins library's coffee shop) and ask students to chat with us for a few minutes ( a good reason to limit tests to 5-10 minutes).  Scheduling tests ahead of time adds another layer of complexity -- and a significant amount of time and effort.
4. Scripting is the most important part of the entire process.  If you're not working with a script that truly gets at the questions you wish to ask, your test will not give you the results you want (seems obvious, but we've struck out with more than one test question...).  Have several people take a look at the script, and schedule a pilot before actually beginning your testing.  
5. Share your results!  Don't get bogged down in writing long, cumbersome reports that few will read.  Write up your results in a couple of pages, append your test instrument, and get it out there so that you can begin to make changes to your interfaces and webpages.  There's no point in conducting these tests if you're not going to use what you learned.

What have you learned from usability testing at your institutions?  We'd love to hear from you!

Help us serve you better, honors reseachers

Last week, Diane blogged about Duke's User Studies Initiative and mentioned two of the user studies that have resulted from it.

I have worked closely with one those studies and have had the privilege of learning from nine motivated seniors who plan to graduate with distinction in May.  In addition to maintaining at least the minimum GPA determined by their departments, these nine students are conducting research under the direction of a faculty member and writing honors theses based on that research.

In 2004, approximately 12% of Duke's students graduated with distinction, and the university set out to double that number within the next several years.  In 2009, nearly 23.6% of seniors graduated with distinction, closing in on the university-wide goal.

In order to support this university priority, the Libraries began offering special services to support honors researchers, ranging from increased borrowing privileges (honors researchers enjoy the same check-out periods as graduate students at all campus libraries) to a designated group study room (Bostock 112, for those of you who are interested).

Last spring, we decided to learn more about honors students' research processes:  Which tools do students use to find resources and organize their research?  Do they tend to meet with a librarian at some point during their year of research?  Where does the library fit into their research, both in terms of services and resources?  We hoped that this information would help us improve our existing services and perhaps even add a few new support mechanisms.

After several discussions with the dean who supports honors researchers and the Directors of Undergraduate Studies in English, history, public policy, biology and program II, we recruited a total of nine students from these departments (our goal was to interview students from a range of disciplines to ensure that we learned about the research habits germane to humanists, social scientists and natural scientists).

Each student agreed to be interviewed at the beginning of his or her research, at the midpoint and then directly following completion.  Using a script that we tested last spring, I have interviewed all nine students at least twice and have completed the study with the three public policy students who submitted their theses in December.

Chatting with these highly engaged students has been a joy, and I am indeed learning more about what the libraries can do for them.  Some things that I'm hearing most frequently?  Students would like more support in analyzing their data (and are thrilled to learn about Data Services); they really use Perkins Library as a space to write and would love for it to be open 24-7; and students are fairly low-tech (think binders, Word docs and stacks of PDFs rather than Evernote and RefWorks) in their approaches to organizing their research.  

Stay tuned for more details as we conclude our interviews this spring and analyze our data this summer...

The "U" Word

This week, several of us at Perkins gave a presentation at the LAUNC-CH Conference on creating a user-centered library.  Our presentation title was "Know Thy User: the Duke Libraries User Studies Initiative" and we talked about the initiative as a staff development effort and looked at the two studies that are underway as part of the initiative.

Our presentation title comes from Microsoft’s Arnie Lund, who famously exhorted “Know thy user, and you are not thy user.”  The User Studies Initiative is designed to get us past assumptions and presumptions about our users by building staff competence and confidence in doing user studies.  We’re trying to create a group of library staff who have the skills and comfort level necessary to do quantitative and qualitative studies – not necessarily large generalizeable ones, but local studies that can help us make changes to services and collections.   

The heart of the presentation was reports by Linda Daniel, Shawn Miller, and Emily Daly on the two studies they’re conducting.  Emily’s study looks at students writing honors theses at the beginning, midpoint and end of their research and writing process.  Linda and Shawn interviewed faculty and grad students in the Cultural Anthropology department to better understand their research and teaching processes, using a research approach adapted from the "Multidimensional Framework for Academic Support" conducted at the University of Minnesota.

The “U” word – users – is on our minds these days.  Actually, it's good to think of multiple “U” words: user, usability, usage.  We tend to conflate user studies with usability testing and analysis of use of collections and services, but they are different concepts that demand different kinds of studies.  Improving the user experience is a strategic direction in the Libraries new Strategic Plan, Sharpening Our Vision.  We want to “understand library users’ research and library experiences and use that information to shape collections, spaces and services.”  The User Studies Initiative is one way to help us accomplish that goal.

Ever noticed those "get it at Duke" buttons in Wikipedia?

If not, you might consider installing the LibX browser extension for Duke Libraries.  This browser plug-in will enable you to do many things, one of which is to see the references listed in Wikipedia that Duke subscribes to.  Simply click the "get it at Duke" icon that appears beside the reference and in many cases, you'll be taken to the full-text of the article or report.

And there are other cool things that the LibX tool bar will help you do:

• Search Duke's online catalog, e-journals, articles, databases, Google Scholar, WorldCat, or library web pages directly from your browser 
• Search for names, terms or citations on a web page in Duke Library resources, Google Scholar, WorldCat, or the library web site simply by highlighting and right clicking
• Quickly view the library web site, Ask a Librarian, the Duke online catalog, your library account, and more

Unfortunately, the LibX browser extension may only be installed for Firefox and Internet Explorer -- no word on plans to develop LibX for Safari or Chrome. 

Other thoughts about using LibX?  Let us know!

Field Experience Students

This semester, we are doubly fortunate to have two UNC library school students doing field studies in Instruction & Outreach. Alice Whiteside and Julie Adamo are spending the semester immersing themselves in the work that we do, and bringing their energy, enthusiasm and good spirits to our department.


Alice is in her final semester at SILS, and also works at the Sloane Art Library at UNC. Her master’s paper is looking at new models of library instruction. Julie is finishing up her master’s paper on patient memoirs in ARL library collections, while also working at Ipas, a women’s health nonprofit in Chapel Hill. These are two busy librarians.

As part of their field experience, Julie and Alice are helping with usability testing using the Morae software, doing an environmental scan of library instruction tutorials and updating some of our own tutorials, observing library instruction sessions, and contributing to this blog as well as to Library Hacks. They are also researching new technologies and their use for library instruction and giving presentations about those technologies at Instruction & Outreach staff meetings. Look for their technology reports in this blog over the next few months.

We are delighted that Alice and Julie are working in Instruction & Outreach, and we’re grateful to UNC SILS for managing such an efficient and useful field experience program (thanks to Stephanie Peterson and Jeff Pomerantz!)