Listening labs is Mark Hurst open-ended version of the traditional think-aloud test. He has put together some tips on how to moderate a open-ended test.

Some highlights:
- Don't write out specific tasks before the test, since the test should be based on where, how, and why people will use the site
- Don't lead the user in any way
- Act only on the lead of the user
- Avoid opinion-based questions
- Avoid conditional or theoretical "if" questions since they won't spotlight users' real-world actions
- Keep the user in "use mode", and avoid "critique mode"

Links:

• The article Four Words to Improve User Research

Henrik Olsen - January 25, 2005

See also: Tips and guidelines (65) 

The Usability Test Data Logger is an Excel spreadsheet developed by Todd Zazelenchuk, which can be used to collect, analyse, and present results of usability tests. It allows you to measure task completion rates, analyse questionnaire data, and summarise participant comments. It automatically generates charts and includes a timer to measure task completion times.

Links:

• The Usability Test Data Logger

Henrik Olsen - August 17, 2004 - via Column Two

See also: Tools (51) 

According to Janice Fracer, usability testing is most effective when it's a low-stress routine activity, rather than a special event that requires a lot of attention. Successful organizations conduct usability tests on a regular, fixed schedule, integrate results quickly into the product, and spend less money.

To develop a effective culture of usability you should:
- Test regularly and constantly (once a month or more)
- Train a couple of staff members to conduct the tests
- Test with five people at a time
- Perform the tests in-house
- Keep reports crisp and to the point
- Make changes immediately
- Leave recruiting to others

Links:

• The article The Culture of Usability

Henrik Olsen - July 15, 2004

See also: Tips and guidelines (65) 

Imagine a usability test where five out of five participants completed all tasks successfully. What are the chances that 50 or 1000 will have a 100% completion rate? By calculation confidence intervals, you will be able to tell that the chances lies somewhere between 95% and as low as 48%.

In his article, Jeff Sauro shows us how calculate confidence intervals of usability tests.

Links:

• The article Restoring Confidence in Usability Results

Henrik Olsen - July 08, 2004

See also: Tips and guidelines (65) 

The discussion about how many users is enough for a usability test has been going on for years. Research by Jakob Nielsen and Tom Landauer showing that tests with five users will reveal an average of 85% usability problems has been seen as a proof that five is enough.

According to Laura Faulker, Nielsen and Laudauer's prediction is right. Five users will reveal 85% usability problems - on average. In a study, she found that problems found with five users range from nearly 100% down to only 55%. Thus, relying on a single set of five users, we run the risk that nearly half the problems could be missed.

Dr. Eric Schaffer concludes that "…for a routine usability test run 12 people for each segment. For an important one where the stakes are high run 30. If resources are really tight, you can drop to five-six per segment, but this is bad."

Links:

• Enough is enough... but five probably isn't
• Faulkners paper Beyond the five-user assumption
• Jakob Nielsen's article Why You Only Need to Test With 5 Users

Henrik Olsen - July 01, 2004

How does one measure how well a website communicates the value of its offerings? Jared M. Spool from UIE has come up with a variant of usability testing called Inherent Value Testing. It goes something like this:

1. Recruit a minimum of six loyal users and six inexperienced users, who meet the target profile.
2. Start with the experienced group and ask them to give you a tour of the site and share the features they use and like the best.
3. Bring in your new users and let them work through the same scenarios as the experienced users, find out which pages they visit, what they like and what the don't like about the site
4. Compare the results and find out whether the site revealed the same benefits for the new users as those you heard form the experienced users.

Observing experienced users will tell you which offerings people appreciate, while observing new users will tell you which precious offerings that people fail to spot.

Links:

• The article Inherent Value Testing
• The article Conducting Inherent Value Testing

Henrik Olsen - February 29, 2004

If you heard about Mark Hurst, you probably also heard about listening labs. In his October 1, 2003 newsletter, Mark explains the how and why of the method.

Listening labs is Mark's version of the traditional think-aloud test. But instead of predefining tasks for the users to conduct, you give them tasks on-the-fly based on what they want to do on the site. In this way your test will not only tell you if users can do what you want them to, but also if the product can do what the users want to do.

Links:

• The article Four Words to Improve User Research

Henrik Olsen - October 01, 2003

Dan Willis has created a condensed overview of some of the core techniques used in information architecture. The descriptions are in a comic book form and serve as entertaining reminders of some of our development options. Willis one-pagers cover sitepath diagramming, topic mapping, free listing, card sorting, and personas.

Links:

• IA Classics: Tools of the Trade in Comic Book Form

Henrik Olsen - April 28, 2003 - via Usability Views

See also: Site and flow diagramming (4)  Posters (5)  Card sorting (8)  Personas (13)  The design process (14) 

Karl Fast has made a thorough test of three screen recording systems to decide which is most suitable in usability testing settings.

His conclusions:
"ScreenCam had the best performance of any program tested, but the lack of support for Windows 2000 and XP makes it hard to recommend."

"Camtasia offers the best blend of performance, features, and ease of use among the programs tested." "The only drawback is price, but at $150 it's still within the range of almost every budget. Highly recommended."

"My first impression of HyperCam was that for $30 I was getting what I paid for. But once I fiddled with it and found the "secret" of using Camtasia's TSCC codec, I was entirely satisfied. Unless you need the extra features of Camtasia, HyperCam will probably do the job..."

Links:

• The article Recording Screen Activity During Usability Testing
• Lotus ScreenCam
• TechSmith Camtasia
• Hyperponics HyperCam

Henrik Olsen - April 06, 2003

See also: Tools (51) 

Will Schroeder looks at some common Usability myths that have cemented themselves into our profession's foundation and started questioning how they got there.

Links:

• UIE - Usability Myths Need Reality Checks

Tim Lucas - March 23, 2003

See also: Research (93)  Web page design (23)  Navigation (46)