Lo-fi usability testing – Part 2: Preparation

We’ve already covered in a previous article what usability is and why you need to test it.  In this second instalment, I’ll be showing you what you need to do to prepare for your usability tests.

What do you need?

Day 236 by Jamie Henderson, on Flickr

 

Assuming you have a product or website to test, you’re going to need a moderator.  This is simply a fancy way of saying “person facilitating and overseeing the test”.  The moderator is responsible for organising and running a product usability test, then writing up and handing over the report that outlines the issues uncovered.

Moderator responsibilities #

The moderator is not a glorified note-taker, though; there’s a little more to it.  A good moderator should:

  • Understand what workflows, features need to be tested
  • Understand the user persona(s) to find representative test participants
  • Prepare the room beforehand and ensure a relaxed environment for the tester
  • Create a real-life scenario for the test user
  • Have the good judgement to know when to get involved or when to let the tester make their own mistakes
  • Understand the product well enough to be able to resolve the simplest errors
  • Know how to answer questions with questions to elicit further insight
  • Pay attention to detail – be aware of changes in body language and reactions of tester
  • Be able to identify if a tester gets uncomfortable and be able to put them at ease

Creating the scenario #

A scenario is a story that represents typical user activities and focuses on a single feature or group of related features.  Scenarios should be:

  • Short – Time is precious during usability testing, so you don’t want to spend too much time on reading or explaining scenarios
  • Specific – The wording of the scenario should be unambiguous and have a specific objective
  • Realistic – The scenario should be typical of the activities that an average user will do with the software
  • Familiar – The scenario should explain the task in the user’s language and related to their usual context
  • Not too detailed – Explaining exactly how the user should achieve the goal will bias how they attempt to do so, defeating the purpose of the test

An example of a scenario:

“You’ve forgotten your password for this website.  Go and request a reminder.”

How many test users?

Why you only need to test with 5 users, Jakob Nielsen

 

Jakob Nielsen writes in detail about the maths behind his assertion that you find the majority of usability issues after running only five tests.  From direct experience, I can happily say that this really does seem to be the case in real life, providing all five users are representative of a single user persona.  If you need to test multiple user personas, for example both novice users and power users, you’re going to need a minimum of five users for each.

Of course, there’s nothing stopping you testing with more than five users.  In practice, you’ll probably find the law of diminishing returns comes in to play.  Given that we came into lo-fi usability testing as a means of keeping the time, effort and cost down, we can safely stick with five.

Preparing the room #

  • If you can, run the usability test in the user’s own environment
  • Otherwise, find a quiet room where you and the test participant are unlikely to be disturbed
  • If you’re using a laptop, provide a separate keyboard and mouse if you can, unless you specifically need to test usability of the software or website with the laptop’s built-in touchpad
  • Allocate about forty-five minutes for the session to allow a few minutes’ preamble, but no longer to avoid user fatigue
  • Bring a few pens and lots of paper to take down notes
  • Sit the user in the ‘driving seat’
  • Sit alongside the user where you can see what they’re seeing on the screen, but also be able to see their face and how they use the mouse
  • Keep an eye on the time, particularly if you’re running several sessions back-to-back, though avoid having a huge clock in front of the user – it’s not a timed examination

That covers the preparation for a usability test.  Next time, ten top tips for running the usability test itself.  Thanks for reading.

Read part 3 now!  Ten top tips for running usability tests


Read more from Jock

The Practitioner's Guide to Product Management by Jock Busuttil

“This book has been a genuine game changer for me”
– pb (Amazon)

“I wish this book was published when I started out in product management”
– KejiA (Amazon)

3 thoughts on “Lo-fi usability testing – Part 2: Preparation

  1. Hi Jock, thanks very much for sharing this post. As someone who works on running a usability tool (http://intuitionhq.com) it’s always interesting to see other peoples feeling on testing, and their suggested techniques for doing so.

    I certainly think moderated usability testing has it’s place, but these days there are many remote, unmoderated tools that can really simplify the testing process as well as reducing costs. I find the key is finding a balance between in-person, moderated testing and remote testing, based on the goals and objectives of each site and going on from there. Obviously bigger sites have much more time and money to invest on developing a project than smaller sites/individuals.

    I really like you text on creating the scenarios – this is super useful information that can be applied across all kinds of usability testing techniques. I look forward to seeing your next post in the series.

    Thanks again for sharing.

    Jacob.

    • Hi Jacob,

      I do agree there are many remote usability testing tools available. In fact I’ve collected a few elsewhere on this blog. I think what you miss out on with remote testing is the easy ability to observe the body language of the person. But when you’re balancing convenience with depth of information maybe that’s an acceptable (or pragmatic) compromise to make.

      Thanks for your comment,

      Jock

  2. Pingback: Lo-fi usability testing – Part 1: Background | I Manage Products

Agree? Disagree? Join the conversation: