Archive for the ‘UsabilityResearch’ Tag

Paper accepted for HCI2009

Happy to tell you that my paper
Controlling the Usability Evaluation Process
under Varying Defect Visibility

was accepted for presentation at HCI2009 in Cambridge, Sep 1-5

This paper is part of my work on estimating how many test sessions are required to push a usability evaluation study towards a certain goal, e.g. 80% of usability problems being discovered. In my last year’s paper at HCI I proved that the current approaches of doing such estimations are flawed.

Percentage of UP discovered = 1-(1-p)^n
p:= basic detection probability (visibility)
n:= number of sessions

With the Nielsen/Landauer formula you are at risk to terminate your study before you actually reached the goal.

The problem is that the geometric series formula introduced by Nielsen and Landauer in 1992 neglects the fact that usability problems differ in their visibility. Varying defect visibility means that there is no single parameter p in the equation. Instead, p varies across usability problems – some are easier to identify than others. Varying problem visibility results in a much slower progress of a study. Thus, the geometric series formula is too optimistic.

This year I will present a formula that improves accuracy by accounting for varying visibility of problems. When running a study you can use this formula to estimate the number of problems that remain undiscovered in the system. You can also compute confidence intervals for this estimate.

The new formula will allow you to make an informed decision whether to finish a study or to continue with further test persons.