Heterogeneity paper accepted for HCI2008

Heterogeneity in the Usability Evaluation Process
accepted as a full paper on HCI2008 in Liverpool

This is the next work on the usability evaluation process. Since 1992 the Virzi/Nielsen/Landauer/ formula was the only choice to predict how many users to test for a certain goal (say 80% of the usability defects). The formula is the cumulative geometric function (CGF):


also known as the curve of diminishing returns, because the closer you approach 100% the more users you have to test to detect new defects.

This formula is inherently flawed. The problem is the single p. There is no single p! Instead,  evaluators differ in their skills and some defects are much harder to detect than others. Past researchers believed that the formula still holds with the mean of p. This is not the case!

When defect heterogeneity occurs, the evaluation process proceeds slowlier than predicted by the CGF. Industrial practitioners planning and controlling their usability studies take a considerable risk of stopping the study too early. Defects remain undetected, which is harmful.

If usability evaluations is your business, attend the conference, read the paper and here the talk in order to learn:

  • how to detect heterogeneity in usability studies
  • how to determine heterogeneity impact
  • and how to deal with it.

Tags: ,


No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: