Archive for the ‘Events’ Category

Paper accepted for HCI2009

Happy to tell you that my paper
Controlling the Usability Evaluation Process
under Varying Defect Visibility

was accepted for presentation at HCI2009 in Cambridge, Sep 1-5

This paper is part of my work on estimating how many test sessions are required to push a usability evaluation study towards a certain goal, e.g. 80% of usability problems being discovered. In my last year’s paper at HCI I proved that the current approaches of doing such estimations are flawed.

Percentage of UP discovered = 1-(1-p)^n
p:= basic detection probability (visibility)
n:= number of sessions

With the Nielsen/Landauer formula you are at risk to terminate your study before you actually reached the goal.

The problem is that the geometric series formula introduced by Nielsen and Landauer in 1992 neglects the fact that usability problems differ in their visibility. Varying defect visibility means that there is no single parameter p in the equation. Instead, p varies across usability problems – some are easier to identify than others. Varying problem visibility results in a much slower progress of a study. Thus, the geometric series formula is too optimistic.

This year I will present a formula that improves accuracy by accounting for varying visibility of problems. When running a study you can use this formula to estimate the number of problems that remain undiscovered in the system. You can also compute confidence intervals for this estimate.

The new formula will allow you to make an informed decision whether to finish a study or to continue with further test persons.



CHI paper received “Best of CHI” award nomination

we are happy that our CHI 2008 paper  “Introducing Item Response Theory for Measuring Usability Inspection Processes” received the Best paper Honorable Mention. So, don’t miss our fancy presentation on Wednesday April 9 at 9:00 AM .

Tags: ,

CHI paper accepted

glad to tell you, that our full paper “Intrducing Item Response Theory for Measuring the Usability Evaluation Process” is accepted for the CHI 2008 in Florence, Italy.

In this paper we treat the problem of heterogeneity in the usability evaluation process. Former stochastic models of Virzi, Nielsen&Landauer and Lewis employ the curve of diminishing returns P=1-(1-p)^n for predicting the outcome of an evaluation process. The problem with this model is the unique parameter p: It is unlikely that the probability of an individual evaluator detecting a usability defect is always the same.

As a solution we introduce the Rasch model for modeling the process. This model allows for a unique p_ij for each pair of evaluator and defect. With the Rasch model two basic parameters are introduced: the difficulty of defects and the skills of evaluators. The Rasch model allows for measuring both on a truly metric scale.

Admittedly, this is kind of very theoretical stuff (although I prefer the term “groundbreaking” ). But we tried hard to explain the basic concepts and procedures and describe three practical scenarios. At the time of the CHI, these scenarios will be available for download as runnable programs in the statistical computing environment R.

Photos from HCI2007, Lancaster

Some photos from the HCI2007, Lancaster are online. Find them at

You find photo streams from others with the tag hci2007. But be aware that there also appear photos from the HCI International in Beijing.

Have FUN!

Powered by ScribeFire.

Impressions from the HCI2007: Jared Spools keynote

In brief: Never miss a HCI again. It was a well organized conference with very nice and interesting people. The food was better than expected and the English humour lived up to the expectations.

Jared Spool, originally uploaded by xeeliz.

Particular arousing was the keynote of Jared Spool. He acted as an evangelist, a capitalist and a magican. He communicated a very clear vision of what the research challenges in near futures are :

He made a very clear argument on HCI as an engineering discipline (as opposed to a craft). Usability is becoming a mission-critical quality in large-scale ecommerce applications. He suspects , that finding usability effects is still far from being a development activity with reproducable outcome. In his words, the question is not any longer if testing 5 or 8 users suffices to find 80% of the defects. In contrast, companies will ask for a guarantee, that 99% or 99,9% (he was not sure about the counts of nines) of the defects are trapped…

… and resolved. Spool was very optimistic with the concept of usability patterns, which he considers to have design knowledge at hand if an application has to (re)designed.

You can guess that these claims excited me, because I’m doing active research on improving a pattern-based inspection method and on an advanced measurement model for evaluation processes.

Powered by ScribeFire.