CHI paper accepted

glad to tell you, that our full paper “Intrducing Item Response Theory for Measuring the Usability Evaluation Process” is accepted for the CHI 2008 in Florence, Italy.

In this paper we treat the problem of heterogeneity in the usability evaluation process. Former stochastic models of Virzi, Nielsen&Landauer and Lewis employ the curve of diminishing returns P=1-(1-p)^n for predicting the outcome of an evaluation process. The problem with this model is the unique parameter p: It is unlikely that the probability of an individual evaluator detecting a usability defect is always the same.

As a solution we introduce the Rasch model for modeling the process. This model allows for a unique p_ij for each pair of evaluator and defect. With the Rasch model two basic parameters are introduced: the difficulty of defects and the skills of evaluators. The Rasch model allows for measuring both on a truly metric scale.

Admittedly, this is kind of very theoretical stuff (although I prefer the term “groundbreaking” ). But we tried hard to explain the basic concepts and procedures and describe three practical scenarios. At the time of the CHI, these scenarios will be available for download as runnable programs in the statistical computing environment R.

Advertisements

No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: