Are We Testing Utility? Analysis of Usability Problem Types

Abstract

<p>Usability problems and related redesign recommendations are the main outcome of usability tests although both are questioned in terms of impact in the design process. Problem classifications aim to provide better feedback for designers by improving usability problem identification, analysis and reporting. However, within the classifications, quite little is discussed about the types and the contents of usability problems as well as the types of required design efforts. We address this problem by scrutinizing the findings of three empirical usability tests conducted in software development projects. As a result, 173 problems were classified into 11 categories. Specific focus was placed on the distinction between the utility and usability types of problems, in order to define the correct development phase and method to fix the problem. The number of utility problems varied from 51% to 74%, which shows that early usability testing with a think-aloud protocol and an open task structure measure both utility and usability equally well.<br></p>