Scope of usability tests in IS development

Abstract

<p>Despite being a common, established concept in wide usage, usability tests can vary greatly in their goals, techniques, and results. A usability test that one purchases and performs for a specific software product may result in either minor user interface improvements or radical U-turns in development. Researchers have discussed such variation as a problem that concerns testing method’s scientific reliability and validity. In practice, what “kind of data” one can expect to obtain from the selected method has more importance than whether one always obtains the same data. This expectation about information content or “scope” has importance for those who select and conduct usability tests for a specific purpose. However, researchers rarely explicitly state or even discuss scope: too often they adopt the premise that, because a usability test involves users, it brings the (necessary) user-centeredness to the design (i.e., takes socio- technical fundamentals as inherently given). We reviewed the literature on testing practices and analytical consideration and searched for the scope of a usability test that could deliberately approach the socio-technical tradition and equally develop both the system and the user organization. A case example represents a possible realization of the extended scope of usability test.<br></p>