Seeking the real item difficulty: bias-corrected item difficulty and some consequences in Rasch and IRT modeling

Abstract

<p>When the response pattern in a test item deviates from the deterministic pattern, the percentage of correct answers (<em>p</em>) is shown to be a biased estimator for the latent item difficulty (<em>π</em>). This is specifically true with the items of medium item difficulty. Four elements of impurities in <em>p</em> are formalized in the binary settings and four new estimators of <em>π</em> are proposed and studied. Algebraic reasons and a simulation suggest that, except the case of deterministic item discrimination, the real item difficulty is almost always more extreme than what <em>p</em> indicates. This characteristic of <em>p</em> to be biased toward a medium-leveled item difficulty has a strict consequence to item response theory (IRT) and Rasch modeling. Because the classical estimator of item difficulty <em>p</em> is a biased estimator of the latent difficulty level, the item parameters <em>A</em> and <em>B</em> and the person parameter <em>θ</em> within IRT modeling are, consequently, biased estimators of item discrimination and item difficulty as well as ability levels of the test takers.<br></p>