Using mathematical competencies to predict item difficulty in PISA

This paper reports an analysis of features of mathematics assessment items developed for the OECD’s Programme for International Student Assessment survey (PISA) in relation to a set of six mathematical competencies. These competencies have underpinned the PISA mathematics framework since the inception of the PISA survey; they have been used to drive mathematics curriculum and assessment review and reform in several countries; and the results of the study are therefore likely to be of interest to the broad mathematics education community. We present a scheme used to describe this set of mathematical competencies, to quantify the extent to which solution of each assessment item calls for the activation of those competencies, and to investigate how the demand for activation of those competencies relates to the difficulty of the items. We find that the scheme can be used effectively, and that ratings of items according to their demand for activation of the competencies are highly predictive of the difficulty of the items.