A three-way classification of sources of item difficulty in tests and examinations

What do test takers mean when they say ‘this item is difficult’? What do test analysts mean when they say ‘this item is difficult’? The answer to the first question comes out of experience. The answer to the second question comes out of empirics. The notion of difficulty covers a considerable diversity of sources, materials and methods. Test analysts seem obliged to collapse all senses of difficulty under one heading and so it might be useful to attempt a classification or typo logy of some of the possible sources of difficulty in test items. This presentation describes such a system and applies it to test items in multiple- choice and short-response items, in tests that are discipline-specific and in tests of generic skills.