Reviewing Measurement Invariance of Questionnaire Constructs in Cross-National Research: Examples from ICCS 2016

nternational studies face the challenge of obtaining comparable measures across diverse national contexts, which are characterized by differences with regard to many factors such as language, culture, and educational contexts. In addition, in many countries there may also be differences across sub-national contexts (for example across regions or language groups). International studies in the field of education are increasingly incorporating ways of assessing the measurement invariance of the data obtained from tests or questionnaire used in these programs. In particular with regard to questionnaire items, there is a growing body of research indicating that questionnaire formats such as Likert-type rating scale items may not measure respondents’ beliefs consistently across diverse cultural or linguistic contexts. This paper describes how issues of measurement invariance were reviewed based on data from the international field trial of the IEA Civic and Citizenship Education Study (ICCS 2016), which studies the ways young people in lower secondary education are prepared to undertake their roles as citizen in wide range of countries (see Schulz, Ainley, Fraillon, Losito, & Agrusti, 2016; Schulz, Fraillon, Ainley, Losito & Kerr, 2008; Schulz, Ainley, Fraillon, Kerr & Losito, 2010). The survey includes tests and questionnaire to assess students’ cognitive and non-cognitive learning outcomes as well as contextual variables. The field trial analyses incorporated explorative and confirmatory factor analysis, as well as covariances (multiple-group modelling) and item response theory (IRT) which specifically focused on a review of measurement invariance. This paper compares the different methodologies used for analysis, presents examples from the analyses of ICCS 2016 field trial data, and discusses implications for future research on measurement invariance of questionnaire data in cross-national studies.