Investigating DIF: The role of response format for students following the syllabus for Swedish as a second language

Experiences from the Swedish National Tests in Social Studies

Abstract

In developing the national tests in Social Studies, a critical issue is ensuring that the items work in the same way for subgroups of students, i.e., the test is fair and shows no signs of severe Differential Item Functioning (DIF). In Sweden, the national tests will be digitalized, but items with a selected response format (SR) is also intended to increase. This study aims to analyze how different items (defined by varying formats and contents) function for students following the syllabus of Swedish as a second language (SvA) compared to the general student population. The study is based on data from the Swedish national tests in Social Studies in 2018, 2019, 2022, and 2023. The results reveal that there are DIF between the two syllabus groups: Students following the SvA syllabus have more difficulties compared to Sv students, with SR items. However, SvA students perform significantly better on some constructed response items where they are asked to describe, analyze, and elaborate on a topic using their own language. As this study shows that the SvA group has significantly more difficulties compared to the general student group with the SR items that are intended to increase due to digitalization, there seems to be an obvious risk for this group to be disadvantaged.