Latent Covariates for Improved Causal Inference in Large-Scale Assessments

Well-constructed achievement tests are a key strength of educational large-scale assessments (LSAs), and participants’ proficiencies are often the primary focus of subsequent analyses, such as comparisons between different respondent groups. In these non-randomized comparisons, estimating covariate-adjusted group differences is common practice. However, rather than using latent proficiencies directly, researchers often rely on fallible test scores (e.g., sum scores or factor scores) as manifest outcomes or covariates. This talk will highlight the importance of latent variables for causal inference by presenting analytical and empirical results on the impact of measurement error. First, the theoretical conditions under which latent variables are necessary for causal effect estimation will be outlined, and the bias introduced by measurement error will be derived (Sengewald et al., 2019; Sengewald & Pohl, 2019). Then, different strategies to account for measurement error in causal effect analyses will be summarized, and a recent extension of the R package EffectLiteR for differential effect analysis with latent variables will be introduced (Sengewald & Mayer, 2024). Beyond these theoretical insights, an application of the different approaches using LSA data from the German National Education Panel Study (NEPS; Blossfeld & Roßbach, 2019) will demonstrate practical differences.