In this digital ITEMS module, Mairead Shaw and Dr. Jessica Flake review two different frameworks for multilevel measurement modelling: (1) multilevel modelling and (2) structural equation modelling; and demonstrate the entire process in R with working code and available data, from preparing the dataset, through writing and running code, to interpreting and comparing output for the two approaches.
Module Overview
Clustered data structures are common in many areas of educational and psychological research (e.g., students clustered in schools, patients clustered by clinician). In the course of conducting research, questions are often administered to obtain scores reflecting latent constructs. Multilevel measurement models (MLMMs) allow for modelling measurement (the relationship of test items to constructs) and the relationships between variables in a clustered data structure. Modelling the two concurrently is important for accurately representing the relationships between items and constructs, and between constructs and other constructs/variables. The barrier to entry with MLMMs can be high, with many equations and less-documented software functionality. This module reviews two different frameworks for multilevel measurement modelling: (1) multilevel modelling and (2) structural equation modelling. We demonstrate the entire process in R with working code and available data, from preparing the dataset, through writing and running code, to interpreting and comparing output for the two approaches.
Mairead Shaw, McGill University
Ph.D. student in Quantitative Psychology at McGill University
Research on effect sizes in multilevel modelling and measurement in replication studies
Teaches multilevel modelling
Co-author of open-source teaching materials at www.learn-mlms.com
Jessica K. Flake, McGill University
Assistant Professor of Quantitative Psychology at McGill University
Assistant Director for Methods at the Psychological Science Accelerator
Research on measurement practices and the appropriate and transparent use of latent variable models in psychological research
Teaches courses on introductory statistics, measurement theory, and multilevel modelling
Co-author of open-source teaching materials at www.learn-mlms.com
Introduction
Upon completion of this ITEMS module, learners should be able to:
- • Define a multilevel measurement model (MLMM)
• Identify when an MLMM is needed
• Describe and execute an MLMM in a multilevel modelling framework
• Describe and execute an MLMM in a structural equation modelling framework
Section 1: Multilevel Modelling Overview
Upon completion of this section, learners should be able to:
- Understand when and why to use multilevel models
- Recognize when the model is cross-sectional or repeated measures
- Write and understand multilevel modelling equations
- Interpret output for fixed and random effects
Section 2: Measurement Modelling Overview
Upon completion of this section, learners should be able to:
- Define measurement modelling
- List the two elements of a structural equation model
- Code a confirmatory factor analysis using lavaan in R
- Interpret loadings, variances, and fit indices for CFA
Section 3: Multilevel Measurement Modelling
Upon completion of this section, learners should be able to:
- Define a multilevel measurement model (MLMM)
- Understand when to use an MLMM
- Describe two different approaches to multilevel measurement modelling
- State two issues that arise from not using an MLMM
Section 4: Multilevel Measurement Models in a Multilevel Modelling Framework
Upon completion of this section, learners should be able to:
- State the 5-step process for conducting an MLMM in MLM framework
- Restructure data to be used with MLM framework
- List and interpret equations for multilevel measurement models in MLM framework
- Execute multilevel measurement model in R package nlme and interpret output
Section 5: Multilevel Measurement Models in an SEM Framework
Upon completion of this section, learners should be able to:
- State the six-step process for conducting an MLMM in an SEM framework
- Specify a multilevel CFA
- Execute a MLMM in an SEM framework using R package lavaan and interpret the output
- Compare a single-level CFA to a multilevel CFA