Date of Graduation

5-2019

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Educational Statistics and Research Methods (PhD)

Degree Level

Graduate

Department

Rehabilitation, Human Resources and Communication Disorders

Advisor/Mentor

Wen-Juo Lo

Committee Member

Xinya Liang

Second Committee Member

Ronna Turner

Third Committee Member

Zhang Qingyang

Keywords

Confirmatory factor analysis, Measurement invariance, multilevel modeling

Abstract

The test of measurement invariance (MI) investigates whether observed items measure a construct in the same way across different groups or over times. Examining MI is a prerequisite for multiple group comparisons in psychological tests (Schmitt & Kuljanin, 2008). With the prevalence of multilevel data in educational research (e.g., students nested within schools), establishing MI across multiple groups or waves of nested data has brought increasing attention. Two popular techniques for the test of multilevel MI include the multiple-group multilevel confirmatory factor analysis (MMCFA) and the design-based approaches. The MMCFA approach estimates sample covariance matrices at different levels separately. The design-based approach treats nested data as single-level and accounts for data dependency by adjusting the test statistics and standard errors of parameter estimates. Both approaches have been examined in previous studies assuming equal within- and between-level factor structures (e.g., Kim, Kwok & Yoon, 2012), yet the performance of these two approaches on models with unequal cross-level factor structures has not been examined thoroughly. The purpose of this study is to compare the MMCFA and the design-based approaches for evaluating the between-level MI when factor structures differ across levels. Two simulation studies were designed to evaluate the statistical power and Type I error rates of the two estimation approaches. The manipulated conditions included the factor structure, between-level factor variance, number of clusters, cluster size, size of noninvariance, and location of noninvariance. Model comparisons were conducted based on the scaled log-likelihood ratio tests. Results showed that power rates in the MMCFA approach were generally higher than those in the design-based approach across conditions, especially when the cross-level factor structures were different. The between-level factor variance, number of clusters and cluster size were three major factors that impacted the statistical power and Type I error rates with these two approaches. The strengths and limitations of each approach in multilevel MI evaluations as well as the practical implications were discussed at the end.

Download

Share

COinS