Author ORCID Identifier:
Tree-Based Differential Item Functioning Detection Methods
Date of Graduation
8-2025
Document Type
Dissertation
Degree Name
Doctor of Philosophy in Educational Statistics and Research Methods (PhD)
Degree Level
Graduate
Department
Counseling, Leadership, and Research Methods
Advisor/Mentor
Turner, Ronna
Committee Member
Albert Cheng
Second Committee Member
Samantha Robinson
Third Committee Member
Wen-Juo Lo
Keywords
differential item functioning; item bias; partial credit model; rasch model; recursive partitioning; test fairness and validity
Abstract
Despite the availability of numerous methods for detecting differential item functioning (DIF), the continued development and evaluation of innovative, data-driven approaches remains essential. Tree-based methods, in particular, represent a significant advancement in DIF detection. Unlike some traditional techniques, they can simultaneously screen multiple variables for DIF without discretizing continuous variables, and do not require the pre-specification of focal and reference groups; capabilities that are especially valuable in today’s diverse and multifaceted assessment contexts. However, research systematically examining the performance of these methods under realistic measurement conditions is limited. This dissertation, in three simulation studies, critically examines the robustness and practical limitations of global and item-focused recursive partitioning approaches across a broad range of empirically realistic conditions. Additionally, effect size measures are incorporated alongside statistical significance, another understudied topic with tree-based methods, to identify DIF on the item-level with the global methods, and to enhance practical interpretation of DIF detection for both methods. In the first study, the Rasch tree method is evaluated for dichotomous items, demonstrating strong detection capabilities under balanced DIF conditions and large DIF magnitudes, but reduced accuracy with unbalanced DIF and higher contamination rates. The second study compares two item-focused trees for dichotomous data, Rasch-IFT and Logistic-IFT methods, finding that the methods generally yield high item-level true positive rates, though with a tendency to overidentify negligible and moderate DIF underscoring the importance of integrating an effect size criterion. The final study extends this work to polytomous items, comparing the global partial credit tree, PCM-TREE, to the item-focused partial credit tree, PCM-IFT. While PCM-TREE offers conservative and reliable DIF screening in terms of identifying which covariates induce DIF, PCM-IFT excels at identifying item-level DIF, particularly with continuous covariates. Collectively, these studies provide evidence supporting the validity of tree-based methods as flexible tools for DIF detection, while also identifying conditions in which their effectiveness is limited. The studies provide guidance for practitioners in selecting appropriate methods for complex, real-world testing scenarios and contribute to advancing fairness in educational and psychological assessment.
Citation
Asamoah, N. B. (2025). Tree-Based Differential Item Functioning Detection Methods: Exploring Their Performance in Diverse Measurement Scenarios. Graduate Theses and Dissertations Retrieved from https://scholarworks.uark.edu/etd/5869
