Date of Graduation
12-2020
Document Type
Dissertation
Degree Name
Doctor of Philosophy in Educational Statistics and Research Methods (PhD)
Degree Level
Graduate
Department
Rehabilitation, Human Resources and Communication Disorders
Advisor/Mentor
Turner, Ronna C.
Committee Member
Liang, Xinya
Second Committee Member
Crawford, Brandon L.
Third Committee Member
Ames, Allison
Keywords
DIF; Differential Item Functioning; Educational tests; Educational measurements; Effect Size; Item Bias; POLYSIBTEST; SIBTEST
Abstract
A standardized effect size for the SIBTEST/POLYSIBTEST procedure is proposed, allowing for Differential Item Functioning (DIF) to be classified with a single set of DIF heuristics regardless of whether data are dichotomous or polytomous. This proposed standardized effect size accounts for both variability in responses and whether participants are included in the SIBTEST/POLYSIBTEST calculations. First, a new set of unstandardized effect size heuristics are established for dichotomous data that are more aligned with Educational Testing Service (ETS) standards using two and three parameter logistic (2PL and 3PL) models. Second, a standardized effect size is proposed and compared to other DIF heuristics with SIBTEST in a variety of data conditions, with a set of standardized heuristics proposed for dichotomous data. Using the heuristics for the standardized effect size, the application is extended to polytomous data with comparisons made to other proposed DIF heuristics with POLYSIBTEST when there are three to seven item response options. The proposed standardized effect size has practical importance due to there not being an agreed upon way of classifying the magnitude of DIF present in polytomous items using POLYSIBTEST. The standardized effect size also provides a single set of DIF heuristics for classifying DIF in 2PL, 3PL, and graded response models; therefore, making it easier when researchers and practitioners are assessing items for DIF using a variety of scale formats.
Citation
Weese, J. D. (2020). Development of an Effect Size to Classify the Magnitude of DIF in Dichotomous and Polytomous Items. Graduate Theses and Dissertations Retrieved from https://scholarworks.uark.edu/etd/3896
Included in
Educational Assessment, Evaluation, and Research Commons, Numerical Analysis and Computation Commons, Statistical Methodology Commons
