Date of Graduation

12-2020

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Educational Statistics and Research Methods (PhD)

Degree Level

Graduate

Department

Rehabilitation, Human Resources and Communication Disorders

Advisor/Mentor

Ronna C. Turner

Committee Member

Xinya Liang

Second Committee Member

Brandon Crawford

Third Committee Member

Allison Ames

Keywords

DIF, Differential Item Functioning, Educational tests, Educational measurements, Effect Size, Item Bias, POLYSIBTEST, SIBTEST

Abstract

A standardized effect size for the SIBTEST/POLYSIBTEST procedure is proposed, allowing for Differential Item Functioning (DIF) to be classified with a single set of DIF heuristics regardless of whether data are dichotomous or polytomous. This proposed standardized effect size accounts for both variability in responses and whether participants are included in the SIBTEST/POLYSIBTEST calculations. First, a new set of unstandardized effect size heuristics are established for dichotomous data that are more aligned with Educational Testing Service (ETS) standards using two and three parameter logistic (2PL and 3PL) models. Second, a standardized effect size is proposed and compared to other DIF heuristics with SIBTEST in a variety of data conditions, with a set of standardized heuristics proposed for dichotomous data. Using the heuristics for the standardized effect size, the application is extended to polytomous data with comparisons made to other proposed DIF heuristics with POLYSIBTEST when there are three to seven item response options. The proposed standardized effect size has practical importance due to there not being an agreed upon way of classifying the magnitude of DIF present in polytomous items using POLYSIBTEST. The standardized effect size also provides a single set of DIF heuristics for classifying DIF in 2PL, 3PL, and graded response models; therefore, making it easier when researchers and practitioners are assessing items for DIF using a variety of scale formats.

Share

COinS