Date of Graduation
8-2024
Document Type
Dissertation
Degree Name
Doctor of Philosophy in Educational Statistics and Research Methods (PhD)
Degree Level
Graduate
Department
Counseling, Leadership, and Research Methods
Advisor/Mentor
Boykin, Allison
Committee Member
Liang, Xinya
Second Committee Member
Robinson, Samantha
Third Committee Member
Turner, Ronna C.
Keywords
Bayesian simulation; Reliability; Standard error of measurement (SEM); Standardized tests; Teachers; Value-added models (VAMs)
Abstract
Abstract Background: Value-added models (VAMs) are statistical tools used to gauge a teacher’s impact on student performance by analyzing standardized test scores. These models project students’ future performance based on past scores and compare the projection to actual outcomes, accounting for differences in student backgrounds. However, the standard error of measurement (SEM) inherent in all measurement tools is often overlooked in VAMs. Aims and Objectives: This study aims to investigate the impact of test reliability on teacher and school score estimates within a Bayesian framework. We will precisely manipulate the reliability of standardized tests by adjusting the standard error of measurement (SEM) and compare the resulting estimates with those generated by the Tennessee Value-Added Assessment System (TVAAS) Composite score, a widely used VAM. Methods: Employing a Monte Carlo simulation design, we explore how varying levels of test SEM, sample size at the classroom level, number of years or grades, and number of teachers affect estimates of teacher effectiveness produced by TVAAS. By systematically altering these parameters, we assess their individual and collective influence on the accuracy and reliability of teacher effectiveness measures. Results: Our analysis indicates that higher levels of test reliability correspond to more precise measures of teacher effectiveness and rankings. Additionally, we highlight SEM’s significant role in shaping value-added modeling outcomes. Furthermore, we identify the interplay between factors such as sample size, number of grades, and intraclass correlation (ICC) in determining the reliability and accuracy of teacher effectiveness measures and rankings. Conclusion: Our findings underscore the importance of considering not only the reliability parameter but also the number of teachers involved in the TVAAS model. The study illuminates the complex dynamics in estimating teacher effectiveness and emphasizes the need for a comprehensive understanding of factors influencing VAM outcomes. Keywords: VAMs, standardized tests, reliability, SEM, Bayesian
Citation
Kamgue, M. J. (2024). Effects of Measurement Error in Student Pre-Post Test Score on the Recovery of the Estimates of Teachers Value-Added Scores. Graduate Theses and Dissertations Retrieved from https://scholarworks.uark.edu/etd/5420