IMPROVING METHODS FOR DETECTING SYSTEMATIC BIAS IN ASSESSMENT TO ENSURE THE OBJECTIVITY OF DIGITAL CONTROL RESULTS IN HIGHER EDUCATION INSTITUTIONS
Keywords:
digital assessment, rater effects, many-facet Rasch model, HEMIS monitoring, blind second marking, calibration, objective grading.Abstract
The thesis proposed a workflow to detect systematic bias in digital assessment. The author combined many-facet Rasch measurement, blind second marking, control charts, and false discovery rate control. Simulated tests distinguished rater leniency, severity, central tendency, halo effects, and drift over time. Each signal triggered proportionate feedback or moderation. The author recommended scheduled HEMIS reports, rotating blind reviews, and calibration sessions, and identified validation on real higher-education data as the next step.
References
1.National Statistics Committee of the Republic of Uzbekistan. (2026). Higher education in the Republic of Uzbekistan (as of the beginning of the 2025/2026 academic year). Press release, 26 May 2026. https://stat.uz/img/news/higher-education_p92388.pdf
2.Decree of the President of the Republic of Uzbekistan No. PF-5847 of 08.10.2019. On approval of the Concept for the development of the higher education system of the Republic of Uzbekistan until 2030. https://lex.uz/docs/4545884
3.Digital Government of the Republic of Uzbekistan. (n.d.). Higher Education Process Management – HEMIS. Retrieved November 6, 2026. https://dgov.uz/en/solution/detail/50/
4.Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4(1), 25–29. https://doi.org/10.1037/h0071663
5.Saal, F. E., Downey, R. G., & Lahey, M. A. (1980). Rating the ratings: Assessing the psychometric quality of rating data. Psychological Bulletin, 88(2), 413–428. https://doi.org/10.1037/0033-2909.88.2.413
6.ENQA, ESU, EUA, & EURASHE. (2015). Standards and guidelines for quality assurance in the European Higher Education Area (ESG). Brussels. https://www.enqa.eu/wp-content/uploads/2015/11/ESG_2015.pdf
7.Harvey, L., & Green, D. (1993). Defining quality. Assessment & Evaluation in Higher Education, 18(1), 9–34. https://doi.org/10.1080/0260293930180102
8.Bloxham, S., den-Outer, B., Hudson, J., & Price, M. (2016). Let’s stop the pretence of consistent marking: Exploring the multiple limitations of assessment criteria. Assessment & Evaluation in Higher Education, 41(3), 466–481. https://doi.org/10.1080/02602938.2015.1024607
9.Linacre, J. M. (1989). Many-facet Rasch measurement. Chicago: MESA Press. https://www.winsteps.com/a/Linacre-MFRM-book.pdf
10.Engelhard, G., Jr. (1994). Examining rater errors in the assessment of written composition with a many-faceted Rasch model. Journal of Educational Measurement, 31(2), 93–112. https://doi.org/10.1111/j.1745-3984.1994.tb00436.x
11.Myford, C. M., & Wolfe, E. W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4(4), 386–422. https://pubmed.ncbi.nlm.nih.gov/14523257/
12.Wind, S. A., & Peterson, M. E. (2018). A systematic review of methods for evaluating rating quality in language assessment. Language Testing, 35(2), 161–192. https://doi.org/10.1177/0265532216686999
13.Page, E. S. (1954). Continuous inspection schemes. Biometrika, 41(1–2), 100–115. https://doi.org/10.1093/biomet/41.1-2.100
14.Lucas, J. M., & Saccucci, M. S. (1990). Exponentially weighted moving average control schemes: Properties and enhancements. Technometrics, 32(1), 1–12. https://doi.org/10.1080/00401706.1990.10484583
15.Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x