Quality of Multiple-Choice Items on Buffer Solution Material: A Study of Reliability, Difficulty, Discrimination, and Distractor Efficiency
DOI:
https://doi.org/10.59890/ijels.v4i5.11Keywords:
Item Analysis, Multiple-Choice Test, Chemistry Education, Buffer Solution, Item Difficulty, Discrimination Index, Distractor EffectivenessAbstract
Assessment quality depends not only on test reliability but also on the psychometric functioning of individual items. In chemistry education, this issue is important because teacher-constructed multi-ple-choice tests are often used to evaluate student achievement, yet their item quality is not always examined systematically. This study aimed to analyze the quality of multiple-choice items on the buffer solution topic in Grade XI chemistry. Method: This study employed a quantitative descriptive design using a Classical Test Theory approach. The data consisted of 25 multiple-choice items answered by 35 students and analyzed using ANATES. The analysis focused on descriptive score statistics, test reliability, item difficulty, discrimination power, distractor quality, and item-level decision making. Results: The results showed that the test had a mean score of 20.49, a standard deviation of 2.89, an odd-even correlation of 0.63, and a reliability coefficient of 0.77. However, item quality was uneven. Eight items had a difficulty value of 100.00% and a discrimination value of 0.00%. Only seven items were retained without revision, two required light revision, eight required revision, and eight needed replacement. Items in the moderate difficulty range tended to show better discrimination and more effective distractors than very easy items. Novelty: The study demonstrates that acceptable reliability at the test level does not guarantee good quality at the item level. These findings highlight the importance of routine item analysis for revising teacher-made chemistry assessments and building a more valid and informative item bank.
References
Adebusuyi, O. F., Bamidele, E. F., & Adebusuyi, A. S. (2022). The Role of Knowledge and Epis-temological Beliefs in Chemistry Teachers STEM Professional Development and Instruc-tional Practices: Examination of STEM-Integrated Classrooms. European Journal of Sci-ence and Mathematics Education, 10(2), 243–255. https://doi.org/10.30935/scimath/11799
Ahmad, K. A., Kadir, N. L. A., Zakaria, M. Z., Zamri, M. L., Rahman, S. N. A., & Mustafa, N. M. (2021). Distractor Efficiency of an Intermediate English Proficiency Course Final Ex-amination Paper. AJATeL, 11(2), 103–112.
https://doi.org/10.37134/ajatel.vol11.2.10.2021
Al-Hummayani, F. M. (2020). Evaluation of the Multiple-Choice Question Item Analysis of the Sixth Year Undergraduate Orthodontic Tests at the Faculty of Dentistry, King Abdulaziz University, Saudi Arabia. Egyptian Orthodontic Journal, 57(6), 1–18. https://doi.org/10.21608/eos.2020.31532.1000
Ali, I., Haral, M. N., Tahira, F., Ali, M. W., & Imran, A. (2020). An Evaluation, Assortment, and Optimization of the Question Items. Education and Urban Society, 52(8), 1264–1280. https://doi.org/10.1177/0013124519896831
AlKhatib, H. S., Brazeau, G. A., Akour, A., & AlMuhaissen, S. (2020). Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clin-ical Clerkship Assessment Items. BMC Medical Education, 20(1). https://doi.org/10.1186/s12909-020-02107-3
Al‐lawama, M., & Kumwenda, B. (2023). Decreasing the Options’ Number in Multiple Choice Questions in the Assessment of Senior Medical Students and Its Effect on Exam Psycho-metrics and Distractors’ Function. BMC Medical Education, 23(1). https://doi.org/10.1186/s12909-023-04206-3
Amini, N., Michoux, N., Warnier, L., Malcourant, E., Coche, E., & Berg, B. (2020). Inclusion of McQs Written by Radiology Residents in Their Annual Evaluation: Innovative Method to Enhance Resident’s Empowerment? Insights Into Imaging, 11(1). https://doi.org/10.1186/s13244-019-0809-4
Berenbon, R., & McHugh, B. (2023). Do Subject Matter Experts’ Judgments of Multiple‐Choice Format Suitability Predict Item Quality? Educational Measurement Issues and Practice, 42(3), 13–21. https://doi.org/10.1111/emip.12570
Chen, D., Harman, A. E., Sun, H., Ye, T., & Gaiser, R. R. (2023). A Comparison of 3- And 4-Option Multiple-Choice Items for Medical Subspecialty in-Training Examinations. BMC Medical Education, 23(1). https://doi.org/10.1186/s12909-023-04277-2
Drábeková, J., Rumanová, L., & Pavlovičová, G. (2022). Numeracy as a Part of Financial Litera-cy: Student Case Study in Slovakia. International Journal of Emerging Technologies in Learning (Ijet), 17(10), 186–199.
https://doi.org/10.3991/ijet.v17i10.29659
Du, J., Chen, X., Wang, T., Zhao, J., & Li, K. (2023). The Effectiveness of the Fully Online Flipped Classroom for Nursing Undergraduates During the COVID‐19: Historical Control Study. Nursing Open, 10(8), 5766–5776.
https://doi.org/10.1002/nop2.1757
Falcão, F., Pereira, D. M., Gonçalves, N., Champlain, A. D., Costa, P., & Pêgo, J. M. (2023). A Suggestive Approach for Assessing Item Quality, Usability and Validity of Automatic Item Generation. Advances in Health Sciences Education, 28(5), 1441–1465. https://doi.org/10.1007/s10459-023-10225-y
Ganji, K. K., Ananthakrishnan, N., Manivasakan, S., Alruwaili, M. K., Alonazi, M. A., & Algarni, H. A. (2025). Analyzing the Relationship Between Psychometric Indices of Item Analysis With Attainment of Course Learning Outcomes: Cross-Sectional Study in Integrated Out-come-Based Dental Curriculum Courses. BMC Medical Education, 25(1). https://doi.org/10.1186/s12909-025-07871-8
Mallaiah, J., Williams, O., & Allegrante, J. P. (2024). Development and Validation of a Stroke Lit-eracy Assessment Test for Community Health Workers. Health Education & Behavior, 51(5), 764–774.
https://doi.org/10.1177/10901981241245050
Rashwan, N. I., Aref, S. R., Nayel, O. A., & Rizk, M. H. (2024). Postexamination Item Analysis of Undergraduate Pediatric Multiple-Choice Questions Exam: Implications for Developing a Validated Question Bank. BMC Medical Education, 24(1). https://doi.org/10.1186/s12909-024-05153-3
Semiun, T. T., Wisrance, M. W., & Napitupulu, M. H. (2022). English Summative Test: The Quality of Its Items. English Education Journal of English Teaching and Research, 7(2), 119–127. https://doi.org/10.29407/jetar.v7i2.18347
Wise, M. J. (2020). The Advantages of Five-Option Multiple-Choice Items in Classroom Tests of Student Mastery. Journal of Education Teaching and Social Studies, 2(4), p59. https://doi.org/10.22158/jetss.v2n4p59
Yimer, S. T., & Feza, N. (2019). Learners’ Conceptual Knowledge Development and Attitudinal Change Towards Calculus Using Jigsaw Co-Operative Learning Strategy Integrated With GeoGebra. International Electronic Journal of Mathematics Education, 15(1). https://doi.org/10.29333/iejme/5936
Zahra, N. S., Widyastuti, W., & Ridfah, A. (2022). Psychometric Characteristics of the Culture Fair Intelligence Test Scale 2. Jp3i (Jurnal Pengukuran Psikologi Dan Pendidikan Indonesia), 11(2), 123–136. https://doi.org/10.15408/jp3i.v11i2.25947
Zahran, E., & Mustafa, A. (2023). Item Analysis of Multiple Choice Questions of a Genetic Term Exam in an Egyptian Veterinary College for a Viable Question Bank. Educational Evalua-tion and Learning Management Journal, 0(0), 0–0. https://doi.org/10.21608/eelmj.2023.172846.1001
Zubairi, N. A., Alahmadi, T., Ibrahim, M. I. M., Hegazi, M. A., & Gadi, F. U. (2025). Effective Use of Item Analysis to Improve the Reliability and Validity of Undergraduate Medical Ex-aminations: Evaluating the Same Exam Over Many Years: A Different Approach. Pakistan Journal of Medical Sciences, 41(3), 810–815. https://doi.org/10.12669/pjms.41.3.10693
