EFFECTIVENESS OF A FACULTY DEVELOPMENT PROGRAM ON THE QUALITY OF A-TYPE MULTIPLE CHOICE QUESTIONS IN UNDERGRADUATE MEDICAL AND DENTAL EDUCATION: A QUASI-EXPERIMENTAL STUDY
DOI:
https://doi.org/10.33279/jkcd.v16i03.1042Keywords:
Assessment Quality, Dental Education, Faculty Development Program, Medical EducationAbstract
Objectives: This study aimed to evaluate the impact of structured FDP on the quality of A-type MCQs constructed by health educators for undergraduate examination.
Materials and Methods: A single-group pretest-posttest quasi-experimental study was carried out in a public sector health professions institution. A one-day FDP was organized for 15 faculty members using NBME (National Board of Medical Examiners) item writing guidelines along with the Revised Bloom’s Taxonomy. Complete paired data were obtained from the participants, each submitted fi ve MCQs, both prior to and post the intervention. The MCQs were evaluated using a 15-criterion rubric with score ranging 15-75. The primary outcome measure was the difference in the mean total quality score at faculty level. Other outcome measures included changes in the proportion of cognitive levels and decrease in item writing flaws. Paired T test and Cohen’s d were applied for comparative and statistical analysis.
Results: Ten of the fifteen faculty members submitted complete sets of pre and post intervention MCQs, resulting in a total item count of 100 for analysis. There was an increase in the mean total quality score from 55.44 pre-intervention to 56.72 post-intervention, equating to a mean gain of 1.28 with a p-value of 0.003. There was also a corresponding increase in the percentage of higher-order MCQs from 34% to 50%, while flawed items decreased from 74% to 56%. Significant improvements were seen only in curriculum alignment (p=0.014) and single-concept focus (p=0.018) among other criteria. Significant gains were also recorded in both structural quality (p=0.040) and content validity (p=0.042) after the FDP.
Conclusion: Participation in a structured FDP resulted in notable improvements in MCQ quality, increased cognitive level of the questions, and decreased item writing flaws.
References
Lee HY, Yune SJ, Lee SY, Im S, Kam BS. The impact of repeated item development training on the prediction of medical faculty members’ item difficulty index. BMC medical education. 2024;24(1):599.
Eleragi AM, Miskeen E, Hussein K, Rezigalla AA, Adam MI, Al-Faifi JA, et al. Evaluating the multiple-choice questions quality at the College of Medicine, University of Bisha, Saudi Arabia: a three-year experience. BMC medical education. 2025;25(1):233.
Yakout SM, Hadi K, Alolayan M, Alasmari MM, Alhaji KM, Alobaid LM, et al. Analysis Of Difficulty Index, Discrimination Index And Distractor Efficiency In Multiple Choice Questions. Journal of Advanced Zoology. 2024;45(3).
Downing SM. The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Advances in Health Sciences Education. 2005;10(2):133-43.
Khan HF, Qayyum S, Beenish H, Khan RA, Iltaf S, Faysal LR. Determining the alignment of assessment items with curriculum goals through document analysis by addressing identifi ed item flaws. BMC medical education. 2025; 25(1):200.
Krathwohl DR. A revision of Bloom's taxonomy: An overview. Theory into practice. 2002;41(4):212-18.
NBME. Constructing written test questions for the basic and clinical sciences (6th ed.). (2021).
Awan SL, Manzoor S, Gillani I, Kayani ZA, Farooq H, Khurshid S. Types of item writing flaws in multiple choice questions in medical education and their effect on passing rate of students. Pakistan Journal of Physiology. 2023;19(4):49-52.
Karthikeyan S, O’Connor E, Hu W. Barriers and facilitators to writing quality items for medical school assessments–a scoping review. BMC medical education. 2019;19(1):123.
Singh G, Singh R, Ananthakrishnan N. Analyzing the quality of multiple-choice questions in microbiology for second-year MBBS students. MGM Journal of Medical Sciences. 2024;11(4):651-58.
Haider A, Omar N, Ghafoor K, Mahmood F, Ali M, Mudassar S. Faculty development: Need assessment survey in a newly established Medical College of Lahore. The Professional Medical Journal. 2024;31(08):1235-41.
Moineau G, León-Bórquez R. WFME recognition programme. Medical teacher. 2025;47(1):168.
Wajid G, Sethi A, Khan RA, Aamir HS. World federation for medical education: appropriateness of basic medical education standards in Pakistan. Pakistan journal of medical sciences. 2019;35(5):1185.
Rudolph MJ, Daugherty KK, Ray ME, Shuford VP, Lebovitz L, DiVall MV. Best practices related to examination item construction and post-hoc review. American journal of pharmaceutical education. 2019;83(7):7204.
Ambusaidi M. Test Item Writing Competence Among Oman College of Health Sciences Nurse Faculty: Widener University; 2022.
Qureshi S, Khan R. Challenges faced by faculty of medical education due to the structural variation in its departments across medical colleges. Pak J Med Health Sci. 2023;17(01):148-150.
Guraya SY, Chen S. The impact and effectiveness of faculty development program in fostering the faculty’s knowledge, skills, and professional competence: A systematic review and meta-analysis. Saudi journal of biological sciences. 2019;26(4):688-97.
Bligh J, Brice J. Further insights into the roles of the medical educator: the importance of scholarly management. Academic Medicine. 2009;84(8):1161-65.
Ayub R, Kiran F, Shabnam N, Rauf A, Fatima F. Effects of a long term faculty development program on improvement in quality of MCQs: an impact evaluation study. BMC medical education. 2025;25(1):541.
Hye T, Hossain MF, Roni MA. Interventions to Improve the Quality of Multiple-Choice Questions: A Systematic Review and Meta-Analysis. American journal of pharmaceutical education. 2025:101872.
Rauf AR, Sultana S. Effect of faculty training on quality of Multiple Choice Questions. Rawal Medical Journal. 2021;46(2) :430.
Nemec EC, Welch B. The impact of a faculty development seminar on the quality of multiple-choice questions. Currents in Pharmacy Teaching and Learning. 2016;8(2):160-63.
Gupta P, Meena P, Khan AM, Malhotra RK, Singh T. Effect of faculty training on quality of multiple-choice questions. International Journal of Applied and Basic Medical Research. 2020;10(3):210-4.
Beg MA, Tabassum T, Ali S. Role of faculty development workshop for improving MCQS quality in basic medical sciences. 2021.
Nedeau-Cayo R, Laughlin D, Rus L, Hall J. Assessment of item-writing fl aws in multiple-choice questions. Journal for nurses in professional development. 2013;29(2):52-7.
Tarrant M, Knierim A, Hayes SK, Ware J. The frequency of item writing fl aws in multiple-choice questions used in high stakes nursing assessments. Nurse Education Today. 2006;26(8):662-71.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Amina Gul, Nowshad Asim, Muhammad Saleh Faisal, Muhammad Ihtesham Khan, Muhammad Salman, Sumaira Javed
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material
- The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
- Attribution — You must give appropriate credit , provide a link to the license, and indicate if changes were made . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- NonCommercial — You may not use the material for commercial purposes .
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
