A subjective comparative analysis of quality of Short Answer Questions(SAQ) Items between Orthopedics and Other specialties used in internal examinations in MBBS course in Medical College of Lahore
Keywords:
Cognition, Quality, Short Answer QuestionsAbstract
Objective: To compare the subjective quality of various items used in short answer questions (SAQ) papers between Orthopaedics and other specialties used in internal examinations in MBBS course in a Medical College of Lahore.
Methods: This retrospective descriptive study was conducted over a period extending from 25th October 2019 to 25th February 2020 at Avicenna Medical College Lahore. Previously used short answer questions (SAQ) papers (in internal assessments) of MBBS examination were collected from various departments of medical college. They were segregated in two groups as those from Orthopedic Surgery and those from Non Orthopedics subjects like general surgery, medicine and other subjects in the Miscellaneous group. SAQ in both the groups were assessed for the level of cognitive domain like C1(Recall),C2(Identify) and C3(Analyze) level and for clinically focused and relevant scenarios. The statements were evaluated whether scientifically correct and leading to an appropriatediagnosis without providing a lead into the question that would follow. All items were checked for mistakes in language or spelling and grammar. A comparison was made and P value calculated with Chi-square test. P value of < 0.05 was considered significant.
Results: A total of 12 Orthopaedic SAQ papers comprising of 84 items and 14 miscellaneous papers comprising of 101 items were evaluated and compared. The number of C1 questions in Orthopaedic SAQ papers were 24(13%),C2 questions were 32(17.4%) and C3 was 28(15.2%).In the miscellaneous group C1 questions were 33(17.4%),C2 questions were 31(16.8%) and C3 questions were 37(20.1%). The number of SAQ scenarios were focused in 74(40%) SAQ in Orthopaedics versus 48(25.9%) in miscellaneous group(P value 0.013).The quality of statement was clear in 56(30.3%) SAQ in Orthopaedics while only 14(7.6%) of the miscellaneous papers had clear statement. Spelling mistakes were detected in 12(21.1%) SAQ of Orthopaedics and 22(38.8%) in miscellaneous group. Grammatical mistakes were noted in 10(17.5%) Orthopaedics SAQ and 5(8.8%) in miscellaneous group.
Conclusion: The number of Orthopaedic SAQ with cognitive levels(C3,C4) were less than other specialty SAQ. However, the quality of scenarios were better focused and statement much clear in Orthopaedic SAQ than in miscellaneous group.
References
Cantillon P, Hutchinson L, Wood D: Editors: ABC of learning and teaching in medicine: 2003. BMJ Publishing Group Ltd, BMA House, Tavistock Square, London.
Gunderman RB. 2nd ed.Achieving Excellence in Medical Education; Springer-Verlag London Limited 2006. P96
Zia ul Islam, Usmani A. Pak Psychometric analysis of Anatomy MCQs in Modular examination. J Med Sci.2017; 33 (5):1143-1150.
Raina SK, Singh M, Sood A, Chander V. Assessment of need for capacity building in framing multiple choice questions for undergraduate medical students. J Sci Soc.2016;43:11-3.
WHO Guidelines for Quality Assurance of Basic Medical Education in the Western Pacific Region 2006. Manila: WHO Regional Office for the Western Pacific. http://iris.wpro.who.int/handle/10665.1/5593. p16
Chan C. Assessment: Short Answer Questions, Available: Accessed: DATE April 19, 2020, Assessment Resources@HKU, University of Hong Kong[http://ar.cetl.hku.hk]:https://ar.cetl.hku.hk/am_saq.htm
Patil R, Vell K, Boratne AV. Evaluation of multiple choice questions by item analysis in a medical college at Pondicherry, India. Int J Community Med Public Health. 2016;3:1612-1616.
Tariq S, Tariq S, Maqsood S, Jawed S, Baig M. Evaluation of cognitive levels and item writing flaws in medical pharmacology internal assessment examinations. Pak J Med Sci. 2017;33(4):866-870.
Joint Information Systems Committee (JISC). "What Do We Mean by e-Assessment?" JISC Info Net, available at JISCinfonet.ac.uk. Retrieved April 19, 2020.
Tarrant M, Ware J. Impact of item-writing flaws in multiple choice questions on student Achievement in high stakes nursing assessments. Med Educ. 2008;42:198-206.
Buckwalter JA, Schumacher R, Albright JP, Cooper RR. Use of an educational taxonomy for evaluation of cognitive performance. J Med Educ. 1981;56:115–121.
Baig M, Ali SK, Ali S, Huda N. Evaluation of Multiple Choice and Short Essay Question items in Basic Medical Sciences. Pak J Med Sci. 2014;30:3-6.
