Abstract
Online exams, based on multiple-choice questions (MCQs), offer many positive features as an assessment format: flexibility, inclusivity, reduced cost, broad curriculum coverage, and more. However, this format is also highly vulnerable to academic misconduct and has a reputation for only being able to assess “lower order” learning. This chapter details pragmatic guidance which can be used by educators to write MCQs for the assessment of higher order learning. These MCQs require students to apply existing knowledge to actively solve problems rather than recalling facts. The writing of these MCQs requires educators to identify assumed knowledge that students need to answer the question: a “cognitive bridge” between the question content and the answer. Using this MCQ format should then make it harder for students to cheat, particularly when combined with practical tips for the design and management of the examination itself. Finally, the chapter proposes the use of exams based on higher order MCQs as a viable alternative to other “higher order” assessment formats such as essays and written coursework, whose basic validity is fundamentally compromised by plagiarism and the widespread availability of essay mills and artificial intelligence-based content generation systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aiken, L. R. (1982). Writing multiple-choice items to measure higher-order educational objectives. Educational and Psychological Measurement, 42, 803–806. https://doi.org/10.1177/001316448204200312
Bibler Zaidi, N. L., Grob, K. L., Yang, J., Santeen, S. A., Monrad, S. U., Miller, J. M., & Purkiss, J. A. (2016). Theory, process, and validation evidence for a staff-driven medical education exam quality improvement process. Medical Science Educator, 26, 331–336. https://doi.org/10.1007/s40670-016-0275-2
Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, 347–364. https://doi.org/10.1007/BF00138871
Billings, M., DeRuchie, K., Haist, S. A., Hussie, K., Merrell, J., Paniagua, M. A., Swygert, K. A., & Tyson, J. (2020). Constructing written test questions for the health sciences. National Board of Medical Examiners.
Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of educational objectives. The classification of educational goals, handbook I: Cognitive domain. Longmans Green.
Bretag, T., Harper, R., Burton, M., Ellis, C., Newton, P., & Rozenberg, P. (2019). Contract cheating: A survey of Australian university students. Studies in Higher Education, 44, 1837–1856. https://doi.org/10.1080/03075079.2018.1462788
Brothen, T. (2012). Time limits on tests: Updating the 1-minute rule. Teaching of Psychology, 39, 288–292. https://doi.org/10.1177/0098628312456630
Brothen, T., & Wambach, C. (2004). The value of time limits on internet quizzes. Teaching of Psychology, 31, 62–64. https://doi.org/10.1207/s15328023top3101_12
Burns, E. R. (2010). “Anatomizing” reversed: Use of examination questions that foster use of higher order learning skills by students. Anatomical Sciences Education, 3, 330–334. https://doi.org/10.1002/ase.187
Butler, A. C. (2018). Multiple-choice testing in education: Are the best practices for assessment also good for learning? Journal of Applied Research in Memory and Cognition, 7, 323–331. https://doi.org/10.1016/j.jarmac.2018.07.002
Cheek, L. (2020). Proctoring problems: Bar students urinate in bottles and buckets over fears online exams will be terminated – Legal Cheek. In Leg. Cheek – Leg. News Insid. Insight Careers Advice. https://www.legalcheek.com/2020/08/proctoring-problems-bar-students-urinate-in-bottles-and-buckets-over-fears-online-exams-will-be-terminated/. Accessed 12 Oct 2022.
Conijn, R., Kleingeld, A., Matzat, U., & Snijders, C. (2022). The fear of big brother: The potential negative side-effects of proctored exams. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.12651
Corrigan-Gibbs, H., Gupta, N., Northcutt, C., Cutrell, E., & Thies, W. (2015). Deterring cheating in online environments. ACM Transactions on Computer-Human Interaction, 22(6), 1–28. https://doi.org/10.1145/2810239
Darabi, M., Macaskill, A., & Reidy, L. (2017). A qualitative study of the UK academic role: Positive features, negative aspects and associated stressors in a mainly teaching-focused university. Journal of Further and Higher Education, 41, 566–580. https://doi.org/10.1080/0309877X.2016.1159287
de Jager, K., & Brown, C. (2010). The tangled web: Investigating academics’ views of plagiarism at the University of Cape Town. Studies in Higher Education, 35, 513–528. https://doi.org/10.1080/03075070903222641
Dellinges, M. A., & Curtis, D. A. (2017). Will a short training session improve multiple-choice item-writing quality by dental school faculty? A pilot study. Journal of Dental Education, 81, 948–955. https://doi.org/10.21815/JDE.017.047
Dempster, E. R., & Kirby, N. F. (2018). Inter-rater agreement in assigning cognitive demand to Life Sciences examination questions. Perspectives in Education, 36(1), 94–110.
Di Giusto, F., Müller Werder, C., Reichmuth, A., Adams-Hausheer, D., & Christian, J. (2019). Multiple-choice questions: Teaching guide for higher and professional education. https://doi.org/10.21256/zhaw-19339
Elkhatat, A. M. (2022). Practical randomly selected question exam design to address replicated and sequential questions in online examinations. International Journal of Educational Integrity, 18, 8. https://doi.org/10.1007/s40979-022-00103-2
Ellis, C., van Haeringen, K., Harper, R., Bretag, T., Zucker, I., & McBride, S. (2020). Does authentic assessment assure academic integrity? Evidence from contract cheating data. Higher Education Research & Development, 39(3), 454–469. https://doi.org/10.1080/07294360.2019.1680956
Foltýnek, T., Meuschke, N., & Gipp, B. (2019). Academic plagiarism detection: A systematic literature review. ACM Computing Surveys, 52(6), 1–42. https://doi.org/10.1145/3345317
Foxe, J. P., Frake-Mistak, M., & Popovic, C. (2017). The instructional skills workshop: A missed opportunity in the UK? Innovations in Education and Teaching International, 54(2), 135–142. https://doi.org/10.1080/14703297.2016.1257949
Garg, M., & Goel, A. (2022). A systematic literature review on online assessment security: Current challenges and integrity strategies. Computers & Security, 113, 102544.
Golden, J., & Kohlbeck, M. (2020). Addressing cheating when using test bank questions in online classes. Journal of Accounting Education, 52, 100671. https://doi.org/10.1016/j.jaccedu.2020.100671
Haladyna, T. M. (1997). Writing test items to evaluate higher order thinking. Allyn and Bacon.
Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 37–50. https://doi.org/10.1207/s15324818ame0201_3
Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items (1st ed.). Routledge/Taylor & Francis Group.
Hoseana, J., Stepanus, O., & Octora, E. (2022). A format for a plagiarism-proof online examination for calculus and linear algebra using Microsoft Excel. International Journal of Math Education in Science and Technology, 1–19. https://doi.org/10.1080/0020739X.2022.2070084
Kar, S. S., Lakshminarayanan, S., & Mahalakshmy, T. (2015). Basic principles of constructing multiple choice questions. Indian Journal of Community and Family Medicine, 1(2), 65–69. https://doi.org/10.4103/2395-2113.251640
Karizaki, V. M. (2021). Different approaches for reducing cheating in online assessments. Journal of Medical Imaging and Radiation Sciences, 52(4), 650–651. https://doi.org/10.1016/j.jmir.2021.07.005
Karpen, S. C., & Welch, A. C. (2016). Assessing the inter-rater reliability and accuracy of pharmacy faculty’s Bloom’s Taxonomy classifications. Currents in Pharmacy Teaching and Learning, 8(6), 885–888.
Karpicke, J. D., Butler, A. C., & Roediger, H. L., III. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? Memory, 17(4), 471–479. https://doi.org/10.1080/09658210802647009
Kim, M.-K., Patel, R. A., Uchizono, J. A., & Beck, L. (2012). Incorporation of Bloom’s taxonomy into multiple-choice examination questions for a pharmacotherapeutics course. American Journal of Pharmaceutical Education, 76(6), 114. https://doi.org/10.5688/ajpe766114
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory Into Practice, 41(4), 12–218. https://doi.org/10.1207/s15430421tip4104_2
MacFarlane, L.-A., & Boulet, G. (2017). Multiple-choice tests can support deep learning! Proceedings of the Atlantic Universities’ Teaching Showcase, 21, 61–66. Retrieved from https://ojs.library.dal.ca/auts/article/view/8430
Marano, E., Newton, P. M., Birch, Z., Croombs, M. Gilbert, C., & Draper, M. J. (2023). What is the student experience of remote proctoring? A pragmatic scoping review. Retrieved from https://osf.io/jrgw9/
Mate, K. E., & Weidenhofer, J. (2021). Are online examinations a viable alternative to paper-based examinations for assessment of human physiology? Proceedings of the Australian Conference on Science and Mathematics Education, 78–83. Retrieved from https://openjournals.library.sydney.edu.au/index.php/IISME/article/view/15493
Medland, E. (2019). ‘I’m an assessment illiterate’: Towards a shared discourse of assessment literacy for external examiners. Assessment & Evaluation in Higher Education, 44(4), 565–580. https://doi.org/10.1080/02602938.2018.1523363
Miguel, C., Castro, L., Marques dos Santos, J. P., Serrão, C., & Duarte, I. (2021). Impact of COVID-19 on medicine lecturers’ mental health and emergency remote teaching challenges. International Journal of Environmental Research and Public Health, 18(13), 6792. https://doi.org/10.3390/ijerph18136792
Morrish, L. (2019). Pressure vessels: The epidemic of poor mental health among higher education staff. Higher Education Policy Institute, 20. Retrieved from https://www.hepi.ac.uk/2019/05/23/pressure-vessels-the-epidemic-of-poor-mental-health-among-higher-education-staff/
Munoz, A., & Mackay, J. (2019). An online testing design choice typology towards cheating threat minimisation. Journal of University Teaching & Learning Practice, 16(3). https://doi.org/10.53761/1.16.3.5
Newton, P. M. (2016). Academic integrity: A quantitative study of confidence and understanding in students at the start of their higher education. Assessment & Evaluation in Higher Education, 41(3), 482–497. https://doi.org/10.1080/02602938.2015.1024199
Newton, P. M. (2018). How common is commercial contract cheating in higher education and is it increasing? A systematic review. Frontiers in Education, 3. https://doi.org/10.3389/feduc.2018.00067
Newton, P. M. (2023). ChatGPT performance on MCQ-based exams. Retrieved from https://edarxiv.org/sytu3/
Newton, P. M., Da Silva, A., & Berry, S. (2020a). The case for pragmatic evidence-based higher education: A useful way forward? Frontiers in Education, 5. https://doi.org/10.3389/feduc.2020.583157
Newton, P. M., Da Silva, A., & Peters, L. G. (2020b). A pragmatic master list of action verbs for Bloom’s taxonomy. Frontiers in Education, 5. https://doi.org/10.3389/feduc.2020.00107
Newton, P. M., & Essex, K. (2022). How common is cheating in online exams and did it increase during the COVID-19 pandemic? A systematic review. https://doi.org/10.21203/rs.3.rs-2187710/v1
Newton, P. M., & Lang, C. (2016). Custom essay writers, freelancers, and other paid third parties. In T. Bretag (Ed.), Handbook of academic integrity (pp. 249–271). Springer Singapore.
Nguyen, J. G., Keuseman, K. J., & Humston, J. J. (2020). Minimize online cheating for online assessments during COVID-19 pandemic. Journal of Chemical Education, 97(9), 3429–3435. https://doi.org/10.1021/acs.jchemed.0c00790
Novick, P. A., Lee, J., Wei, S., Mundorff, E. C., Santagelo, J. R., & Sonbuchner, T. M. (2022). Maximizing academic integrity while minimizing stress in the virtual classroom. Journal of Microbiology & Biology Education, 23, e00292–e00221. https://doi.org/10.1128/jmbe.00292-21
Olt, M. R. (2002). Ethics and distance education: Strategies for minimizing academic dishonesty in online assessment. Online Journal of Distance Learning Administration, 5(3).
OpenAI. (2023). GPT-4 Technical Report.
Palmer, E. J., & Devitt, P. G. (2007). Assessment of higher order cognitive skills in undergraduate education: Modified essay or multiple choice questions? Research paper. BMC Medical Education, 7, 49. https://doi.org/10.1186/1472-6920-7-49
Pastore, S. (2022). Assessment literacy in the higher education context: A systematic review. Intersection, 4(1).
Piza, F., Kesselheim, J. C., Perzhinsky, J., Drowos, J., Gillis, R., Moscovic, K., et al. (2019). Awareness and usage of evidence-based learning strategies among health professions students and faculty. Medical Teacher, 41(12), 1411–1418. https://doi.org/10.1080/0142159X.2019.1645950
Pleasants, J., Pleasants, J. M., & Pleasants, B. P. (2022). Cheating on unproctored online exams: Prevalence, mitigation measures, and effects on exam performance. Online Learning, 26(1), 268–284. https://doi.org/10.24059/olj.v26i1.2620
Rakes, G. C. (2008). Open book testing in online learning environments. Journal of Interactive Online Learning, 7(1), 1–9. Retrieved from https://www.ncolr.org/jiol/issues/pdf/7.1.1.pdf
Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement Issues and Practice, 24(2), 3–13. https://doi.org/10.1111/j.1745-3992.2005.00006.x
Rowland, C. A. (2014). The effect of testing versus restudy on retention: A meta-analytic review of the testing effect. Psychological Bulletin, 140(6), 1432–1463. https://doi.org/10.1037/a0037559
Rush, B. R., Rankin, D. C., & White, B. J. (2016). The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. BMC Medical Education, 16, 250. https://doi.org/10.1186/s12909-016-0773-3
Sabrina, F., Azad, S., Sohail, S., & Thakur, S. (2022). Ensuring academic integrity in online assessments: A literature review and recommendations. International Journal of Information and Educational Technology, 12(1), 60–70. https://doi.org/10.18178/ijiet.2022.12.1.1587
Schuwirth, L. W. T., & Van Der Vleuten, C. P. M. (2004). Different written assessment methods: What can be said about their strengths and weaknesses? Medical Education, 38(9), 974–979. https://doi.org/10.1111/j.1365-2929.2004.01916.x
Scully, D. (2017). Constructing multiple-choice items to measure higher-order thinking. Practical Assessment, Research, and Evaluation, 22(22), 4. https://doi.org/10.7275/swgt-rj52
Sharples, M. (2022). Automated essay writing: An AIED opinion. International Journal of Artificial Intelligence, 32, 1119–1126. https://doi.org/10.1007/s40593-022-00300-7
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795
Stringer, J. K., Santen, S. A., Lee, E., et al. (2021). Examining Bloom’s taxonomy in multiple choice questions: Students’ approach to questions. Medical Science Educator, 31, 1311–1317. https://doi.org/10.1007/s40670-021-01305-y
Sullivan, D. P. (2016). An integrated approach to preempt cheating on asynchronous, objective, online assessments in graduate business classes. Online Learning, 20(3), 195–209. https://doi.org/10.24059/olj.v20i3.650
Tarrant, M., Knierim, A., Hayes, S. K., & Ware, J. (2006). The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Education Today, 26(8), 662–671. https://doi.org/10.1016/j.nedt.2006.07.006
Thomas, J., & Scott, J. (2016). UK perspectives of academic integrity. In T. Bretag (Ed.), Handbook of academic integrity (pp. 39–53). Springer Singapore.
Ullah, A., Xiao, H., & Barker, T. (2019). A study into the usability and security implications of text and image based challenge questions in the context of online examination. Education and Information Technologies, 24, 13–39. https://doi.org/10.1007/s10639-018-9758-7
Urbina-Garcia, A. (2020). What do we know about university academics’ mental health? A systematic literature review. Stress & Health, 36(5), 563–585. https://doi.org/10.1002/smi.2956
Varble, D. (2014). Reducing cheating opportunities in online test. Atlantic Marketing Journal, 3(3), 9. Available at: https://digitalcommons.kennesaw.edu/amj/vol3/iss3/9
Veale, C. G. L. (2022). Considering the impact of image-based search engines for online chemistry assessments. Journal of Chemical Education, 99(3), 1497–1502. https://doi.org/10.1021/acs.jchemed.1c01075
Veloski, J. J., Rabinowitz, H. K., Robeson, M. R., & Young, P. R. (1999). Patients don’t present with five choices: An alternative to multiple-choice tests in assessing physicians’ competence. Academic Medicine, 74(5), 539–546. https://doi.org/10.1097/00001888-199905000-00022
Whisenhunt, B. L., Cathey, C. L., Hudson, D. L., & Needy, L. M. (2022). Maximizing learning while minimizing cheating: New evidence and advice for online multiple-choice exams. Scholarship of Teaching and Learning in Psychology, 8(2), 140–153. https://doi.org/10.1037/stl0000242
Willingham, D. (2006). How knowledge helps. American Federation of Teachers, Spring. Available at https://www.aft.org/periodical/american-educator/spring-2006/how-knowledge-helps
Xu, X., Kauer, S., & Tupy, S. (2016). Multiple-choice questions: Tips for optimizing assessment in-seat and online. Scholarship of Teaching and Learning in Psychology, 2(2), 147–158. https://doi.org/10.1037/stl0000062
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2024 Springer Nature Switzerland AG
About this entry
Cite this entry
Newton, P.M. (2024). Guidelines for Creating Online MCQ-Based Exams to Evaluate Higher Order Learning and Reduce Academic Misconduct. In: Eaton, S.E. (eds) Second Handbook of Academic Integrity. Springer International Handbooks of Education. Springer, Cham. https://doi.org/10.1007/978-3-031-54144-5_93
Download citation
DOI: https://doi.org/10.1007/978-3-031-54144-5_93
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54143-8
Online ISBN: 978-3-031-54144-5
eBook Packages: EducationReference Module Humanities and Social SciencesReference Module Education