Document Recommendation for Medical Training Using Learning to Rank

Abstract

This paper describes a general approach where Learning-to-Rank (LtR) algorithms are used to suggest training materials, such as videos, simulations, tutorials, and articles to students based on their mistakes while using an e-learning platform. In particular, the method is discussed and implemented for MedSim, computer-based medical simulations that replicate clinical scenarios for medical students. While interacting with a virtual patient, the student may ask questions, do medical examinations or therapy, and eventually arrive at a final diagnosis. The basic idea of the approach is to generate a query based on the mistakes that a student makes while interacting with a virtual patient. Based on this query, the LtR algorithm then provides the student with suggestions for training material to review. We evaluated the approach with different LtR algorithms and different queries and dataset sizes (Haridas et al. in Educ. Inf. Technol. 1–19, 2020 [1]). We studied the extensibility of the trained models by adding new training material to the test set. The accuracy is measured using NDCG@10. The results of the evaluation suggest that a training size of around 1000 is sufficient to obtain accurate results. The comparison of algorithms suggests that listwise algorithms perform better than pair-wise algorithms, also that in particular, LambdaMart performs very well on this problem.