Abstract
During workplace-based learning, e.g., clinical or during an internship, supervisors’ quality of feedback and assessment is crucial for trainees’ expertise development. Electronic portfolios (ePortfolios) are often used as tools for longitudinal assessment at the workplace. So far, ePortfolios have not realized their full potential, since they are often not well tailored to the workplace and trainees’ needs. Potential data about trainees’ behavior at the workplace is generally underused, and the management of the data is complex. It is expected that ePortfolios enhanced with learning analytics may serve as means to improve the quality of feedback and assessment regarding trainees’ progress and development. This chapter addresses this by outlining an approach that is applied in a European 7th framework project (www.project-WatchMe.eu).
This chapter shows the development of an ePortfolio environment enhanced with learning analytics, to be used at the workplace in medical, veterinary, and teacher education. Evaluation took place by means of a quasi-experimental design regarding the impact of this environment on trainees’ motivation, their assessment experience, and their use. Data gathered in four institutes for medical, veterinary, and teacher education (n = 217) showed that trainees were highly motivated for their internships and positively evaluated the perceived feedback. The use of learning analytics features varied. In general visual feedback by means of a timeline of trainees’ progress was mostly used, while trainees barely used the features with written feedback. It is concluded that the promise of learning analytics connected to ePortfolios can only be fulfilled when developed and implemented through the eyes of the users.
Similar content being viewed by others
References
Agudo-Peregrina A, Iglesias-Pradas S, Conde-González MA, Hernández-García A (2014) Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Comput Hum Behav 31:542–550. https://doi.org/10.1016/j.chb.2013.05.031
Barbera E (2009) Mutual feedback in e-portfolio assessment: an approach to the netfolio system. Br J Educ Technol 40(2):342–357. https://doi.org/10.1111/j.1467-8535.2007.00803.x
Barrett HC (1998) Strategic questions: what to consider when planning for electronic portfolios. Learn Lead Technol 26:6–13
Billett S (2004) Workplace participatory practices: conceptualising workplaces as learning environments. J Work Learn 16(6):312–324. https://doi.org/10.1108/13665620410550295
Bryant LH, Chittum JR (2013) ePortfolio effectiveness: a(n ill-fated) search for empirical support. Int J ePortfolio 3(2):189–198
Butler P (2006) A review of the literature on portfolios and electronic portfolios. Massey University College of Education, Palmerston North
Dekker-Groen A, Van der Schaaf M, Stokking K (2012) Performance standards for teachers supporting nursing students’ reflection skills development. J Nurs Educ Pract 2(1):9–19. https://doi.org/10.5430/jnep.vn1p9
Doyle GJ, Garrett B, Currie LM (2014) Integrating mobile devices into nursing curricula: opportunities for implementation using Rogers’ diffusion of innovation model. Nurse Educ Today 34(5):775–782. https://doi.org/10.1016/j.nedt.2013.10.021
Duijn CC, Welink LS, Mandoki M, ten Cate OT, Kremer WD, Bok HG (2017) Am I ready for it? Students’ perceptions of meaningful feedback on entrustable professional activities. Perspect Med Educ 6:256. https://doi.org/10.1007/s40037-017-0361-1
Dysthe O, Engelsen KS (2011) Portfolio practices in higher education in Norway in an international perspective: macro-, meso-and micro-level influences. Assess Eval High Educ 36(1):63–79. https://doi.org/10.1080/02602930903197891
Elias T (2011) Learning analytics: definitions, processes and potential. Creative Commons Attribution 3.0
Ericsson A, Pool R (2016) Peak: secrets from the new science of expertise. Harcourt, Houghton Mifflin
Ericsson KA, Charness N, Feltovich PJ, Hoffman RR (eds) (2006) The Cambridge handbook of expertise and expert performance. Cambridge University Press, New York
Eynon B, Gambino LM, Török J (2014) What difference can ePortfolio make? A field report from the connect to learning project. Int J ePortfolio 4(1):95–114
Feltovich PJ, Prietula MJ, Ericsson KA (2006) Studies of expertise from psychological perspectives. In: Ericsson KA, Charness N, Feltovich P, Hoffman RR (eds) Cambridge handbook of expertise and expert performance. Cambridge University Press, Cambridge, pp 41–67
Frank JR (2005) The CanMEDS 2005 physician competency framework. Better standards, better physicians. Better care, Ottawa, The Royal College of Physicians and Surgeons of Canada
Gibbs G, Simpson C (2003) Measuring the response of students to assessment: the assessment experience questionnaire. In: 11th Improving Student Learning Symposium, pp 1–12
Greller W, Drachsler H (2012) Translating learning into numbers: a generic framework for learning analytics. J Educ Technol Soc 15(3):42–57
Grossman P, Hammerness K, McDonald M (2009) Redefining teaching, re-imagining teacher education. Teach Teach Theory Pract 15(2):273–289. https://doi.org/10.1080/13540600902875340
Hamp-Lyons L, Condon W (2000) Assessing the portfolio: principles for practice, theory, and research. Hampton Press, Cresskill
Hattie J, Timperley H (2007) The power of feedback. Rev Educ Res 77(1):81–112. https://doi.org/10.3102/003465430298487
Jafari A, Kaufman C (eds) (2006) Handbook of research on eportfolios. IDEA Group Publishing, Hershey
Jarafsky D, Martin JH (2008) Speech and language processing: an introduction to natural language processing, computational linguistics and speech recognition, 2nd edn. Prentice Hall, Upper Saddle River
Jonker G, Hoff RG, Ten Cate O (2015) A case for competency-based anaesthesiology training with entrustable professional activities: an agenda for development and research. Eur J Anaesthesiol (EJA) 32(2):71–76. https://doi.org/10.1097/EJA.0000000000000109
Kane MT (1992) An argument-based approach to validity. Psychol Bull 112(3):527
Kennelly E, Osborn D, Reardon R, Shetty B (2016) Guidance for ePortfolio researchers: a case study with implications for the ePortfolio domain. Int J ePortfolio 6(2):117–125. http://www.theijep.com
Kovanović V, Joksimović S, Gašević D, Hatala M, Siemens G (2015) Content analytics: the definition, scope, and an overview of published research. In: Handbook of learning analytics. SoLAR, Edmonton. https://doi.org/10.18608/hla17.007
Laskey KB (2008) MEBN: a language for first-order Bayesian knowledge bases. Artif Intell 172(2–3):140–178. https://doi.org/10.1016/j.artint.2007.09.006
Leijen A, Slof B, Malva L, Hunt P, van Tartwijk JWF, van der Schaaf MF (2017) Performance-based competency requirements for learner teachers and how to assess them. Int J Inf Educ Technol 7(3):190–194. https://doi.org/10.18178/ijiet.2017.7.3.864
McNamara DS, Allen LK, Crossley SA, Dascula M, Perret CA (2017) Natural language processing and learning analytics. In: Lang C, Siemens G, Wise A, Gašević D (eds) Handbook of learning analytics. SoLAR, Edmonton, pp 93–104. https://doi.org/10.18608/hla17.008
Mislevy RJ, Behrens JT, Dicerbo KE, Levy R (2012) Design and discovery in educational assessment: evidence-centered design, psychometrics, and educational data mining. JEDM-J Educ Data Min 4(1):11–48
Mulder M (2014) Conceptions of professional competence. In: Billett S, Harteis C, Gruber H (eds) International handbook of research in professional and practice-based learning. Springer, Dordrecht, pp 107–137
Peacock S, Murray S, Scott A, Kelly J (2011) The transformative role of ePortfolios: feedback in healthcare learning. Int J ePortfolio 1(1):33–48. http://www.theijep.com
Rezgui K, Mhiri H, Ghédira K (2014) Ontology-based e-Portfolio modeling for supporting lifelong competency assessment and development. Procedia Comput Sci 112:397–406. https://doi.org/10.1016/j.proc.2017.08l041
Royal College of Veterinary Surgeons (RCVS) (2014) Practice standards scheme manual. Royal College of Veterinary Surgeons, London
Ryan RM, Deci EL (2000) Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 55(1):68. https://doi.org/10.1037/110003-066X.55.1.68
Sadler DR (2010) Beyond feedback: developing student capability in complex appraisal. Assess Eval High Educ 35(5):535–550. https://doi.org/10.1080/02602930903541015
Selwyn N (2015) Data entry: towards the critical study of digital data and education. Learn Media Technol 40(1):64–82. https://doi.org/10.1080/17439884.2014.921628
Siemens G, Gašević D (2012) Guest editorial – learning and knowledge analytics. Educ Technol Soc 15(3):1–2
Ten Cate O (2005) Entrustability of professional activities and competency-bases training. Med Educ 39:1176–1177
Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, Van der Schaaf M (2015) Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no 99. Med Teach 37(11):983–1002
Toulmin SE (2003) The uses of argument. Updated edition. Cambridge University Press, Cambridge, UK
Van der Schaaf MF, Stokking K, Verloop N (2008) Developing and validating a design for teacher portfolio assessment. Assess Eval High Educ 33(3):245–262. https://doi.org/10.1080/02602930701292522
Van der Schaaf M, Baartman L, Prins F (2012) Exploring the role of assessment criteria during teachers’ collaborative judgement processes of students’ portfolios. Assess Eval High Educ 37(7):847–860
van der Schaaf M, Donkers J, Slof B, Moonen-van Loon J, van Tartwijk J, Driessen E, ... Ten Cate O (2017) Improving workplace-based assessment and feedback by an E-portfolio enhanced with learning analytics. Educ Technol Res Dev 65(2):359–380. https://doi.org/10.1007/s11423-016-9496-8
Van Schaik S, Plant J, O’sullivan P (2013) Promoting self-directed learning through portfolios in undergraduate medical education: the mentors’ perspective. Med Teach 35(2):139–144
Warren SJ, Lee J, Najmi A (2014) The impact of technology and theory on instructional design since 2000. In: Spector JM, Merrill MD, Elen\ J, Bishop MJ (eds) Handbook of research on educational communications and technology, 4th edn. Springer, New York, pp 89–99
Wisman-Zwarter N, Van der Schaaf M, Ten Cate OT, Jonker G, Van Klei WA, Hoff R (2016) Defining the content of Anaesthesiology training with entrustable professional activities. A delphi study. Eur J Anesthesiol 33(8):559–567
Acknowledgment
This study was conducted within the framework of the “Workplace-Based e-Assessment Technology for competency-Based Higher Multi-Professional Education” (WATCHME) project, supported by the European Commission 7th Framework Programme (grant agreement No. 619349). We thank all participants and members of the WATCHME project for their contributions: Charité Universitätsmedizin Berlin, Germany; Jayway, Denmark; Maastricht University, the Netherlands; Mateum BV., the Netherlands; NetRom Software SRL, Romania; Tartu Ülikool, Estonia; University of California San Francisco, USA; University Medical Center Utrecht, the Netherlands; University of Reading, UK; University of Veterinary Medicine, Hungary; and Utrecht University, the Netherlands.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this entry
Cite this entry
van der Schaaf, M.F. (2019). Electronic Portfolios Enhanced with Learning Analytics at the Workplace. In: McGrath, S., Mulder, M., Papier, J., Suart, R. (eds) Handbook of Vocational Education and Training. Springer, Cham. https://doi.org/10.1007/978-3-319-49789-1_89-1
Download citation
DOI: https://doi.org/10.1007/978-3-319-49789-1_89-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49789-1
Online ISBN: 978-3-319-49789-1
eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education