Abstract
User-adaptive visualization and explanatory visualization have been suggested to increase educational effectiveness of program visualization. This paper presents an attempt to assess the value of these two approaches. The results of a controlled experiment indicate that explanatory visualization allows students to substantially increase the understanding of a new programming topic. Furthermore, an educational application that features explanatory visualization and employs a user model to track users’ progress allows students to interact with a larger amount of material than an application which does not follow users’ activity. However, no support for the difference in short-term knowledge gain between the two applications is found. Nevertheless, students admit that they prefer the version that estimates and visualizes their progress and adapts the learning content to their level of understanding. They also use the application’s estimation to pace their work. The differences in eye movement patterns between the applications employing adaptive and non-adaptive explanatory visualizations are investigated as well. Gaze-based measures show that adaptive visualization captivates attention more than its non-personalized counterpart and is more interesting to students. Natural language explanations also accumulate a big portion of students’ attention. Furthermore, the results indicate that working memory span can mediate the perception of adaptation. It is possible that user-adaptation in an educational context provides a different service to people with different mental processing capabilities.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Bednarik, R., Myller, N., Sutinen, E., Tukianinen, M.T.: Effects of experience on gaze behavior during program animation. In: Proceedings of the 17th Annual Psychology of Programming Interest Group Workshop (PPIG), pp. 49–61 (2005)
Blumenktants, M., Starovisky, H., Shamir, A.: Narrative algorithm animation. Proceedings of ACM Symposium on Software Visualization, pp. 17–26 (2006)
Boud D.: Enhancing Learning Through Self Assessment. Routledge, New York, NY (1995)
Boyle C., Encarnacion A.O.: Metadoc: an adaptive hypertext reading system. User Model. User-Adapt. Interact. 4(1), 1–19 (1994)
Brusilovsky, P.: Program visualization as a debugging tool for novices. In: Proceedings of the 5th International Conference on Human-Computer Interaction (INTERCHI; Adjunct Proceedings), pp. 29–30 (1993)
Brusilovsky, P.: Explanatory visualization in an educational programming environment: connecting examples with general knowledge. In: Proceedings of the 4th International Conference on Human-Computer Interaction, pp. 202–212 (1994)
Brusilovsky, P., Loboda, T.D.: WADEIn II: a case for adaptive explanatory visualization. In: Proceedings of the 10th Conference on Innovation Technology in Computer Science Education (ITiCSE), pp. 48–52 (2006)
Brusilovsky, P., Spring, M.: Adaptive, engaging, and explanatory visualization in a C programming course. In: Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp. 1264–1271 (2004)
Brusilovsky, P., Su, H.-D.: Adaptive visualization component of a distributed Web-based adaptive educational system. In: Proceedings of the 6th International Conference on Intelligent Tutoring Systems (ITS), pp. 229–238 (2002)
Byrne M.D., Catarambone R., Stasko J.T.: Evaluating animations as student aids in learning computer algorithms. Comput. Educ. 33(5), 253–278 (1999)
Case R., Kurland M.D., Goldberg J.: Operational efficiency and the growth of short-term memory span. J. Exp. Child Psychol. 33, 386–404 (1982)
Conati C., Merten C.: Eye-tracking for user modeling in exploratory learning environments: an empirical evaluation. Knowl.-Based Syst. 20(6), 557–574 (2007)
Daily L.Z., Lovett M.C., Reder L.M.: Modeling individual differences in working memory performance: A source activation account. Cogn. Sci. Multidiscipl. J. 25(3), 315–353 (2001)
Dancik, G., Kumar, A.N.: A tutor for counter-controlled loop concepts and its evaluation. In: Proceedings of 2003 Frontiers in Education Conference, Session T3C (2003)
Daneman M., Carpenter P.A.: Individual differences in working memory and reading. J. Verbal Learn. Verbal Behav. 19(4), 450–466 (1980)
Dmitrienko A., Molenbergs G., Chuang-Stein C., Offen W.: Analysis of Clinical Trials Using SAS: A Practical Guide. SAS Publishing, Cary (2005)
Graf, W., Krueger, H.: Ergonomic evaluation of user interfaces by means of eye movement data. In: Proceedings of the 3rd World Conference on Educational Multimedia, Hypermedia and Telecommunications Conference on Human-Computer Interaction (HCI), pp. 659–665 (1989)
Henderson J.M., Pierce G.L.: Eye movements during scene viewing: Evidence for mixed control of fixation durations. Psychon. Bull. Rev. 15(3), 566–573 (2008)
Hundhausen C.D., Douglas S.A., Stasko J.T.: A meta-study of algorithm visualization effectiveness. J. Vis. Lang. Comput. 13(3), 259–290 (2002)
Hooge I.Th.C., Vlaskamp B.N.S., Over E.A.B.: Saccadic search: on duration of a fixation. In: Gompel, R.P.G., Fischer, M.H., Murray, W.S., Hill, R.L. (eds) Eye Movements: A Window on Mind and Brain, pp. 581–596. Elsevier, Amsterdam, The Netherlands (2003)
Jarc, D.J., Feldman, M.B., Heller, R.S.: Assessing the benefits of interactive prediction using Web-based algorithm animation courseware. In: Proceedings of the 31st SIGCSE Technical Symposium on Computer Science Education, pp. 377–381 (2000)
Just M.A., Carpenter P.A.: Eye fixations and cognitive processes. Cogn. Psychol. 8, 441–480 (1976)
Kenward M.G., Roger J.H.: Small sample inference for fixed effects from restricted maximum likelihood. Biometrics 53, 983–997 (1997)
Kerren, A., Stasko, J.: Algorithm animation—introduction. In: Diehl, S. (ed.) Software Visualization State of the Art Survey, pp. 1–15. Springer, (2002)
Kerren, A., Mueldner, T., Shakshuki, E.: Novel algorithm explanation techniques for improving algorithm teaching. In: Proceedings of the ACM Symposium on Software Visualization (SoftVis), pp. 175–176 (2006)
Krebs, M., Lauer, T., Ottmann, T., Trahasch, S.: Student-built algorithm visualizations for assessment: flexible generation, feedback and grading. In: Proceedings of the 9th Conference on Innovation Technology in Computer Science Education (ITiCSE), pp. 281–285 (2005)
Kobsa A., Koenemann J., Pohl W.: Personalised hypermedia presentation techniques for improving online customer relationships. Knowl. Eng. Rev. 16(2), 111–155 (2001)
Kumar, A.N.: Model-based generation of demand feedback in a programming tutor. In: Proceedings of the 11th International Conference on Artificial Intelligence in Education (AIED), pp. 425–432 (2003)
Kumar, A.N.: Results from the evaluation of the effectiveness of an online tutor on expression evaluation. In: Proceedings of the 36th SIGCSE Technical Symposium on Computer Science Education, pp. 216–220 (2005)
Lahtinen, E., Ahoniemi, T.: Annotations for defining interactive instructions to interpreter based program visualization tools. In: Rössling G. (ed.) Electronic Notes in Theoretical Computer Science, Vol. 178, pp. 121–128 (2007)
Littell R.C., Milliken G.A., Stroup W.W., Wolfinger R.D., Schabenberger O.: SAS for Mixed Models. 2nd edn. SAS Publishing, Cary (2006)
Loboda, T.D., Brusilovsky, P.: Adaptation in the context of explanatory visualization. In: Proceedings of the 3rd European Conference on Technology Enhanced Education (ECTEL), pp. 250–261 (2008)
Loftus G.R.: Eye fixations on text and scenes. In: Rayner, K. (eds) Eye Movements in Reading: Perceptual and Language Processes, pp. 359–376. Academic Press, New York, NY (1983)
Manor B.R., Gordon E.: Defining the temporal threshold for ocular fixation in free-viewing visuocognitive tasks. J. Neurosci. Methods 128(1–2), 85–93 (2003)
McCullagh P., Nelder J.A.: Generalized Linear Models. CRC Press, Boca Raton, FL (1989)
Moreno, A., Myller, N., Sutinen, E., Ari, M.B.: Visualizing programs with Jeliot 3. In: Proceedings of the Working Conference on Advanced Visual Interfaces (AVI), pp. 373–376 (2004)
Naps, T.L., Eagan, J.R., Norton, L.L.: JHAVE—an environment to actively engage students in Web-based algorithm visualizations. In: Proceedings of the 31st SIGCSE Technical Symposium on Computer Science Education, pp. 109–113 (2000)
Naps T.L., Rössling G., Almstrum V., Dann W., Fleischer R., Hundhausen C., Korhonen A., Malmi L., McNally M., Rodger S., Velázquez-Iturbide J.Á.: Exploring the role of visualization and engagement in computer science education. ACM SIGCSE Bull. 35, 131–152 (2002)
Naps T.L., Rössling G., Anderson J., Cooper S., Dann W., Fleischer R., Koldehofe B., Korhonen A., Kuittinen M., Leska C., McNally M., Malmi L., Rantakokko J., Ross R.J.: Evaluating the educational impact of visualization. ACM SIGCSE Bull. 35(4), 124–136 (2003)
Nevalainen, S., Sajaniemi, J.: An experiment on short-term effects of animated versus static visualization of operations on program perception. In: Proceedings of the 2nd International Workshop on Computing Education Research, pp. 7–16 (2006)
O’Regan J.K.: Optimal viewing position in words and the strategy-tactics theory of eye movements in reading. In: Rayner, K. (eds) Eye Movements and Visual Cognition: Scene Perception and Reading., pp. 333–354. Springer-Verlag, New York, NY (1992)
Paas F., Renkl A., Sweller J.: Cognitive load theory and instructional design: Recent developments. Educ. Psychol. 38, 1–4 (2003)
Price B.: A principled taxonomy of software visualization. J. Vis. Lang. Comput. 4(3), 211–266 (1993)
R Development Core Team: R: A Language and Environment for Statistical (2009)
Rayner K.: Eye movement in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372–422 (1998)
SAS Institute Inc.: SAS 9.2 help and documentation (2008)
Salthouse T.A., Ellis C.L.: Determinants of eye-fixation duration. Am. J. Psychol. 93(2), 207–234 (1980)
Stasko, J., Badre, A., Lewis, C.: Do algorithm animations assist learning? An empirical study and analysis. In: Proceedings of the 5th International Conference on Human-Computer Interaction (INTERCHI), pp. 61–66 (1993)
Sweller J.: Cognitive load during problem-solving: Effects on learning. Cogn. Sci. 12, 257–285 (1988)
Turner M.L., Engle R.W.: Is working memory capacity task dependent?. J. Mem. Lang. 28, 127–154 (1989)
Velichkovsky B.M., Rothert A., Kopf M., Dornhoefer S.M., Joos M.: Towards an express diagnostics for level of processing and hazard perception. Transp. Res. F 5(2), 145–156 (2002)
Velichkovsky, B.M., Joos, M., Helmert, J.R., Pannasch, S.: Two visual systems and their eye movements: evidence from static and dynamic scene perception. In: Proceedings of the 27th Conference of the Cognitive Science Society, pp. 2283–2288 (2005)
Weber G., Brusilovsky P.: ELM-ART: an adaptive versatile system for Web-based instruction. Int. J. Artif. Intell. Educ. 12(4), 351–384 (2001)
Yamamoto, Y., Hirose, H.: Result of applying study support system that flow chart diagram displays by synchronizing with source program code to education. In: Proceedings of the 10th World Conference on E-Learning (E-LEARN), pp. 1186–1192 (2005)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Loboda, T.D., Brusilovsky, P. User-adaptive explanatory program visualization: evaluation and insights from eye movements. User Model User-Adap Inter 20, 191–226 (2010). https://doi.org/10.1007/s11257-010-9077-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11257-010-9077-1