Abstract
Design sketching is an important and versatile skill for engineering students to master in order to translate their design thoughts effectively onto a visual medium, whether it is to proficiently produce hand-drawn sketches onto paper, seamlessly interact with intelligent sketch-based modeling interfaces, or reap the various educational benefits associated with drawing. Traditional instructional approaches for teaching design sketching are frequently constrained by the availability of experienced human instructors or the lack of supervised learning from self-practice, while relevant intelligent educational applications for sketch instruction have focused more on assessing users’ art drawings or cognitive developmental progress. We introduce PerSketchTivity, an intelligent pen-based computing educational application that not only teaches engineering students how to hone and practice their design sketching skills through stylus-and-touchscreen interaction, but also aiding their motivation and self-regulated learning through real-time feedback. From the qualitative results of our usability tests of our application from eight university student participants of varying skill levels and disciplines, we observed that participants well-rated the usability of the application while also providing valuable feedback to improve the application even further.
Access provided by Autonomous University of Puebla. Download chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
1 Problem Statement and Context
1.1 Benefits of Design Sketching
Engineering students must achieve expertise in a wide range of computing skills during their studies in order to properly prepare them for their future careers. However, design sketching continues to be a valuable skill that is applicable in a variety of use cases, since this skill enables engineers to translate their design thoughts effectively on a visual medium. These use cases can range from proficiently producing rough preliminary sketches and polished drafting designs using pen and paper, to seamlessly interacting with gesture-driven intelligent user interfaces and emerging sketch-based modeling applications with a stylus-and-touchscreen. Furthermore, engineering students can take advantage of the educational benefits associated with sketching in general, such as enhancing the communications of existing ideas [6] and inspiring a generation of new ones [5]. Mastery of design sketching can lead to benefits in creating diagrammatic representations that hold many advantages over textual representations alone, such as grouping information more usefully for improved problem-solving inference [8] and better representing ideas for conveying engineering ideas [1]. Combined with current engineering education, sketching therefore provides assistance in accessing knowledge that is based in the applied sciences through aesthetic and social inquiry.
1.2 Challenges of Traditional Design Sketching Instruction
While mastery of design sketching enables engineering students to exploit this skill for their coursework and subsequent real-world usage in post-university careers, the process of them achieving mastery provides its own challenges. From our interviews with two industrial design professors from a large research university and from university students who are learning how to sketch in order to apply this skill to their coursework, we were given insights into both the traditional sketching pedagogy and the difficulties that both students and instructors experienced. Among these difficulties for learners include low self-efficacy [2], discouragement from comparison with peers, and lack of motivation to practice and self-regulate their learning [11]. As for instructors, factors include difficulty in catering their teaching to each and every individual. Traditional approaches for teaching sketching principles primarily stem from either in-class instruction and dedicated workshops, which are constrained by the time and availability of experienced human instructors; or rote self-practice and sketching guidebooks, which lack supervised feedback for alleviating the possibility of incorrect learning and bad sketching habits [10].
1.3 Prior Work
With the challenges associated with traditional design sketching instruction, there is potential in engineering students taking advantage of the growing ubiquity and accessibility of pen-capable computing devices to utilize intelligent pen-driven educational applications, where such an application can guide the students’ study of design sketching fundamentals through sketching practice and automated human instructor-emulated visualized feedback. We can see such examples for sketching in the domain of art (e.g., [3, 4]), where sketching interfaces guide students in drawing foundational concepts by providing step-by-step cues on drawing and automated real-time feedback on their drawing performance. Similar examples also exist for assessing young children’s developmental progress through sketching (e.g., [7]) using playful interfaces that additionally guide children how to sketch basic shapes. However, there is a lack of work in developing appropriate educational applications that cater specifically to users with developed fine motor skills with little to no design sketching background for more technical domains such as engineering and design.
1.4 Proposed Solution
In this paper, we therefore introduce PerSketchTivity, our intelligent educational application for teaching design sketching to users through sketching practice and corresponding automated feedback on their sketched input, in order to address the various challenges that both students and instructors encounter during design sketching instruction. Our approach for automatic understanding of freehand sketch input relies on: (1) a variety of geometric, feature, temporal, contextual, multi-modal, and domain information that provides context about what shape is being drawn on the screen, (2) a way to describe the sketch information, and (3) the application of machine learning techniques that take advantage of, and learn from, the information provided. Additionally, insights from educational sketching books (e.g., [9]) help influence the design decisions of our interface’s interactive lessons using existing successful practice for sketch instruction.
2 Method Employed
We implemented a web-based intelligent educational application that utilizes Paper.JS—an open source javascript vector graphics scripting framework—and which runs on top of HTML5 Canvas. Our application’s interface is therefore platform-independent and appropriate for pen-driven input modalities. The accompanying hardware setup that we employed for our application consisted of a Wacom Cintiq touchscreen monitor that accepted accurate stylus input, but can easily be adapted to other hardware setups that accept stylus input such as mobile tablets and tabletop touchscreens.
In the latest version of our application, we designed an interface that consisted of several exercises—specifically those for teaching users how to draw straight lines, squares, and circles—while also accommodating future extensions for lessons on more advanced geometric primitives. We elaborate on our application’s different exercises in this section.
2.1 Straight Line Exercise
Straight lines are the most fundamental technique of sketching, and the very first sketching technique designers and engineers are taught is typically straight lines, as they form a fundamental skill from which to construct all three-dimensional forms. Therefore, we introduce straight lines as our first interactive lesson within our application. Students are exposed to a “connect the dots” style of exercise, and typically practice repeatedly hundreds of times to master the technique. With traditional pedagogy, there is a focus on accuracy, speed, and line quality, so by leveraging existing sketch recognition approaches, we are able to provide real-time feedback in the form of color-coded bars (Fig. 8.1), as well as an evaluation that provides students with additional metrics regarding their accuracy and speed (Fig. 8.2).
2.2 Square Exercise
Comprised of straight lines perpendicular to their adjacent neighboring lines and parallel to lines on the other side, we introduce squares as the next basic shape for students to practice learning sketch technique on. Advancing from line exercises thus allows students to sketch multiple straight lines, in order to construct basic two-dimensional forms (Fig. 8.3). Multi-stroke detection allows students to sketch four lines for the squares.
2.3 Circle Exercise
Using the knowledge of how to draw squares, our application subsequently allows students to practice sketching perfect circles inside reference squares (Fig. 8.4). Our application uses scaffolding in this way as a fundamental concept for students to experience during their sketching lesson, as more complex forms must similarly be “constructed” from basic forms.
2.4 Future Exercises
With students completing their lesson practice on these prior sketching fundamentals for two-dimensional geometric primitives, we would like our application in the future to incorporate more advanced exercises that enables students to learn about perspectives and how to sketch three-dimensional geometric primitives such as cubes, cylinders, cones, and sphere (Fig. 8.5).
3 Results and Evaluation
3.1 Usability Study
In order to evaluate the performance of our current intelligent educational application for design sketching instruction, we recruited eight university students from a large research university for a usability study. They were from a variety of disciplinary backgrounds such as industrial design, mechanical engineering, computer science, and human-computer interaction. These students covered a spectrum of self-reported sketching experience from beginners with little to no sketching experience to intermediate-level students who still wanted to improve their ability.
For our usability study, we asked the participants to practice on four developed exercises, two of which utilized sketch recognition approaches. The focus of the exercises were on fundamental sketching components of lines and circles, each with two of their different exercise styles. One exercise style involved sketching many-at-a-time with an evaluation at the end, while the other exercise style involved sketching one-at-a time and evaluating between sketches. Since we designed our interface as a computer-assisted educational application, we therefore focused on learning outcomes and whether or not the pages and exercises were accomplishing the learning goals instead of traditional usability heuristics (e.g., task completion time, error rate). Our evaluation therefore placed greater emphasis on qualitative feedback, but also included collecting quantitative data using performance metrics to measure the effectiveness of pagination, exercises, and the user interface itself for determining their usability effectiveness.
3.2 Lesson Map Page Evaluation
One component that we included in our application was a lesson map page for showing the user what they will be learning, what they had already learned, and what can be navigated in the educational modules (Fig. 8.6). Using a self-assessed ten-point scale, our participants gave an average of 9.3 in terms of understanding what they will be learning, and an average of 8.0 for understanding what they had learned. In addition to the accompanying qualitative data, participants overall understood what they were going to learn from the lesson map page, but were less clear on what they had already learned or what they would learn next. Their feedback suggested having a more clear path between lessons and a visual indicator of their progress through each lesson.
3.3 Lesson Overview Page Evaluation
Another component included in our application was a lesson overview page for educating users on a specific skill or technique, as well as serving as a navigation for the exercises (Fig. 8.7). Using a ten-point scale, our participants gave an average of 9.4 in terms of understanding how to sketch the prompted shape using video, and an average of 8.9 for understanding how to sketch the prompted shape using pro tips infographics. In addition to the accompanying qualitative data, the lesson overview page received high performance metrics, but participants largely expressed having a desire for less scrolling and more clicking interaction from this page.
3.4 Shape Exercise Evaluation
The main component of the application was the different shape exercises, and participants were asked to progress through the different sketching lessons for the various prompted shapes. Overall, users on average rated the second iteration of the shape exercises at least 8.6 for initially understanding what to do in the exercise, at least 9.1 for their level of focus during the exercise, and at least 9.0 on their ability to sketch the prompted shapes more accurately. The accompanying qualitative data varied for each shape, but the general consensus was that the shape exercises were well-received while some had issues with the timing aspects of the application, such as the exercises being timed and timing oversensitivity of the prompted cues.
3.5 Interactive Coach Evaluation
A supplemental component of the application was the incorporation of an interactive coach to warm up users to the educational experience within the application. We proposed three candidate interactive coaches, and participants were most receptive to the candidate whom visually resembled historical artist Leonardo Da Vinci (Fig. 8.8).
3.6 Assessment Display Evaluation
One other component in the application was the assessment display, which was provided after completing a lesson (Fig. 8.9). Overall, participants reacted positively to and feeling accomplished with the automated assessment of their sketches, especially with real-time deviation lines and regardless of minor evaluation errors. One remark from participants was that they were open to assessment being displayed as either a five-star rating or as a full performance metrics overview, which we can implement in the future as a toggle. An additional remark was an option to view a history of their previous scores to self-determine their own improvements of their sketching ability over time.
3.7 Overall Insights
From our evaluation, we observed the usability testing demonstrated the value of both exercise designs in terms of motivation, states of flow, and subjective opinions on their effectiveness as learning tools. We discovered that one effective approach would be to combine the positive aspects of both exercise styles into a single exercise style (Figs. 8.10 and 8.11).
A major flaw in our current interface design that we partially anticipated was that participants did not fully understand what to do before practicing a new exercise (Fig. 8.12). We believe that more care should be taken in guiding users through the experience and without surprising them. Furthermore, we observed that scrolling should generally be eliminated from the interface due to the way people interacted when using styluses.
4 Contributions
This paper introduces our preliminary prototype and evaluation of the PerSketchTivity system. Our results show that traditional paper-based methods for learning sketching can be translated to the digital medium. Further research will evaluate its effectiveness in raising motivation, self-efficacy, self-regulated learning, and spatial reasoning skills.
References
Atilola O (2007) The effects of representation and analogy on engineering idea generation. Ph.D. thesis, Georgia Institute of Technology, Atlanta, Georgia, USA
Bandura A (1997) Self-efficacy: the exercise of self-control, 1st edn. Freeman, New York
Cummmings D, Vides F, Hammond T (2012) I don’t believe my eyes!: geometric sketch recognition for a computer art tutorial. In: Proceedings of the international symposium on sketch-based interfaces and modeling. SBIM ’12. Eurographics Association, Geneva, Switzerland, pp 97–106
Dixon D, Prasad M, Hammond T (2010) iCanDraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In: Proceedings of the SIGCHI conference on human factors in computing systems., CHI ’10. ACM, New York, USA, pp 897–906
Franken R (2011) Human motivation, 1st edn. Cengage Learning, Boston
Gardner H (1983) Frames of mind: the theory of multiple intelligences, 1st edn. Basic Books, New York
Kim H, Taele P, Valentine S, McTigue E, Hammond T (2013) KimCHI: a sketch-based developmental skill classifier to enhance pen-driven educational interfaces for children. In: Proceedings of the international symposium on sketch-based interfaces and modeling pages, SBIM ’13. ACM, New York, pp 33–42
Larkin J, Simon H (1987) Why a diagram is (sometimes) worth ten thousand words. Cogn Sci 11(1):65–100
Robertson S (2013) How to draw: drawing and sketching objects and environments from your imagination, 1st edn. Design Studio Press, Culver City
Taele P, Hammond T (2009) Hashigo: a next-generation sketch interactive system for japanese kanji. In: Proceedings of the twenty-first innovative applications artificial intelligence conference, IAAI ’09. AAAI, Palo Alto, California, USA, pp 153–158
Zimmerman B (2008) Investigating self-regulation and motivation: historical background, methodological developments, and future prospects. Am Educ Res J 45(1):166–183
Acknowledgments
Thanks to members of the Sketch Recognition Lab and the IDREEM Lab who helped with the ideas of the project and paper, in particular Seth Polsley from SRL and Bill Scott at Stanford deserve a special shout out. This project is funded in part by NSF Cyberlearning grants 1441331 and 1441291 (EXP: Collaborative Research: PerSketchTivity- Empowering and Inspiring Creative, Competent, Communicative, and Effective Engineers through Perspective Sketching).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Williford, B., Taele, P., Nelligan, T., Li, W., Linsey, J., Hammond, T. (2016). PerSketchTivity: An Intelligent Pen-Based Educational Application for Design Sketching Instruction. In: Hammond, T., Valentine, S., Adler, A. (eds) Revolutionizing Education with Digital Ink. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-31193-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-31193-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-31191-3
Online ISBN: 978-3-319-31193-7
eBook Packages: Computer ScienceComputer Science (R0)