Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Dialectical Creativity is the act of formulating a new concept through the original idea (the thesis), developing opposing contradictory ideas (the antithesis), and culminating on a more developed concretized idea that both negates and encompasses both the thesis and the antithesis (the synthesis). Sketching is a fundamental part of ideation. The act of performing ideation with an inherently abstract hand-drawn sketch, complete with messiness, allows the sketcher, through the misinterpretation of their own strokes, to evoke antithetical concepts, enabling the sketcher to quickly develop a creative synthetic idea. In the dialectical process there is a constant tension between creative change and the natural tendency to seek stability. Sketch recognition is the automated understanding of hand drawn diagrams by a computer, and can be used to both enhance creativity and/or idea stability. This paper discusses the Sketch Dialectic and its impact on the field of sketch recognition.

Introduction to the Dialectic

At its most simplistic essence, dialectics is the practice of using logical argumentation to investigate the truth of a theory or opinion. An early influential expression of this concept was in Plato’s Socratic Dialogues, stemming from the idea of having two competing ideas going through a dialogue to eventually come to a state of better understanding. To gain an intuition for the choice of the term, notice that the term dialectic is related to the word ‘dialect.’ ‘Dialectical’, ‘dialect’, and ‘dialogue’ all stem from the same Greek word dialektos from dialegesthai, which means to “converse with each other,” from dia- “across, between” + legein “speak”.

However, this definition consisting of a purely logical argumentation does not get to the heart of the dialectic method, which requires the examination of both the original idea and the idea’s negative to come to a synthetic conclusion. In Plato’s Socratic Dialogues [2], Socrates was a contrarian and provided or encouraged the development of an alternative hypothesis for any concept, trying to prove each idea wrong. The Socratic Method uses repeated oppositional questioning and the removal of hypotheses to produce an improved hypothesis. Socrates says that he never gives birth to any ideas (Theaetetus 148e–151d) [2], but rather uses repeated questioning to see if the ideas hold merit. Socrates himself would say that he has no ideas of his own, but he is just good at testing existing ideas. In this manner, Socrates is providing the antithesis for any idea.

These practices anticipated the Hegelian concept of the dialectic. The Hegelian dialectic further develops the dialectic process into a tri-phase model to explain the process of human history and the creation of truth through the tri-phase model of thesis-antithesis-synthesis. The dialectic works in a spiral-like repetitive process towards an ideal state, where each synthesis becomes the thesis for the cycle of dialectic idealism.

Although the Hegelian dialectic is associated with the terms thesisantithesis-synthesis, Hegel himself attributed these terms to Kant [10, 16], and instead used the terms abstract-negative-concrete and/or immediate-mediated-concrete, preferring these terms because he said that the antithesis implies that the negative comes from outside of a thing, whereas the negative comes from within and is inherent to the thing itself.

While the dialectic is associated with a quest for idealism, truth, and a concept of evolution and progress, a postmodern interpretation of the dialectic, and an abandonment of the concepts of evolution, progress, and truth, may lead us closer to the Deleuzean concept of lines of flight, involution, and becoming. With Deleuze, rather everything is inherently instable, and inevitably finds itself in a period of transformation.

The Deleuzean concept of becoming is differentiated from the dialectical process in many ways, two of which are significant here (a) its abandonment of the concepts of evolution towards and ideal and (b) its abandonment the negative and/or the antithesis. The dialectic is primarily concerned with progress from one point of history to another, with a formulation of betterment. In Plato’s Socratic Dialogues, Socrates is trying to see how close he can get to an ideal, and attempts to approach that space through the examination of the negative. Contrastingly, Deleuze abandons this concept of the negative and progress towards an ideal. For Deleuze, all states and spaces are naturally and inherently unstable. For Deleuze, all objects go through the natural and repetitive process of: I am here, yet “here” will inevitably self-destruct, and I will have to find a new place. Deleuze states: “The term we would prefer for this form of evolution between heterogeneous terms is ‘involution’, on the condition that involution is in no way confused with regression. Becoming is involutionary; involution is creative. To regress is the move in the direction of something less differentiated. But to involve is to form a block that runs its own line ‘between’ the terms in play and beneath assignable relations” [4, 5]. The Deleuzean neoevolutionaryism concept of creation is both a movement of things away from other things without direction and without the Darwinian concept of progress, and a destruction of boundaries of what we thought we saw in order to see a particular entity.

When discussing creativity, both the concept of idea evolution (in terms of the standard concept of the dialectic) and idea involution (in terms of the Deleuzean concept of becoming) lead us to perhaps find a better fit for the current instability. The next section will discuss how the model of creativity fits both the traditional concept of progress as well as the postmodern concept of resettling in a place of inherent instability.

Dialectical Creativity and the Sketch Dialectic

Dialectical Creativity is the act of formulating a new concept through the original idea (the thesis), developing opposing contradictory ideas (the antithesis), and culminating on a more developed concretized idea that both negates and encompasses both the thesis and the antithesis (the synthesis). Traditionally, the Hegelian dialectic and Deleuzean becoming concern itself primarily with the evolution or involution of human society, contrastingly, dialectical creativity refers to ideation or the maturation of an idea.

Dialectical Creativity

Sketching is a fundamental part of ideation. The act of performing ideation with an inherently abstract hand-drawn sketch, complete with messiness, allows the sketcher, through the misinterpretation of their own strokes, to evoke antithetical concepts, enabling the sketcher to quickly develop a creative synthetic idea. The sketch is critical to creativity. In discussing or encouraging creativity, the pen and paper are quick to produce themselves. When experimenting with an idea, designers are quick to create a sketch to test out and manipulate an idea. Brainstorming is the dominant method for group creativity. More concrete design methods, such as CAD systems, are prevalent, but they are readily abandoned during the creative process. The Sketch Dialectic is the process of using Dialectical Creativity within the framework of sketching.

What is it about the sketch, the pen, and brainstorming that makes them so fundamental to creativity? In the dialectical process there is a constant opposition between creative change and the natural tendency to seek stability.

In the Hegelian dialectic version of Dialectical Creativity, the sketch is used to help designers’ progress to an ideal solution that they are striving towards. The Encyclopedia of Creativity states: “In a dialectical process, the double functions of creativity are exercised. The positive function of creativity generates and constructs new concepts one after another. The negative function of creativity destroys preconceptions, displaces concepts, and breaks mental sets that would block imagination. Concurrent to the process of affirming new concepts, old concepts are being negated” [28]. Yan and Arlin continue to emphasize that any idea can be examined through the dialectical context: “When the process of dialectic is put in the context of a dialectical worldview, the creative nature of dialectical thinking would be magnified. From a dialectical perspective, every ‘thing’ is inherently contradictory with no limitation to the number of contradictions for each thing. In fact, each thing, as a part of a whole and also the whole of many parts, has multiple properties. As the thing interacts with a broader milieu, the number of properties multiplies and the nature of the properties changes. As one property has inherently on set of contradictions, one thing with multiple properties would have multiple sets of contradictions and, therefore, multiple corresponding resolutions. In this light, the development of a creative process is dialectical.”

In this dialectical process there is a constant opposition between creative change and the natural tendency to seek stability. This constant opposition creates a discomfort zone from which new and better ways of representing reality continually emerge. The overall process of dialectical thinking is, therefore in essence a process of self-perpetuating renewal and of self-perpetuating advancement. Some of the highest forms of creativity appear to be dialectical in nature. They often involve processes such as combining and recombining ideas, searching for complementarity, and coordinating multiple perspectives. Arlin in 1990 also noted that being dissatisfied the status quo, seeking new ways to formulate old problems, and noticing discrepancies unnoticed before are elements of intuition linked to the creative processes that are dialectical in nature. In fact, each of these processes of creative thinking can be regarded in part and parcel of problem finding, a concept originally used in the definition of a fifth/postformal stage in cognitive development. [28]

A sketch is a natural component of dialectic creativity. Sketching is inherently abstract, inherently imperfect, and we can harness the imperfections of a sketch to highlight negatives, which help to develop a concrete idea. The sketch allows you to remain in the abstract phase while negative are harnessed.

Deleuzean Becoming

The use of sketch within ideation also fits well to the Deleuzean concept of becoming. When we sketch, we don’t draw what we see, we draw what they know. The sketch is inherently a messy incomplete drawing, and in interacting with the sketch, we complete things that aren’t there. Inevitably our perception is faulty, and thus concept instability is going to develop.

Brainstorming is a free-flowing of ideas from one to the next. Sketching allows you to jot ideas informally on paper, but see them visually. A sketch provides you with the opportunity to explain or prove something to yourself. The pen and paper provide static immutable objects that are much more inflexible than a CAD system which allows you to more easily move around virtual objects. Why would these inflexible objects be preferred to mutable virtual objects?

I argue that it is not their immutability that makes a paper sketch more desirable. Rather, it is the fundamental messiness that is at its core. Sketches are, by their very nature, imperfect, and open for interpretation. When looking at a sketch, a designer has to interpret the sketch, has to make meaning out of the sketch, has to translate the sketch from abstract into reality. It is this internal human translation that breeds creativity. How does this internal translation breed creativity? A sketch breeds creativity through misinterpretation of its original intention. This idea of internal human translation or mistranslation of ones own ideas is supported by the idea that creativity in individual brainstorming (or free-writing or mind mapping) can be superior to that produced by group brainstorming [11].

Creativity and Sketching

The importance of sketching to enhance creativity has been well studied. Shah et al. has developed a concept called C-Sketch (collaborative sketch) that encourages creativity by having people trade sketches, and many of the creative ideas actually come from the previous sketch being misinterpreted. Researchers such as Shah, Jansson, and Smith have studied the inherent imperfections of a sketch, showing that the abstract sketch is often misinterpreted, and that those misinterpretations actually lead to greater creativity and better designs by helping to remove design fixation [18, 25].

It has been shown in architecture that having an abstract sketch is pivotal to removing design fixation on the part of the customer. Architects will often work in a design system, but when they show their designs to clients, client will often choose not to offer any suggestions to the drawings, rather they see them as concrete, something that they have to either accept or reject. However, when they offer clients instead, a hand-sketched diagram, clients take the opportunity to see and identify what is wrong in the design to come up with a pleasing solution. As such architects use programs to purposefully sketchify their design. Likewise, it is pivotal for engineers that they be able to sketch out their designs with a paper and pen in order to be able to properly formulate their ideas [23].

This paper describes several items of research that we are undergoing in the Sketch Recognition Lab and their relation to dialectical creativity in terms of the Hegelian dialectic, which may better praise usefulness and effectiveness concepts of creativity, and in terms of the Deleuzean becoming, which may better praise the concepts of novelty and innovativeness concepts of creativity.

The Sketch Dialectic and Sketch Recognition

The Encyclopedia of Creativity defines the dialectic as composed of three parts: “Thesis: In establishing a thesis, one is required to formulate a concept for analysis. Antithesis: In producing a contradictory view of the thesis, one is in fact generating an alternative concept that is in conflict with or in opposition to the original concept. Synthesis: To integrate the contradictory concepts into a dialectical whole, one would have to construct a whole new concept that is neither p nor q.” [28] The dialectic is a constant struggle between enhancing creativity and the stabilization of that idea.

Sketch recognition is the automated understanding of hand drawn diagrams by a computer, and can be used to both enhance creativity and/or idea stability both in the Hegelian dialectical sense and in the Deleuzean Becoming sense. This section discusses the Sketch Dialectic and its impact on the field of sketch recognition.

Enhancing the Negative to Encourage Artistic Production

Perhaps the most obviously related project going on in our lab is that of our Dialectical Sketching Systems that purposefully maladapt and transform a user’s sketches to the stroke style (using gestural stroke features) of an alternative persona to provoke idea instability and encourage creativity.

Techniques for sketch recognition usually fall into one of three camps: gestural, appearance-based, or perception/geometry-based. Gestural recognition focuses on the path drawn, including personal drawing-style information such as the curvature, timing, and pressure information. Although gestural recognition methods are usually used to define sketch-commands to the user, we can also use gesture recognition features to define a user’s natural style of drawing in a freeform sketch environment. Each sketcher has a personal drawing style in the same way that each person has an identifiable personal handwriting. Gestural sketch recognition features can be measured to create a persona for a particular person’s drawing style. This difference between this program and a standard filter is that a filter is designed to produce a desired effect, whereas the personas are designed to negate the desired effect.

We have developed a method of enhancing the negative to encourage artistic production by first measuring the gestural style of a particular person, then modifying the stroke to match the features of an alternative gestural style. The author assigned this task to her undergraduate Human Computer Interaction class. The students made seven different instances of this concept, producing interesting results. A local artist showed significant interest in the programs, and when asked to test one out for 5 min, he ended up glued to the program for over 2 h at his first sitting, leaving only when he was forced to by prior commitments. Despite several bugs being present in the first version, he emphasized that he wanted to use the program in his art class next fall. He stated that he felt that using the program would help learning artists to produce art more freely and confidently; he strives to have his students produce confident lines, owning the line and confirming each lines correctness even if the line was not the line that was originally intended. His favorite persona was that of the ‘drunkard’ which most significantly altered his original strokes, but he felt strongly that he could still ‘see’ his strokes in the seemingly random mess of strokes. Figure 1 is Andy Warhol inspired progression of a Campbell’s Soup can.

Fig. 1
figure 1

An Andy Warhol inspired progression of a Campbell’s Soup can

Full size image

Usually when you put a pen down on a paper or digital computer, the result is as expected, with the mark following your pen. What if the marks following behind were not yours, but instead those of another personality? Your calm strokes translate to someone’s who is anxious and panicked, or vice versa. What effect does that have on creativity? Mapping a user’s sketched input to a predefined persona can lead to interesting forms of interaction. Sketch modification using personas turns a user’s input into a dynamic sketch slightly beyond the user’s control. This induced modification pushes the user out of her comfort zone and, by altering her work, can produce novel interactions with, and responses to, the computer. A user may find that she is entertained or possibly frustrated by the computer’s alterations of her work, Figs. 2, 3, 4, 5 and 6.

Fig. 2
figure 2

Progression of a car

Full size image
Fig. 3
figure 3

Progression of a mouse

Full size image
Fig. 4
figure 4

When does the drawing cease to be Mickey? What about the Nike sign?

Full size image
Fig. 5
figure 5

Glasses. This is reminiscent of the “Choose Your Own Adventure” stories from childhood (a Rhizomatic narrative?). At each step along the way multiple lines of flight are possible

Full size image
Fig. 6
figure 6

Images produced from a version of the system that changes strokes in real time, forcing users to adapt to the altered versions of their strokes

Full size image

Teaching Art and Creativity

The above system can be used to help teach art and creativity through the enhancement of the negative. In the case above, the negative comes from the computer system. A more formalized artistic instruction system could use sketch recognition techniques to enhance the creation of the synthetic. Interestingly enough, people don’t draw what they see; they draw what they know. This is why traditional vision algorithms work so poorly on hand-sketched data. This is another reason why a sketch is so helpful to people because the contrast between the hand-sketch drawing and the actual object has to be rectified in their mind to create a synthetic ideal that is neither the original object nor their hand-drawn interpretation of the object. We can use dialectical creativity to explain the internal concept of a sketch: Thesis: The object being sketched. Antithesis: The sketched object. Synthesis: The internal perception of the object that both combines and negates both the original and the sketched object.

The goal in many artistic drawing programs is to get people to be able to sketch out the synthesis. As such the first step in such a program is to teach people to sketch the thesis (since what they sketch is already the antithesis) to that they can eventually sketch out the synthesis, Fig. 7.

Fig. 7
figure 7

The iCanDraw system teaching a user how to draw a face and an earlier version teaching perspective drawing of a door

Full size image

The Sketch Recognition Lab has created the iCanDraw [7] program that uses sketch recognition techniques combined with vision and artificial intelligence techniques to teach users how to draw what they see (the thesis), as opposed to what they know (the antithesis). The program aims to teach them hand-eye coordination to provide them empowerment over the pen tool and the original object itself. The iCanDraw system teaches users how to draw a human face accurately. The system takes in a photograph that a user would like to learn how to draw and performs face recognition on that photograph to obtain a template of the face. The system then teaches the user to draw the face in a step-by-step manner using the process defined in the book ‘Drawing on the Right Side of the Brain’ [8, 9], and uses sketch recognition to provide real-time feedback and to provide an overall evaluation metric that compares the a face recognition produced template of the final sketch to the original photograph’s template.

The above system teaches users how to draw what they see rather than what they know, but this is only the first step in teaching drawing, artistry, and creativity. By combining what the users see with what they know, sketchers can create a synthetic concept that is negates both what they see and what they know and that has greater creativity than either idea alone. Figure 8 shows common caricatures found on the internet given an example.

Fig. 8
figure 8

Common caricatures found on the internet given an example

Full size image

Our goal is to be able to aid users to create a synthetic sketch that combines what they see and what they know in a way that is creative. As a first next step, we plan to aid users in the creation of vision-based and personality-based caricatures. In the vision-based caricatures, we will compare the face to the ideal (most common) human face and focus on what is most different from this ideal face. Are the eyes too far apart? Are the ears or nose too big? Then the caricature would enhance this negative to make this even more pronounced. We can use face recognition, sketch recognition, and mathematical models of the face to automatically determine possible variations for caricatures as well as automatically guide and teach the user how to identify variant features and draw their exaggerated caricature counterpoint. In the personality-based caricatures, we will merge a human face with an animal, either chosen by the user or automatically searching for shared adjectives used in describing both the animal and the person.

Collaborative Sketching

Collaborative sketching has proved to be beneficial to removing design fixation and causing improved ideation when using the Modified 635method of C-Sketch [25]. The effects of electronic sketching have been studied by [6]. Distributed sketching can prove to be a necessity in many circumstances as technological advances continue as more and more people choose to work in a distributed fashion, Fig. 9.

Fig. 9
figure 9

The sketches produced by the four groups, shown in the order that they were created. The highlighted sketches were drawn in an isolated context

Full size image

We created a collaborative sketching system called CoSke [3] that allows users to sketch collaboratively. Strokes appear in real-time to users as the user proceeds to lie ink down on the page (as opposed to at the competion of a stroke as is the case in the recent Google Docs sketching program). We tested the program on 12 users divided in to four groups of three. Each group was asked to sketch a dragon, a bowl of fruit, and a house, in that order. Each sketch was performed in a different circumstance (with the circumstance order varied from group to group), either on a traditional piece of paper using colored pencils, using the CoSke system where all users were on individual computers but in the same room (so they could talk to one another), and using the CoSke system where they were placed in separate rooms so they were unable to speak to one another.

The participants were asked which method of interaction they preferred as well as the advantages and disadvantages for each. Disadvantages mentioned on the digital method were always because of technical issues, such as slow down, mis-functioning eraser, no undo, hard to draw straight lines, etc. Contrarily, the disadvantages of paper always included a comment implying that paper impeded the participants’ ability to collaborate. Although none of the question specifically addressed the question of the physical interaction of paper sketching, 10 out of 12 users, including each user who preferred the paper interface, specifically commented on how paper affected their ability to collaborate because of the shared physical space.

Given that users felt obliged to mention this perception despite the lack of a specific question suggests the importance of further research on the following questions: Does paper impede sketch-based collaboration? Do user’s need their own ‘space’ in order to collaborate effectively? Below we list the related comment from each user and their paper or digital preference:

U1: “I believe the paper interface didn’t let multiple users to perform at the same time [in] the collaborative environment” (preferred digital) U2: “Pencil and paper was fun, but [it was] hard to draw at the same time.” (preferred digital) U3: “I like to have control over my own drawing tool. The pencils were easier to control than the pen. However, it is more difficult for a group of people to draw with the same picture on a single sheet of paper collaboratively.” (preferred digital) U4: “Drawing on [the] same piece of paper requires more cooperation.” (preferred digital) U5: “The digital method was easier because you didn’t have to worry about the physical constraints of the other’s hands. You could draw in the same spot[more easily.” (preferred digital) U6: “Both were easy to use, but we had to wait for each other on the paper one, because there wasn’t enough space.”(preferred paper) U7: “It was easier to use digital because I don’t have to share the stylus as opposed to sharing color pencils.” (preferred paper) U8:“[Paper provided an] opportunity for verbal and physical interaction.” (preferred paper) U9: no related comment (preferred digital) U10: “Paper was easier, except that some people had to draw upside down, which was a huge benefit of the digital method.” This user also mentioned that their group did not collaborate and all drew on their own parts of their paper.(preferred paper) U11: Digital was easier because we didn’t have to awkwardly move our hands to make room for others. (preferred digital) U12:no related comment (preferred digital).

Combining speech with sketch proved to be pivotal for effective collaboration. 9/12 users preferred the collaboration with speech, and comment such as: “In different rooms there was no communication … lack of communication was difficult. I ended up drawing over people once or twice.” were quite common. Interestingly, the three who preferred interacting without speech also noted that the lack of speech impeded collaboration: U: “Drawing without knowledge of what others were supposed to do, without set roles made it better.” U: “There was no need to argue or communicate with others, we just drew. Everyone was working by themselves.” U: “Isolated drawing made me feel like I was interacting with a language I knew nothing about and yet spoke more naturally than English… Interaction is very hard to coordinate when isolated, but once immersed, it becomes constant.”

Looking at the figures produced through the different methods, no one method seems to produce overwhelmingly more creative solutions. Rather, it’s possible to generalize that each method performed similarly. Creativity seemed to increase over time, probably because the participants became more comfortable with each other as a creative group.

To Beautify or Not to Beautify within Traditional uses of Sketch Recognition within Design

The Sketch Dialectic implies that in order to maintain the creativity-inspiring functions of a sketch, that when performing sketch recognition it is critical that we need to leave the sketch in its original strokes form. If we allow the sketch recognition to translate hand-drawn objects directly into a concrete form of a human’s intention, just as in the case of working directly with a CAD system we can loose the natural creativity inspiring function of a sketch. This is not to say using sketch recognition in this automated translation method is not without merit, as there are many cases when interacting with a system using a pen is more simpler and more intuitive than interacting with a mouse and palette system. Immediate translation of pen strokes to their concrete intended meaning can significantly improve human computer interaction paradigms in many instances. However, while it improves the interaction paradigm, it does not increase creativity, since users are working with ideas that exist in well-defined and concrete terms, rather than interpretable ideas. I argue that it is in this act of internal interpretation that humans become more creative in that construct.

Sketch recognition does not have to deter creativity. It is only in the act of automatic concretization does sketch recognition hinder creativity. We can still use sketch recognition to understand user’s diagrams while allowing them to maintain creative inspiration. In the design process, imagine if a user could draw and interact with their original strokes, while all the while the strokes are being translated simultaneously into the computer. The user can see and misinterpret their sketches in order to create these lines of flight. However, having their strokes recognized by the computer creates a number of obvious benefits.

The most repeated benefit is that upon finishing their sketch, designers do not have to repeat their final ideas into the computer for use within a CAD system. The computer can automatically interpret the sketch finalize the idea, and synthesize the concept. Usually this synthesis provides the added benefit of seeing this design in action [Gro96], such as a mechanical part in motion [1, 13, 21], generated code [14], mathematical solution of a problem [LZ04], or 3D walk through of a 3D space [20] Users get to work in the way that is most natural to them, significantly improving their interaction experience while still providing the benefits of a CAD tool and the creativity of a piece of paper. Users can get immediate feedback as to whether their design has merit. A blank piece of paper encourages creativity much more than a computer-automated drawing system does. Brainstorming is most often done with a pencil and paper because it does not constrain input. A constrained CAD drawing is easily recognizable by a computer. Graffiti provides a language for drawing shapes, but this language has to be learned. A human can understand and comprehend the drawings of another person, without needing to constrain the drawing style of the sketcher. Sketch recognition allows a computer to understand and automatically process natural hand drawings, Fig. 10.

Fig. 10
figure 10

Recognized diagrams of a mechanical engineering and a flowchart diagram

Full size image

The benefits of sketch recognition in education are plentiful. Instructors and students sketch an abundance of graphical diagrams throughout the educational process. Graphical diagrams are particularly important and prevalent in Physics, and used throughout the learning process. However, correcting these diagrams proves difficult and time-consuming. Oftentimes, these graphical assignments or tests are omitted for these reasons. This is unfortunate as pedagogical studies suggest that not only does testing aid in learning, but that including testing in the educational process is more effective than test preparation alone [RK06]. Roediger explains that students remember more of what they learned when alternating only two study sessions with two testing sessions rather than by having four study sessions. Roediger also describes how early feedback after testing increases the percent of the material learned. Sketches can help in conceptual understanding, and instructors use sketches to communicate ideas [17, 27].

The drawing of these diagrams and automatic feedback is valuable to the student’s learning process; however because the diagrams are often time-consuming to grade, the teacher may omit the drawing requirement and opt for a multiple choice testing scheme. Automatically correcting and understanding students’ graphical diagrams will provide immediate feedback to the student and to the teacher about the student’s understanding. The effect of this learning technology may encourage more instructors to include more graphical diagrams in their set of test questions, because previously time-consuming hand-correction of graphical diagrams can now be done automatically. Sketch recognition can help students to form a better conceptual model of the material, this research allows students to view the same problem in both modes, integrating creative and functional thought by allowing them to hand-draw images, and then providing them with immediate simulation feedback. Particular educational advancements include the ability to:

  1. 1.

    Provide active learning of concepts with a new interactive learning process,

  2. 2.

    Combine creative visualization (by providing students with drawing freedom by allowing them to hand-draw their diagrams as they would naturally) with functional visualization (simulating their own hand drawn diagrams along with enhanced editing possibilities),

  3. 3.

    Enhance learning by providing immediate feedback of homework, classwork, and test question answers,

  4. 4.

    Save instructor resources by automatically correcting student graphical work,

  5. 5.

    Provide instructor awareness of student comprehension by reporting and collating student results to the instructor,

  6. 6.

    Prevent cheating by generating different questions from semester to-semester or even student-to-student.

The benefits from the above scenarios come from the recognition of a finalized diagram. However, online recognition can provide many benefits to the creative process. An interpreted diagram can be manipulated and edited in a way that hand drawn sketches cannot. Perceptual grouping can be used to identify which items should be moved together [24], providing intuitive editing and diagram manipulation that has a significant advantage over static pen and paper diagrams that must be erased and redrawn. Additionally, because the diagram is recognized in real time, the designer can get feedback at any stage along the process to see if the concrete version of their current diagram would hold merit or work in the way that they were supposing. For instance, a civil engineer might want to first check that a particular beam can withstand the necessary force before building an entire bridge based on that assumption. Computers can be used along the process to provide feedback about parts of the design process.

Likewise, a computer that understands the sketch as it unfolds can watch the design process and automatically record design rationale based on the ever-changing sketch. In the work on UML class diagrams, we created a system that users could use in a software design meeting to automatically capture and document design rationale. The software designers designed an API using hand-sketched UML class diagrams in an intelligent room where both the white board and the videotaped session of the meeting was recorded. The hand-drawn UML class diagrams were understood in real time and the creation of editing of these class objects were used to automatically index the recorded meeting for later reference. Users could ask the system, “What were we discussing when we decided to create this class?” Significant events could be ranked and identified, so that users could also ask the system to “provide a history of the five most significant points in the design process.” [15], Fig. 11.

Fig. 11
figure 11

Image of the system being used in a design meeting, a UML diagram produced, its cleaned up interpretation, and the ranking of the creation of each object and link

Full size image

Conclusion

This paper discusses and defines Dialectical Creativity and the Sketch Dialectic and its impact on the field of sketch recognition. In the dialectical process there is a constant opposition between creative change and the natural tendency to seek stability. The imperfections of hand-drawn sketches inherently aid the production of antithetical concepts that can help increase creativity. Sketch recognition is the automated understanding of hand drawn diagrams by a computer, and can be used to both enhance creativity and/or idea stability.