Introduction

The human cognitive system is of interest to researchers developing computational applications for various purposes in multidisciplinary fields [1–7]. For more than three decades, researchers have attempted to understand human cognitive abilities. However, despite initial promise, there has been little advance toward profound comprehension [7]. More specifically, research is still needed in regard to relationships between the cognitive system and actual activity [2]. Currently, data are obtained mainly through sensors [2]. The present study concerns obtaining information about a drawing activity by capturing visual parameters and producing them in spatio-temporal sequence [8, 9] via an electronic tablet (digitizer). The digitizer is a noninvasive, unobtrusive tool that detects behavioral biometrics of perceptual-motor task performance [10]. Such insights may be valuable for developing computational medical evaluations as well as for a variety of human–computer interactions [11] involving cognitive abilities.

Multiple studies have demonstrated the advantages of computerized, objective, spatial, temporal, and pressure measures supplied by the digitizer, for capturing the handwriting process (e.g., [12, 13]). These studies have shown that the non-language-dependent measures (e.g., [14]) reflect individual handwriting uniqueness and provide good indicators of clinical and applied conditions (e.g., mild cognitive impairment or Alzheimer’s disease [15], depression [16], and low/high cognitive load [17, 18]).The overall body of research on computerized handwriting analysis (see Fig. 1) offers evidence that physical, cognitive, clinical, psychological, and situational characteristics of individuals can be captured through these computerized outcome measures. Spatial, temporal, and pressure measures reflect attributes of writing that are sometimes visibly evident on the written page (such as writing errors, legibility, letter size). The interesting point is that these measures also capture behavior that occurs above the page (in the air, between strokes, etc.) that has no visible outcome on the written page [17, 19].

Fig. 1
figure 1

Summary of the computerized handwriting process

Full size image

It seems that these measures reflect individual cognitive processes that are relevant not only to the writing of words and numbers. Therefore, the goal of this study was to examine whether similarities would be found between computerized measures of the complex figure and the paragraph copying. In addition, we examined whether those measures would reflect the level of cognitive demand and predict the quality of both the drawn figure and the written product. For these purposes, we employed the Rey–Osterrieth Complex Figure Test (ROCFT; [20, 21]), a non-verbal, neuropsychological test widely used in clinical and research settings for assessing a variety of cognitive abilities [22]. The test comprises two tasks that are graded in relation to the required cognitive resources, namely copying a two-dimensional geometric design and reproducing the design from memory 3 min later, which is more cognitively demanding [22]. The final product of the two tasks is scored according to the accuracy of the reproduction and of its constituent elements in relation to form and spatial position [20].

Despite the differences between figure drawing and handwriting, there are similarities between the mechanisms required for their production. Both require the translation of visual input—an internal graphemic pattern maintained in the working memory—into a grapho-motor code [1]. Such a process involves use of the visuo-spatial sketch pad which retains and enables recall of the visual memory of images [23, 24].

Moreover, figure- and paragraph-copying tasks do not require additional investment of cognitive resources. The figure to be copied is placed in front of the individual, and handwriting is considered as an ‘over-learned’ skill that becomes automatic over time [25], so that it can be performed without conscious effort [26].

Hence, our first hypothesis was that, as a result of the similarity between the cognitive/working memory resources required for both figure and paragraph copying, significant correlations will be found between the computerized measures relating to the individual’s mastery of the drawing/writing device, the actual movement sequence and automaticity that express individual style, and the use of visuo-spatial working memory.

The second aspect of the study relates to mental workload, which is a central concept in regard to cognitive/educational psychology. The definition and theoretical foundations of mental workload are still being discussed worldwide, but it is often defined as the ratio of the demands of a task to available human resources [27]. From a practical perspective, it is the measure of how much of a person’s mental resources are needed to perform a task [28].

Previous studies have shown that the computerized measures of writing under a high workload condition significantly differ from those obtained when writing under a lower mental workload [17, 29]. Furthermore, the computerized measures were shown to be sensitive to many of the clinically pathological conditions characterized by cognitive deficits that can cause high mental workload in writing (e.g., [15]).

From this, we have derived our second hypothesis—that significant differences are to be found between the spatial, temporal, pressure, pen tilt, and azimuth measures of the ROCFT copying and drawing from memory (which is more cognitively demanding).

Previous studies have also revealed a strong link between poor performance and the mental workload of the participant performing the task (e.g., [30]). We suggest that higher mental workloads during writing or drawing, as reflected in the computerized measures, will be predictive of poorer outcome measures of the drawn/written product. According to Sheridan [31], human error is defined as an action that fails to meet some implicit or explicit criterion. Errors of omission and commission in writing also reflect the individual’s mental workload [32, 33]. Thus, mental workload may be reflected in the number of errors made and in accuracy of performance.

In the context of scores for the drawn product of the Rey–Osterrieth test, an omission is defined as failing to copy part of the figure or not including part of the figure when drawing from memory. Incorrect placing of an element, either when copying or drawing from memory, is also considered an omission. In regard to paragraph copying, omissions and commissions are manifest in how the letters are formed. Thus, the number of erasures and of illegible letters constitutes a measure of automaticity and efficiency in writing production [34] and might be related to mental workload and/or visuo-spatial memory.

Consequently, our third hypothesis was that computerized measures of figure copying, drawing from memory, and paragraph copying will predict the quality of the product (i.e., number of omissions as defined by ROCFT scoring and the mean ratio of erasures and illegible letters in a copied paragraph).

Methods

Participants

Participants were 30 healthy, recently inducted male Israeli soldiers, aged 18. Seventy percent of the participants were born in Israel, 27 % in the former Soviet Union, and 3 % in Europe.

Procedure

The subjects were new army recruits awaiting placement. The researchers received official permission to ask soldiers to volunteer as participants in the study. They were told that they had the right to refuse to participate, that all the information would remain absolutely confidential, and that their participation would not affect their army service in any way. After signing an informed consent, each soldier was asked to copy the Rey figure on a blank A4 paper placed on the digitizer, as part of the ComPET. The figure was then removed and, after a 3-min pause, participants were requested to draw it from memory, again on a blank A4 paper placed on the digitizer in the same manner as the initial copying task.

Materials

The ROCFT: Performance Scores

The ROCFT is a test of visual–spatial organization whose reliability and validity are well established (e.g., [22, 35]). The subject is required to copy a figure and to then draw it from memory 3 min later. During the 3-min interval, the evaluator had a small chat with the subject. Both the copying and drawing from memory were manually scored by two research assistants according to the Waber and Holmes scoring system [36, 37]. The scoring takes into account the number of parts reproduced without errors (accuracy) [22], the degree to which the parts are organized in relation to one another (organization), and the style in which the figure is reproduced (part-oriented or configurationally) [38]. The figure includes 18 elements. Each element that is reproduced both accurately and in proper spatial orientation earns a score of two points. Maximum score = 36. To prevent comparing of scores, one research assistant scored all the copied figures, and the other scored all figures drawn from memory.

Computerized Measures of Drawing and Writing Performance

Objective spatial, temporal, and pressure measures were provided by the Computerized Penmanship Evaluation Tool (ComPET), developed by Rosenblum et al. [39]. This system enables collection and analysis of spatial, temporal, and pressure handwriting data when the subject writes upon a paper affixed to a WACOM Intuos2 (model GD 0912-12X18) x–y digitizing tablet. The participant writes with a wire-less electronic pen with a pressure-sensitive tip (Model GP-110). Displacement, pressure, and pen-tip angle are sampled at 100 Hz by means of a 1300 MHz Pentium (R) M laptop computer.

The following measures were used to compare strokes performed in the figure copying/drawing from memory and the paragraph copy:

  1. 1.

    Temporal measures (in seconds):

  2. 2.1

    Mean and standard deviations of on-paper stroke duration.

  3. 2.2

    Mean and standard deviations of in-air stroke duration (when the pen is raised above the paper)

    The mean velocity of the entire task as measured in cm/sec’ was obtained and analyzed.

  4. 3.

    Spatial measures:

  5. 3.1

    Mean and standard deviations of total stroke path length in millimeters—from starting point to finishing point of each written stroke.

  6. 3.2

    Mean and standard deviations of stroke height (Y axis) in millimeters—i.e., the direct distance from the lowest to the highest point of the stroke.

  7. 3.3

    Mean and standard deviations of stroke width (X axis) in millimeters—i.e., the direct distance from left to right of the stroke.

Measures obtained for each total task included:

  1. 4.

    Pressure—mean and standard deviations of pressure applied to the writing surface throughout the task, measured in non-scaled units from 0 to 1024 (for further details, see [4]).

  2. 5.

    Pen tilt and standard deviation from 0 to 90 degrees throughout the task. Tilt refers to the angle between the pen and the tablet surface (i.e., between the pen and its projection on the tablet) (see Fig. 2).

    Fig. 2
    figure 2

    Pen azimuth and tilt

    Full size image
  3. 6.

    Pen azimuth and standard deviation of azimuth from 0 to 360 degrees throughout the task. The azimuth is the angle between the ‘North’ line—the reference line for 0° (and for 360°)—and the pen’s projection on the tablet (see Fig. 2).

The standard deviation of each measure reflects its variability during performance.

Written-Product Measures

The mean number of erasures and of illegible letters (letters that could not be recognized due to the poor quality of letter closure, rounding of letters, or reversals) in the written product was manually counted and computed as a measure of the formal quality of the written text. An illegible letter that also included erasures was counted as 1.

The criteria for the quality score of letter formation were taken from the standardized handwriting tool, the Hebrew Handwriting Evaluation (HHE, [40]).

Data Analysis

Descriptive statistics of the dependent variables were tabulated and examined. Paired t tests were conducted in order to compare the number of strokes drawn while copying and drawing from memory, and for the ROCFT scores of these tasks. Multivariate analysis of variance (MANOVA) was performed for each of the two tasks in order to compare differences between the means and standard deviations of the computerized temporal spatial, pressure, pen tilt, and azimuth measures.

Pearson correlations were calculated in order to investigate the association between computerized measures of paragraph copying and those of the ROCFT copying and drawing from memory.

Stepwise regression analysis was conducted to find:

  1. 1.

    Variance of predictability of traditional ROCFT final copying and drawing-from-memory scores from the computerized measures of the ROCFT tasks,

  2. 2.

    Variance of predictability of mean written-product formation scores from computerized measures of paragraph-copying performance.

Results

Hypothesis 1 suggested that similarities would be found between objective measures of drawing and writing.

This was supported by the significant correlations, ranging from .37 to .75, between figure-copying and paragraph-copying data for several temporal, spatial, pressure, pen tilt, and azimuth measures, as presented in Table 1.

Table 1 Correlations between figure copying and paragraph copying for identical computerized measures
Full size table

Hypothesis 2 suggested that the computerized measures of the ROCFT drawing-from-memory process (which demands more cognitive resources than copying) would differ from those of the ROCFT copying process.

As a preliminary examination, we compared the traditional scores of the ROCFT copying and drawing-from-memory performance. The significant differences found indicate that participants were indeed more successful in ROCFT copying than in drawing from memory (M = 35.34 ± 1.29; M = 22.26 ± 8.47, t(25) = 8.39, p < .001).

To compare the computerized measures of the task products, a paired t test analysis was conducted. Results indicated significant differences between ROCFT copying and drawing-from-memory, relating to the number of strokes drawn (copying: M = 113.00 ± 20.07; drawing from memory: M = 79.59 ± 27.46 t(33) = 6.97, p < .001).

The difference between these findings was then calculated (number of strokes for figure copying—number of strokes for figure drawing from memory) for each participant (d-stroke) and used as a covariate in the subsequent MANOVA with repeated-measures analyses.

1. MANOVA results with repeated-measures analyses for means and standard deviations of the temporal measures indicated significant differences between copying and drawing from memory (F(4,28) = 13.49, p < .001 2η = .66). A post hoc ANOVA indicated that, in figure drawing from memory, participants spent significantly less time per stroke on-paper and in-air, and that their standard deviation per stroke on-paper was significantly lower than that of figure copying. Means and standard deviations are presented in Table 2.

Table 2 Comparison of temporal measures (stroke duration on-paper and in-air) with means and standard deviations (SD) for copying and drawing from memory
Full size table

Table 3 presents the results for means and standard deviations of spatial measures. MANOVA with repeated measures conducted indicated significant differences between copying and drawing from memory (F(6,26) = 3.44, p = .012 ES 2η = .44). As expected, a post hoc ANOVA indicated that, in the figure drawn from memory, strokes were significantly shorter and narrower, and standard deviation of stroke height was significantly higher, whereas stroke width was significantly lower.

Table 3 Comparison of spatial measures (stroke length, width, and height)—means and standard deviations (SD) for figure copying and drawing from memory
Full size table

MANOVA results with repeated-measures analyses of means and standard deviations of pressure measures indicated significant differences between copying and figure-drawing from memory (F(2,30) = 4.86, p = .015 2η = .24). Post hoc ANOVA indicated that, as expected, standard deviation of applied pressure was significantly higher in drawing from memory as compared to figure copying, reflecting greater performance variability. Means and standard deviations (SD) are presented in Table 4.

Table 4 Comparison of pressure means and standard deviations for copying and drawing from memory
Full size table

MANOVA with repeated-measures analysis of means and standard deviations of pen tilt and azimuth revealed no significant differences between copying and drawing from memory (F(4, 28) = .68, p = .61 2η = .89).

Hypothesis 3 suggested that computerized measures of the drawing and writing process would predict the formal quality of the final production. This was supported by the results of the stepwise regression analysis, indicating that:

  1. 3A.1

    The SD of mean stroke height in the ROCFT copying task predicted 14 % of the ROCFT traditional scores (F(1,30) = 4.85, β = .38, p = .036); mean pressure predicted an additional 12 % (F(2,30) = 4.97, p = .014). Together, these two computerized measures accounted for 26 % of the variance in traditional ROCFT copying scores.

  2. 3A.2

    Mean velocity of the ROCFT drawing-from-memory task predicted 52 % of the traditional score (F(1,26) = 27.69, β = .72, p < .0001), while mean pressure predicted an additional 6 % (F(2,26) = 18.80, p < .0001). Together, these two computerized measures accounted for 58 % of variance of the traditional ROCFT drawing-from-memory score.

  3. 3B

    Mean pressure applied in paragraph copying predicted 12 % of the mean score for formal quality of the written product (F(1,34) = 4.56, β = .−35, p < .040).

Discussion

The aim of this study was to explore whether computerized measures of complex figure-drawing and paragraph-copying tasks can reveal more about the processes involved in their production. We also predicted that these measures would capture the mental workload required for performing these tasks and thus predict their respective performance outcomes.

As hypothesized, several measures of the ROCFT copying were significantly correlated with the paragraph-copying measures. Pressure on the writing/drawing surface and the pen’s tilt and azimuth reflect the individual’s mastery over the tool. Significant correlations were found between mean velocity, standard deviation of mean velocity, and standard deviation of the spatial measures (stroke height, width, and length). We suggest that these measures may reflect the actual movement sequence and level of automaticity that could be expressions of individual style and the way in which the writer utilizes the visuo-spatial WM sketch pad.

The next stage was to examine whether the level of cognitive demand could be detected by means of computerized measures. Analogous to the results of previous studies, the conventional scores of the ROCFT drawing-from-memory task were significantly lower than those of figure copying (e.g., [22, 41]). To draw a figure from memory, the individual uses visual–spatial working memory and attention in order to maintain its visual–spatial image in mind [42].

It is important to note that, despite the higher cognitive demands needed to draw the figure from memory as compared to copying the figure, no significant differences were found in the measures of pen tilt, pen azimuth, and mean pressure. As previously mentioned, significant moderate correlations were found for these measures in the figure- and paragraph-copying tasks (r = .57–.65).The current findings thus reinforce the hypothesis that the measures reflect the individual’s mastery over the pen, independent of the kind of task being performed.

Contrary to our expectations based on the reported literature, and despite the more cognitively challenging demands of drawing a figure from memory, this task was completed in significantly less time (in-air and on-paper) than the figure-copying task. Furthermore, when drawing from memory, individuals drew with significantly shorter and narrower strokes. The less time spent in drawing from memory as well as the shorter and narrower pen strokes may be explained by the participants’ attempts to draw as quickly as possible in order to decrease both the time needed to maintain the memory of the image and the cognitive demand on working memory [12, 24, 43]. While mean temporal and spatial measures were lower, standard deviations of stroke height and applied pressure on the paper were significantly higher. This seems to reflect greater variability in the ROCFT drawing-from-memory performance.

Ultimately, these results indicate that kinematic temporal, spatial, and pressure measures reflect the difference in cognitive/mental load demands of each of the tasks throughout their actual performance. The figure lines drawn from memory demonstrated greater variability than the copied lines, although the figures to be drawn were identical. Variability in drawing from memory may be because this task requires maintaining a visual representation of the figure within the memory while drawing.

From a developmental point of view vis-à-vis handwriting research, greater variability signifies lower levels of automaticity and control during performance. The more skilled and automatic the handwriting act, the less variability there will be in temporal (performance time), spatial (length, height, width), and pressure (applied on or toward a surface) measures, and greater consistency will be evident [44].

This study extends the findings of two previous studies analyzing the influence of cognitive demands on real-life task performance. In the first study, participants were asked to write about true and false autobiographical events, based on the assumption that false writing necessitates higher cognitive demands [29]. In the second study, participants were asked to write three numerical progressions with different gaps and grades of difficulty [17]. Examination of the combined results of the three studies reveals that temporal measures reflected the change in cognitive load only for current and numerical studies. On the other hand, in all three studies, the change in cognitive load was reflected in changes in the spatial measures, of which the tendency of the measures was slightly different across the three studies. Stroke length decreased and standard deviation of stroke height increased in the current and the numerical studies. In the true/false study, stroke length and standard deviation of stroke width increased in the more cognitively demanding task (false writing).

Concerning analysis of activity, in order to draw a figure from memory or complete a series of numbers, the visual representation of the figure or the gap between one number and the next must be retained in working memory throughout the performance. Conversely, when writing a false story in which the narrative is created during the writing process, there is no need to retain any specific morphological or numerical information in visual memory. Previous studies have indicated that, in contrast to tasks with spatial demands, verbal tasks require other cognitive resources such as attention and working memory (e.g., [45]). Park et al. [46] indicated that visuo-spatial and verbal working memories are different processes manifested through the processing speed and performance characteristics of various tasks. It is possible that the amount of active representation and actual cognitive processing required in these tasks influences both the temporal and the spatial measures. Specifically, whereas drawing from memory requires active visual representation, the numerical task is more time-consuming since an arithmetical calculation is also involved, whereas in order to write a fictional story, verbal representation is required.

When examining the combined results of these three studies, the computerized system—including the spatial and temporal measures—is evidently sensitive to changes in cognitive demands and thus captures the ‘lived’ aspects of the human mind [47].

Further support of the sensitivity of the computerized measures to cognitive load was the finding that the quality of the final product could be predicted from the computerized measures of the actual performance; i.e., mean pressure consistently predicted percentage of variance in the product scores of the three tasks (ROCFT copying, drawing from memory, and paragraph copying). This finding relates to the finding in the current study that no significant difference in pressure was found when comparing figure and paragraph copying, or figure copying and drawing from memory, despite the higher cognitive demands entailed.

With respect to mental workload, interaction exists between the individual’s resources and the task demands [27]. Mean pressure applied by the pen on the writing surface was previously found to be sensitive to individual performance capacity. For example, the pressure measure differentiates between people with and without depression or psychomotor slowness [16], as well as between children with typical development and those who are clumsy (developmental coordination disorder [DCD]; [48]). Pressure is one of the most commonly used dynamic features for developing techniques for recognizing handwriting patterns (e.g., [49]), such as for online signature verification (e.g., [50]). Similarly, velocity is also considered to be unique to the individual and a basis of dynamic biometric signature verification (e.g., [51, 52]). In the current study, mean velocity predicted a high percentage (52 %) of variance of the traditional ROCFT drawing-from-memory score. These results are in line with previous findings, indicating that pressure and velocity reflect the individual’s degree of control of psychomotor actions required for accurate task performance (e.g., [13]). Further studies with graded levels of writing/drawing complexity are required in order to find whether these two measures do indeed reflect the individual’s available cognitive resources in performing such tasks.

In sum, we found that the objective computerized process measures are both effective in describing the drawing process and simple to obtain. The system is extremely user-friendly, so that training the research assistant to collect data with this device took less than 1 h. The system continuously detects changes in angle, distance, size, and orientation during performance [9]. Furthermore, it enables glancing into the unconscious aspects of the drawing process such as time ‘in-air’ and amount or variability of pressure [53].

Thus, applying this system with varied drawing tasks can be a practical aid in evaluation of neurological diseases which cause cognitive deterioration such as Alzheimer’s or Parkinson’s [10]. Nowadays there are other clinical evaluations such as the mini-mental-state Examination (MMSE) [54] and clock drawing that are not computerized, even though previous research has indicated the benefits of computerizing the process [55–57].

The insights revealed in this study may be the first step toward achieving automatic activity recognition [2] and contribute to acquisition of more information about the learning and reasoning processes [58] related to individual visual-motor parameters and toward computational applications [9] such as evaluation (and even cognitive robots) [58]. Our results also call for interdisciplinary research with other methods of analysis such as signal processing, machine learning, or biometric analysis (e.g., [9, 10, 59, 60]).

The study has certain limitations, among them the relatively small sample size and the analysis methodology. Future studies of a larger, more heterogeneous (in terms of gender, age, and degree of education) random sampling of participants are recommended, in order to increase generalizability of the results. Additionally, a more advanced analysis method such as a logistic regression-based neural network model [61, 62] is also recommended.