Articles in this special issue display innovative and sophisticated approaches to developing and probing models of self-regulated learning (SRL). I posit a first step in considering what these novel methodological tools offer is developing a clearer sense of data needed to represent the fullness of SRL. I take this tack following this logic: Because characteristics of data shape choices about analyzing data, a researcher needs to probe how features of questions about SRL can be manifested in data.

What are topics of data that describe self-regulated learning?

To provide a heuristic for thinking about SRL, I invented a 5-component model that is compressed by a first-letter acronym: COPES (Winne 1997). The letter C represents conditions a learner identifies as bearing on a task. Conditions could be internal to the learner, e.g., a forecast about efficacy or a judgment about the utility of succeeding at the task. (See Winne and Hadwin 2008, for a fuller treatment of motivation and SRL.) Or, conditions could be external factors in the environment that the learner judges may affect how work on the task might unfold. An example is whether, in the midst of a task, information could be looked up in the Internet or help was available from a peer. Letter O marks operations a learner can carry out. Operations may be unobserved cognitive processes or observable behavior. Operations are how a learner works on the task by manipulating information and elements (objects, people) in the task environment. P refers to products that operations generate. As with conditions, products may be internal, such as a judgment of learning; or external, such as results returned by a search in the Internet. E identifies that an active learner evaluates qualities of work done in a task by monitoring how features of that work match against standards, represented by the letter S.

Using COPES as a guide, what happens when a learner engages in SRL? Answers are both simple and complex. In simplest form, SRL is triggered when a learner evaluates something just done was not done satisfactorily. Because SRL is self regulated, satisfaction is the learner’s prerogative regardless of goals or demands set externally. An evaluation that a feature of a task is less than satisfactory affords an occasion to change something or plan to change it in the future. In contrast, an evaluation of satisfactory lays track for the learner to repeat what was just done. In theoretical terms, the metacognitive operation of monitoring sets a stage for subsequently exercising metacognitive control to regulate tasks in the future. There are fundamentally two options: change or repeat (Winne 2011). In other words, SRL involves monitoring experiences of learning and learning from those experiences—being metacognitive—about how to approach a task. Self-regulating learning is forward-reaching transfer (Salomon and Perkins 1989) that focuses on methods for learning.

Complication quickly intrudes on posing the question, “What changed?” or “What was maintained?” The COPES heuristic affords a learner control over three components. One malleable component is a condition. For example, if a young learner is counting days until an upcoming birthday, maintaining the correspondence between days of the week and the count may be too demanding. An option is to draw a calendar and mark days as the count proceeds. This, of course, is dependent on at least one internal condition, knowledge about calendars as representations of dates; and at least one external condition, access to drawing materials. It also requires a product generated in an embedded task, one in which the learner produces an estimate (a product) that there is sufficient chance of success (an evaluation) in drawing an accurate a calendar. To make that evaluation, the learner may make a vague judgment about competence (efficacy) to draw a calendar or try to mentally picture a calendar. The product of this embedded task establishes conditions for generating a forecast about success in using a calendar representation for counting days. This complexity illustrates that events an experimenter or teacher might observe—the learner expressing frustration, which we infer is due to challenges in counting mentally, then drawing a calendar—may arise from a cascade of embedded tasks, each of which is itself a full expression of COPES. Embedded and typically unobserved tasks are targets for theory. Surfacing as much as possible about embedded tasks, using traces of their products, is one key to nurturing and testing theory (Winne 1982; Zhou and Winne 2012).

A second component a learner can change in self-regulating learning is an operation. In the example just given, the learner didn’t change the operation of assembling a correspondence between days and numbers to, say, searching for an estimate of the number of days. Assembling and searching are two of five operations I propose as a heuristic set for describing operations in SRL. The full 5-element set is: searching, monitoring, assembling, translating and rehearsing—the SMART operations (Winne 2004, 2005). Somewhat imperfectly, a “script” for SRL can be developed by coupling these operations with content to which the operations are applied.

A third component a learner can change in SRL is standards chosen for comparing (monitoring) attributes of work on a task to a list of satisfactory or optimal qualities. In this regard, standards reflect a broad set of theories in educational psychology. The learner just described apparently held a standard about effort or efficiency when operating to answer the question about days remaining until a birthday. Externally representing the task with a calendar was “easier” than doing the computation mentally. It might be inferred the learner also held a standard favoring accuracy over a rough guess about days remaining. These possibilities illustrate that standards can apply to more than the focal product of a task, a count of days in the case of my example. Standards apply to all the features in COPES, such as: clarity about conditions, efficacy expectations about and cognitive load forecast for carrying out operations under particular conditions, qualities of products such as precision and completeness, the usefulness of evaluations in shaping progress toward goals, and even qualities of standards per se such as their number and clarity.

Mapping methods to COPES data

The COPES heuristic makes explicit that temporal sequences and patterns are inherent in SRL. A temporal view of unfolding SRL affords various questions:

  1. 1.

    What marks the start of a time interval or a pattern of events? Correspondingly, what marks the interval’s or pattern’s terminus? Regarding both intervals and patterns of events, there are two kinds of answers to these questions: one where an experimenter selects markers a priori, e.g., every 10 s from when an experimenter says, “Begin.” An alternative is when marks arise “naturally,” e.g., when the student provides an utterance in a think aloud paradigm, such as “Now I understand.” which marks an interval’s terminus or an event.

  2. 2.

    Are spans of an “event” constant or varying? The former is a methodology that samples by rigid bins, e.g., every 10 s or a window of constant span over a timeline of events. The latter methodology generates samples based on whenever target events occur, e.g., what occurs between two target events, such as whatever actions appear between a learner’s choice to seek information from a software avatar and the learner’s choice to try answering another question.

  3. 3.

    Are events patterned? Patterns can be discovered by mining data or by stipulating them a priori. Patterns can be found within a span as well as among spans.

  4. 4.

    What are parameters of patterns? Answers to this question are many. Three common parameters are: length (number of events, duration), purity (the degree to which a pattern of events across time does not contain events that are not part of the pattern), and nesting (whether sub-patterns exist within a larger, enveloping pattern).

  5. 5.

    Does the appearance of a pattern or do parameters of patterns covary with other data of interest? For example, among learners who share a pattern, is there a negative correlation between the duration of that pattern and achievement? In other words, are more efficient learners more expert at learning when they use the pattern?

The COPES heuristic suggests possibilities for topics of fine-grained data that might be gathered to investigate these questions. For example, if a researcher is interested in characterizing metacognitive control, should the time interval be initiated when an evaluation of a product is unsatisfactory—an opportunity to hold an intention about exercising metacognitive control—or should the interval’s terminus be marked by an operation that differs from the operation(s) that generated the product that was judged less than satisfactory?

What studies in this special issue accomplished

Kuvalja et al. (2014) developed a priori categories of children’s self talk and of observable behaviors to operationally define data (see their Table 1). Some of their data are very fine grained (see their Table 4), e.g., a single operation such as searching for a particular object (a building on the game board) or a product (e.g., specifying the next step).

Their log sequential analyses used an invariant interval duration anchored by a focal onsetting event. They identified patterns of varying purity within an interval ±3 s over the timeline centered on the marker. Their analyses employ artistry in selecting the interval duration. Guidelines are not yet available, so an empirical question worthy of attention is how to set that duration.

A second analytic method, t-pattern analysis, was applied by Kuvalja et al. to search for recurring patterns. It requires fixing three parameters of a pattern: frequency, specifically, a minimum number of occurrences to be recognized as a pattern; the chance probability of observing a pattern; and the minimum prevalence of a pattern in the intervals identified (i.e., the density of a pattern). Again, choosing values for these parameters currently requires artistry because empirical work is lacking to guide choices for values of these parameters.

Kuvalja et al.’s analyses identified differences in SRL, manifested in self-directed speech, between children with a specific language impairment and those characterized as normally developing. One might complain their results have limited generalizability because the researchers, rather than data, answered my Question 4—What are parameters of patterns? But, as a leading edge study, this complaint could be set aside because the researchers are contributing to an underdeveloped empirical base that is needed to answer matters that question addresses.

Malmberg et al. (2014) logged data that detailed how students used software tools to work on well-structured and ill-structured problems. Input to their analyses of learning patterns were actions such as highlighting an interesting detail, creating a note (node) in a concept map and linking notes in a concept map. These events reflect operations: monitoring content for interestingness, assembling information from a source into a unit (a note) and assembling units (notes) into larger conceptual structures.

Adapting a method introduced by Winne et al. (1994) that uses matrices describing transitions of events across the timeline of engagement, Malmberg et al. investigated patterns within event sequences under an a priori constraint that a contingency (If A, Then B) must be observed at least three times in a learning session to count as a pattern. This a priori approach to setting a threshold for treating repeated contingencies as a pattern contrasts with Kualija’s et al. who used a statistical threshold they established based on data in hand. While some might be quick to prefer the statistical approach, it should be noted that using statistics to avoid a type I error (claiming a pattern exists in data when it truly is not) cannot assure that the pattern is consequential in the sense of describing a “sizable effect” (role in the overall episode, covariance with an outcome variable) that is worth attention. At the early stage of work like these studies demonstrate, both statistical and “effect size” approaches should be pursued. A challenge here is determining the cost of making a type I error which should guide selection of a statistic’s critical value.

To answer questions of interest in their research, Malmberg et al. supplemented analyses on graphs of transitions across events using several other methodologies. This widened a focus on operations to encompass types of learning strategies based on qualitative analyses of which information students operated on with which software tools. How students used learning strategies served as a way to partition learning patterns that distinguished students who were on track in solving problems compared to students who were off track. In short, their multimethod approach takes a step toward deepening understanding about how learning strategies manifest as patterns of learning events that unfold across the timeline of a learning episode. However, because the rules governing qualitative work often are difficult to replicate, there is a tradeoff. Gains in understanding SRL compete with a goal of generalizability.

Molenaar and Chiu (2014) comment that “current models of cognitive and regulative activities are too general to specify how their micro-level relations support effective collaborative learning”. They used a statistical methodology called social discourse analysis to remedy this fault. Their data were codes applied to transcripts of students’ face-to-face conversations as they collaborated to write a report for school. As students worked, they could receive either of two types of scaffolding: structuring scaffolding that illustrated metacognitive activities by example, and problematizing scaffolding that was designed to straightforwardly guide students’ metacognition. These two conditions were contrasted to collaboration au naturel, without scaffolding. Molenaar and Chiu hypothesized that structuring scaffolds reduce task complexity while problematizing scaffolds increase it. In this context, they explored sequential relations among three classes of activities: metacognitive, cognitive and social.

Social discourse analysis is a sophisticated statistical method. It meets many challenges when serially correlated data are nested within conversational structures, some events are infrequent and temporal partitions are not fixed. In these senses, the methodology is designed to tackle all the questions I posed for considering SRL in light of the COPES heuristic model, including how variation in COPES elements associates with various between-subjects factors.

Social discourse analysis has several benefits. Beyond providing a statistical method for partitioning data and statistical criteria for judging when events form a pattern, it generates effect sizes for a pattern. The methodology also affords examining the onset and distribution of events across the timeline.

Is there a soft spot in this methodology? As is true of all the methods that log and analyze traces, the researcher must be very adept at operationally defining traces and reliably coding data. This powerful a statistical method must be fed high quality data about COPES elements. Statistics cannot rescue data that carry a weak or noisy signal.

Bannert et al. (2014) adopt a methodology called process mining. It was developed to investigate properties of procedural operations in manufacturing, business and management, including whether a process as realized matches the process as it was designed to operate. Process mining also can be used in an exploratory way to identify patterns in temporally sequenced data. Output from the method constructs a “graph” or map that depicts a pattern as events (nodes) and their relationships (edges). Three “effect sizes” are available. Each event is quantified by its importance in the pattern. Relationships are indexed by their significance in the pattern and how tightly the relationship binds nodes it links. These results directly address questions 3 and 4 as posed earlier.

Process mining is a powerful tool for identifying patterns of SRL. Once patterns are identified, other methods can be used to explore realtions to other variables, such as students’ efficacy for learning or achievement. Like several other methodologies, process mining requires artistry of the researcher to set values for various parameters that determine, e.g., when a class of events has a small enough role in the overall system that it should be absorbed into a larger class of events to “simplify” or purify the pattern.

Bannert et al. coded undergraduates’ think aloud data and kinds interactions students had with software tools while they studied a subject. Then, the researchers formed high and low groups distinguished by students’ achievement to explore for variations in patterns. They were able to show that students at widely differing places on the achievement continuum study differently. Question 5, posed earlier, was directly addressed.

Two comparisons of process mining and statistical discourse analysis merit note. One arises when process mining drops classes of events from patterns it discovers. In statistical discourse analysis, the researcher could be shown at the conclusion of an analysis the probability of type I error associated with including an event in a pattern. (It appears the output reports only standard values for p at 0.05, 0.01 and 0.001 but I expect this could be modified to report an exact p.) It is the researcher’s decision whether the risk is bearable when dropping an event class. In process mining, parameters governing an analysis are set a priori. This may result in event classes being aggregated “behind the scene” such that the final pattern that process mining reveals masks how aggregation was done. Researchers using process mining might inquire about events aggregated “out” of a pattern due to values assigned to parameters at the outset of an analysis. The question as to which approach is better is currently moot. The second comparison of note is that process mining offers a benefit in the multiple “shapes” that can be represented as a pattern. Statistical discourse analysis provides a linear timeline. Which is more useful depends on the theoretical issue being investigated.

Kinnebrew et al. (2014) investigated learners’ metacognition in a context of software called Betty’s Brain. This software was designed as an environment where students can work on developing skills to seek information, construct knowledge, investigate a system of relations, assess the correctness of knowledge and consider records of progress as a basis for adapting how they go about learning. Actions available in Betty’s Brain are designed to trace metacognition corresponding to each skill by: setting goals and planning how to reach them, building knowledge, evaluating results and seeking help.

Kinnebrew et al. applied sequence mining techniques to identify patterns among traces representing features of SRL. Intervals are identified by their method according to the relative activity level of a student’s engagement with Betty’s Brain. In this sense, less active learners are not “penalized” when the methodology searches for patterns of SRL. After sequences were identified, the researchers classified them in relation to their model of cognitive skills, as well as emergent patterns such as “trial and error.” Learners also were classified by the frequency of strategy use (patterns) and whether the appearance of patterns across the timeline of their engagement with Betty’s Brain increased, decreased slowly or decreased rapidly.

An interesting result of Kinnebrew et al.’s analyses is that learners were observed to regulate patterns. That is, considering a pattern as a “unit” of regulated learning, the pattern may generate a product that feeds forward to influence subsequent patterns. In other words, patterns may regulate patterns alongside regulation within a pattern. This captures the notion of embedded tasks. Kinnebrew et al. used differences in the frequencies of patterns as well as variations among patterns of patterns to infer differential SRL among learners and divergent effects for two forms of scaffolding. Their methodology addresses all five of the questions I posed, and it introduced an interesting normalization technique to deal with question 2 about whether intervals are constant or varying.

Conclusions

The research presented in this special issue offers a nourishing array of new techniques for investigating SRL. The studies rest on a common assumption: patterns of events can complement (and some may claim, replace) self-reports (including think aloud) and aggregate counts of events in unlocking the puzzle of how SRL is constituted. These researchers and I share the view that further research focusing on patterns of events can sharpen theory and, eventually, recommendations about how to nudge learners toward elevated levels of achievement and satisfaction.

Studies of SRL as patterned activity are at an early stage. What might be recommendations for future work? I offer four.

First, there is broad agreement that SRL is metacognitive. There is less agreement about components that constitute SRL and patterns of components that instantiate metacognitive monitoring and metacognitive control. When researchers adopt various models for SRL, their models guide them to gather different data and consider different approaches to analyzing data (Kuhn 1996). Exploratory work like that presented in this special issue plays an important role; it expands considerations about how events and their patterns describe can SRL. But progressive science prunes models. I recommend the field begin transitioning from exploratory work to confirmatory studies that test models. Methods illustrated in this special issue can support this evolution.

Second, a common element across models of SRL is that learners have goals (e.g., see Winne 2014). Goals instantiate standards for metacognitive monitoring within the COPES complex of factors. A feature of goals is that they have a subject—they are about something. I posit two fundamental categories of subjects for goals. One subject of goals addresses what is learned, the content per se and its features. The second category of goals concerns features of learning as a process. This is a varied list that includes, among many things: operations used, efficacy expectations, perceptions about the pace of progress, judgments about the degree to which study tactics and learning strategies satisfy, and opportunities and consequences when experimenting to adapt operations. Methods that track or induce patterns that characterize SRL need to explicitly identify when goals arise, their position(s) within patterns of SRL, and the subjects of goals. Moreover, because SRL affords learners the option to adapt goals along the timeline of work, methods need to reveal conditions that lead learners to adapt goals. These two kinds of data will enhance modeling SRL as a goal-directed activity.

Third, my experience suggests learners are rarely equipped with a variety of options—tactics and strategies—for carrying out learning. Task analysis, planning, mnemonics, and patterns of tactics that form learning strategies are too infrequently taught in educational institutions. As a result, learners develop capacities for SRL mostly on their own and are rarely supplied feedback to guide them. The upshot is truncated variance in SRL au naturel. Restriction of range can hamper research about SRL. A remedy that also aligns with experimental tests of competing models of SRL is to prepare (train) learners in components of SRL and in multiple patterns that instantiate theory. Learners so equipped would be in stronger position to express variance in exercising SRL. Such training should attend to motivational elements of action. Specifically, learners should come to attribute effort and ability (skill) as keys to success, validly estimate positive efficacy for each option, accurately forecast the incentive for using each option under particular conditions, be clear about the product(s) (outcomes) each option generates, and be able compare profiles of these factors to gauge the utility for exercising the various options. This AEIOU set (attributions, efficacy expectations, incentives, outcome expectations, utilities; Winne and Marx 1989) of motivational factors is a map of conditions that sets the stage for differentially exercising metacognitive control. Correspondingly, traces of motivation should enrich data fed to analytical methods. Zhou and I illustrate one approach (Zhou and Winne 2012).

Fourth, occasions for regulating learning—staying the course or making a change—arise when learners evaluate features of COPES in SRL, including evaluations about evaluations provided externally (e.g., by a software agent). To make a reasoned change, the learner needs an accurate record of what led to that evaluation. As Santayana is often quoted, “Those who cannot remember the past are condemned to repeat it.” I recommend future research about SRL help learners recapture an accurate sense of how they went about learning by inventing displays of traces and patterns of traces that learners can interpret. Traces about how learners examine and use such process feedback then might be gathered simply by having learners circle or talk about a region of a timeline of traces. There are two consequences for methodologies. One is that such data lessen experimenters’ need to infer how learners themselves perceive the learning they are self-regulating. (Note, I do not mean that statistically quantified inferences about patterns should be abandoned.) Another is that these data trace conditions upon which a learner subsequently engages in metacognitive monitoring as a first step toward SRL.

In summary, the analytical methods displayed in this special issue are very welcome. First, each offers a revealing approach to investigating SRL. In their own ways, each is a powerful tool that will accelerate progress in the sector of learning science concerned with how learners fashion their learning. Second, as a set with contrasting features, these methods stimulate deeper thinking about data to gather about SRL and how to manipulate those data in modeling the complex and vital phenomenon of SRL. I look forward to future syntheses of findings as more researchers take up these methods.