Today’s secondary and post-secondary students often tackle learning projects. These major assignments typically involve researching and analyzing complex information about debatable issues. Examples include: Are genetically modified foods safe? What is the better investment in retirement, equities or bonds? What is the fairest way to aggregate votes in elections?

Learning projects present multiple and challenging problems to solve about information. For example, amongst hundreds of thousands or perhaps millions of sources available in the internet, which are best to select for detailed study? Which sources are trustworthy? How should information in a selected source be mined and analyzed? How can information extracted from multiple and divergent sources be reconciled and articulated? How should a presentation—an essay, seminar presentation or infographic—be designed? How well does a draft meet requirements of the assignment? What revisions could improve it?

Almost every post-secondary institution’s website we’ve surveyed offers online support and advertises workshops where, they claim, students can develop skills needed to address this array of information problems. We interpret these institutions have ample but unpublished evidence students are frequently unprepared to solve information problems. A hopeful logic underlies these institutions’ significant investments: Helping students develop expertise to solve a wide array of information problems, what we call round-trip information problem solving (IPS), will accomplish more than broadening and deepening knowledge about only the particular topics addressed in learning projects. Learning projects are opportunities for students to be introduced to, develop and hone skills for IPS they need for lifelong learning in a fast-changing knowledge-intensive economy.

Learning management systems (LMS) are common technologies in colleges and universities, but data these systems gather can play only a limited role in supporting round-trip IPS. LMSs are designed to facilitate students’ access to content and track mainly coarse-grained events, primarily login and logout, resources downloaded, assignments posted and, in many systems, text students post to discussions. These data provide high-level descriptions of students’ work on learning projects. However, these data do not trace how students work on solving information problems. For example, the LMS does not document which information in a downloaded file the student operates on, e.g., what is highlighted or the content of annotations about evidence offered to support a claim. The LMS does not record whether a student searches for information outside its content repository or what queries are formed to search. In short, how students solve information problems is a black box to the LMS. Consequently, LMS data are insufficient for documenting how students engage in IPS. Without other data, evidence is lacking for developing recommendations about how to promote skills for round-trip IPS.

The usual plan for investigating whether interventions intended to promote IPS skills are successful also is underpowered. Typically, a study assesses students’ performance on learning projects and investigates which factors affected success. But one-shot studies, with or without premeasures, and “horse race” studies examine only final products. These studies can’t identify how students faltered or succeeded while developing their learning projects.

To fill these gaps, we recommend instrumentation be developed that can gather extensive, time-stamped, fine-grained data about what students do as they work on information problems, which information they work with, and how they adapt tactics and strategies in response to feedback. That is, we endorse instrumentation that gathers data about how learners self-regulate learning (Winne 2013, 2017a, b).

Data about just one learning project is a beginning but not sufficient. If extensive, time-stamped, fine-grained data could be gathered describing multitudes of students’ work across multiple learning projects, these big data could be mined to identify patterns of work forged by successful students. Broadcasting those patterns to all the other students is an intervention significantly more grounded in evidence than findings from the field’s gold standard, the randomized controlled trial (Winne 2006, 2017b). Big data like this supply raw material for tailoring learning analytics and cyclically updating how changes in patterns of IPS activities are or are not successful. Over cycles of a student’s work sessions, just-in-case and just-in-time learning analytics could offer person-centered and group-referenced analytics about how to productively self-regulate learning for improving round-trip IPS (Winne 2017b). To approach these goals, we are developing nStudy. nStudy is an all-purpose online system students can use anywhere they connect to the internet. With nStudy, students search for information; select, analyze, and catalog information they mine from found sources in html, pdf or video format; draft learning projects and receive peer reviews to revise those drafts, and refine drafts. nStudy is implemented as an extension to the Chrome web browser supported by a server-side database that warehouses logged trace data. Trace data record the information learners operate on and operations they apply to that information. Peripheral systems on the server extract data, analyze them and generate learning analytics for delivery to students and their instructors (or researchers) on demand or when conditions are matched. In the remainder of this article, we sketch nStudy’s features and information tools we are developing to gather and analyze big data for describing and advancing work on self-regulated round-trip IPS.

1 nStudy

Once the nStudy extension is installed in the Chrome web browser, students can use tools for identifying and operating on information. As students use a tool, events are logged without intrusion beyond using the tool itself. For example, when the student selects text in a webpage by dragging the cursor across a string, nStudy automatically displays a menu. The student can select one of several optional operations, e.g., highlight the selected text or link the text to a previously bookmarked source. Each event creates or modifies an nStudy artifact. For example, highlighting text creates a quote of the selection linked to an anchor within the web page containing the selected text. Operating on information in a source creates a bookmark for that source. These data are ambient, i.e., created in the course of carrying out everyday studying versus interrupting work to create data. nStudy’s ambient trace data are intended to provide as thorough an account as possible of observable events describing how the student works and the information the student used in work. If nStudy’s data were “played back,” it would be as if an observer was watching over the student’s shoulder while completely and precisely identifying each event in the study session.

Supporting systems work on the server. A database logs events occurring in the browser. URLs identify sources of information students view and associate to artifacts students create and modify. If the student makes a note, the database records the “trigger” for the note, e.g., selected text or the URL, the note content—text generated and other features such as checkboxes marked that a researcher prepared for students. Each event is time stamped accurate to approximately 1/10th second. Queries can be constructed to extract particular data from the database and format the file for input to external analysis tools. We are developing R scripts, python packages to analyze data and output raw material which is then reformatted by tools like R Shiny as html learning analytics presented as text, tables and graphs. Students (and instructors or researchers) are notified that a report is available for interactive viewing in the browser.

1.1 Student Tools

1.1.1 Study View

The Study view (Fig. 1) has 3 sections. The main region of the window displays html pages and pdf-formatted documents. A similar window (not shown) displays video content. Students can create four kinds of artifacts for these sources. A bookmark preserves the URL of source information. Students can title a bookmark or leave this to default to the URL. A description can summarize or comment on content in the URL. Tags index characteristics of the bookmark according to the student’s preferred cataloging system or a fixed set of tags made available by a researcher.

Fig. 1
figure 1

Study view of nStudy showing steps to create a quote and a note using a debate template

Full size image

By selecting text in an html or pdf display, or clicking at a point during play of a video, students can annotate select information in a source. When text is selected in web pages and pdf documents, nStudy automatically displays a popup menu. Options invite the student to highlight the text and create a quote of the selection, or create a note or a term. Choosing any of these options highlights the selected text and copies it as a quote to the sidebar on the left side. Quoted text can be tagged to index it by metadata. For example, the selected text can be tagged by type of information, such as “hypothesis;” or marked as a task, such as “verify.” Tags already registered in nStudy, because they were installed beforehand by an instructor or researcher or previously created by the student, are displayed in a type-ahead filtered dropdown list. As the student begins entering characters in the field for tags, e.g., the letter “i,” nStudy reminds the student about tags beginning with that letter, e.g., “important,” “interpretation,” or “introduction” (to a learning project report in progress). Clicking a tag on the list applies it to the artifact.

Notes elaborate highlighted text (or a time mark or span in a video). Notes can introduce or remind the student of a schema for complex information if the instructor or researcher has configured a note template. Note templates are web forms. Figure 1 shows a “debate” note template. Every note can be titled and tagged independently of the selected text to which it is associated. Fields for note contents include: textbox, list of checkboxes and radio buttons, date, link to another nStudy artifact, and thumbnail image. Each field can be labeled as a guide about content to enter in it. A set of labeled fields in a note template represents a schema for information. Within a textbox field, the instructor or researcher can install replacement text, a description in gray font of the kind of information to enter into the field. Replacement text disappears when the student begins to type.

Figure 1 shows steps to create highlighted text, a quote of the selection and a note using the comment template. At the right is a region called the gutter. Each artifact a student creates is automatically referenced by a “nub” in the gutter. Nubs are color coded by class of artifact and show their relative locations in a document. Clicking a nub scrolls to its associated quote and shows any associated notes or terms.

A term is a subclass of the note artifact. Terms are reserved for key concepts in the domain of a learning project. Terms can represent: technical words or phrases (autocracy, electron shell), characters (Romeo, Dr. Who), political figures (Justin Trudeau, Angela Merkel), significant events (Brexit, Sputnik)—whatever is deemed significant in the domain. Instructors and researchers can pre-populate any student’s nStudy database with a set of terms. nStudy automatically adds to the links field in a term’s template any other terms mentioned in the description field. Students, instructors and researchers can view a graphical display of terms (nodes) and relations (edges) as a termnet.

At left is a sidebar that can be opened or collapsed by clicking the tab marked «. Headers identify sections in the sidebar listing all tags applied in the document, each quoted selection and the titles of notes and terms generated in the document. Each header identifies the number of artifacts of its type and can be expanded to show titles of the artifacts within it by clicking ►. Clicking an artifact operates the same as clicking a nub; nStudy scrolls to the associated quote and, if the quote is elaborated as a note or term, opens the note’s or term’s window to show its contents. Titles and the contents of artifacts can be searched by entering text in the Search field. Searching filters the sidebar to show the number of artifacts of each type that contain the text.

Videos can be annotated much like text. Clicking a button pauses play and pops up a menu of options as when text is selected. For quotes, students manually enter what was spoken or a description of what was viewed at that spot. Notes and terms are available. Nubs along the video scrub bar mark relative locations of artifacts across the video’s timeline.

1.1.2 Library View

The Library supports browsing and filtering nStudy’s artifacts by metadata and content within artifacts. A top-level “My Library” folder can be expanded to show a tree listing artifacts’ titles organized by type and, within type, by last date accessed. This mimics the Chrome browser’s history view. Right/control clicking an artifact can open it or copy it into a student-created and labeled folder.

Students can filter artifacts to focus on those with particular data or metadata, e.g., notes containing the text “diplomacy” created within a defined time span or edited within a defined time span, and with specific tags.

Artifacts can be viewed as a table with selectable column attributes, e.g.: name, tags, and date last viewed. Artifacts also can be viewed in “flash card” format with content and metadata. Clicking a link opens the artifact’s source, scrolls to its quoted text within the artifact and opens the artifact itself.

1.1.3 Hub View

The Hub is where information is exchanged among students. Like most systems, exchanges are posted to a panel in temporal order. Each entry is identified by the name (or pseudonym) of its author, a timestamp and a topic, nStudy’s approach to providing threaded discussions. Students author a contribution by typing text in a textbox. The contribution can be tagged and labeled as belonging to a topic. Responses to other students’ posts are developed by first selecting the post or a fragment of it, then developing a reply. Anchors (html links in elevated exponent like format) within the reply link back to the material being replied to in the post, explicitly preserving the “thread” of discussion.

Near the textbox where students enter their contributions to a topic is a dropdown menu labeled “Role” and a display area. Clicking the dropdown menu shows a list of roles configured by the instructor or researcher; examples could be task manager, critic and fact checker. When a role is selected, various “starter” phrases are presented in the display area. For example, the critic role might be offer these starters: “What evidence is there for your claim ___?” “How reliable is evidence for ___?” Clicking a starter pastes it into the textbox where the student can complete the starter or add additional comment.

1.1.4 Map View

Artifacts in nStudy can be added to or removed from a special class of folder that incorporates Cartesian coordinates for spatially displaying artifacts as nodes in a concept map. Students can manually position artifacts and represent relationships between them by adding and labeling lines to join artifacts. As in other views, an artifact’s source can be opened by right/control clicking an artifact’s representation and selecting “Open” from a menu. A sidebar, like that in the study view, catalogs the artifacts the student includes in a map.

1.1.5 Essay View

To draft and finalize the product of a learning project, nStudy provides an html editor with features the student can use to format text (e.g., bold font, hyperlink) and layout (e.g., headings, bullet and numbered lists, row-and-column table). Students can enter text directly or paste the contents from another nStudy artifact into their essay (except a map).

2 The Future of nStudy

Note templates, replacement text, roles and starter phrases in the hub, and pre-stocked terms and tags afford instructors and researchers means to guide students about tackling information problems. Information supplied through these channels is, however, static and unresponsive to any particular student’s work. As demonstrated in the field of intelligent tutoring systems, adaptive software can benefit students (Ma et al. 2014). While AI is beyond the scope of nStudy, we are developing features and capabilities linked to the human learning process that we hypothesize can benefit students’ workflow and content learning as they tackle information problem solving in learning projects.

2.1 Automatic Question Generation

Achievement improves when students are posed questions before and while they study text (Fritz et al. 2014) and when the questions invite self-explaining relational information (Bisra et al. 2016). We have prototyped and are integrating into nStudy a system that automatically generates questions to serve these purposes.

2.2 Automatic Summarization and Recommendation

In the early phases of work on a learning problem, one of the information problems students need to solve is judging whether a lengthy document includes content relevant to a learning project. This can be challenging per se but especially when the student has not yet developed a good grasp of new content in the domain of the learning project. We are developing a system that automatically summarizes documents and another that recommends documents based on a student’s highlighting and terms created (Odilinye et al. 2015).We also plan research to investigate whether a summary of a student’s draft report about a learning project can guide productive revisions of that product.

2.3 Learning Analytics

Self-regulating learners depend on feedback about more than their achievement. Using a work flow that incorporates data extraction, formatting analysis and report generation, we are designing learning analytics reports about how students study and use artifacts in drafting reports within a learning project (Winne et al. 2017). Our approach is to ground learning analytics reports in empirical findings of learning science about how to help learners process information and learn content (e.g., Marzouk et al. 2016).

3 Summary

nStudy and accouterments planned for it are designed to fill a gap in learning technologies by illustrating a form of instrumentation that gathers fine-grained details about how students select, process and use information as they define and work through information problems in learning projects (Roll and Winne 2015). The system has been developed so it can be agnostic with regard to a particular theory of IPS or self-regulated learning. Note templates, roles and topic starters in discussions, learning analytics and other features can be configured according to a researcher’s or instructor’s preferred models. If an agent were created to observe students’ work on the fly, the data processed by the agent and characteristics of its interactions with learners are free of a particular computational framework and theories of learning and motivation.

nStudy illustrates the potential to gather extensive, detailed, time stamped ambient trace data about learning activities. It and systems like it offer tools for developing and researching supports for self-regulated learning based on evidence about how learning is carried out. Moreover, the trace data nStudy gathers are theorized to be the kinds needed to develop learning analytics tailored to students’ individual ways of working on learning projects. To the extent nStudy and systems like it become widely used, big data can be acquired about how IPS unfolds. We predict such big data can provide stronger foundations for achieving objectives now attributed to learning projects. Such data also will be a significant resource for advancing learning science (Winne 2017b).