Keywords

1 Introduction

In Computer Science Education, teaching novice programmers to code has been a challenge for many years [8]. Intelligent Tutoring Systems have been developed to assist in teaching novices basic programming skills. These often give feedback and instructions in syntax-free forms [8, 10]. This makes it easier for students who are not well versed in programming to understand the instructions.

We have developed an Intelligent Tutoring System that uses a correctly written program to create all of the instructions and feedback for the student. The students are guided to reproduce the instructor’s program line by line.

The rest of the paper is structured as follows. Section 2 presents the background to the project. It provides definitions of terminology used in the document as well as a review of previous work related to this project. Section 3 outlines the problem which we wish to address with our system. Section 4 describes the methods used to achieve the system’s key functions. Section 5 details the implementation of the system and our results. Section 6 discusses the limitation of our program. Section 7 summarises the results and contributions of the project.

2 Background

2.1 Definitions

Below are definitions of terminology used in this paper.

  • Intelligent Tutoring Systems: Software designed to emulate the impact of human tutors on a student. They engage students in sustained reasoning activity. Interactions with the student are based on a deep understanding of the students behaviour [4].

  • Novice programmers: Students who are being exposed to programming for the first time [6].

  • Novice programs: The sort of programs that a student will encounter when they start learning to program [1, 3].

  • Plan: An abstracted form of a procedure in an algorithm that represents the intended function of a piece of code [7]. Plans can consist of a single operation or a composition of multiple plans [6].

  • Program narrations: Detailed, syntax-free representations of an algorithm. They are presented in natural language and describe the steps taken by the program [1, 3].

  • Program alphabet: The collection of all semantic tokens used in a program [1, 2].

  • Plan similarity: Two distinct plans x and y are similar if these conditions are met:

  1. 1.

    both plans contain exactly the same characters, or

  2. 2.

    the following three conditions are true:

    1. (a)

      the number of tokens in x is the same as in y and

    2. (b)

      every token in plan x is of the same literal category as the token in the same position in plan y, e.g.: both are identifiers, keywords, or numeric values.

    3. (c)

      if there is a numeric literal in the same position in both plans, the floating value of both literals must be the same, i.e.: \(6=6.0\), but \(6.2\ne 6.0\).

Conditions 2a, 2b and 2c must be satisfied for plans to be similar, except when Condition 1 is true [1].

  • Plan equivalence: Two plans x and y are equivalent if:

  1. 1.

    both plans contain the same number of tokens.

  2. 2.

    all tokens, except for identifiers, are exactly the same.

  3. 3.

    for each identifier a in plan x there is the identifier b in the same position of plan y, where b is the equivalent identifier to a between the two program alphabets [1].

2.2 Plan-Oriented Teaching

There have been studies on the effectiveness of how content in an introductory programming course is presented. It has been investigated which areas of knowledge are the most problematic, as well as which areas are foundational to others, and hence the order in which they should be presented. The concepts present in the understanding of programming can be classified into two broad categories: language constructs and plan composition [5]. Language constructs refer to the knowledge of how to use the syntax of the language to produce correct code. Plan composition refers to the ability to construct algorithms that will perform a task. Studies have been conducted to investigate how these two concepts should be taught in relation to each other.

One study found that students struggle with understanding the syntax of the language and that they tend to focus on the implementation rather than problem solving [6]. It was suggested that focussing first on abstract plan composition and then teaching how to implement plans would be effective. A second study found that the order in which concepts are taught does not significantly change the amount a student learns [8]. It was found that teaching syntax before program planning is more efficient in that it places less strain on the students. Yet another study found there to be a strong link between the aptitude in both language constructs and plan composition [5]. This suggested that skills in language constructs and plan composition reinforce each other, thus showing the importance of developing both. While it is not conclusive which area should be taught first, it is clear that a good understanding of both language constructs, plan composition, and the relation between the two is important in teaching novice programmers.

2.3 Intelligent Tutoring Systems

Intelligent Tutoring Systems are software that aid students in the mastery of skills and concepts. These programs provide this by responding to the actions of the student. They identify problems in the student’s work and give feedback to help correct the misunderstandings that caused that error.

Students who can interact one-on-one with human tutors often score significantly higher in tests than students who do not [10]. An Intelligent Tutoring System can provide natural language feedback to a student which should hopefully provide a level of improvement as well. This could prove highly useful in institutions where there are not the resources to provide one-on-one attention to all students.

A tutoring system called ProPL [10] emulated the interactions with a human tutor through the use of natural language in a text-based chat system. The system interacts with the student as they work through a set of problems. The system tries to encourage the student to think through their problems, rather than explain how to correct their mistakes. The results of the study of ProPL suggest that interacting with a tutoring system in this manner resulted in the students engaging with problems at a higher level of abstraction. ProPL requires the structure of Knowledge Construction Dialogues to be manually specified for each problem [10]. These are scripted conversations designed to guide the student’s reasoning to solve the problem.

Another tutoring system (PROUST [7]) is able to detect bugs in Pascal code and provide natural language explanations of errors. PROUST requires a brief describing the goals of the program to be written for each problem [7].

Automatic Feedback Generator [11] is a tutoring system for the Python programming language. It is suggests corrections to a student’s code so that it satisfies the requirements of the exercise. For each task the lecturer provides a template solution and possible corrections in the form of an Error Modelling Language. This requires the lecturer to predict specific errors that the students are likely to make and how those will make their code differ from the template.

An unnamed system uses a Model-Based Reasoning approach [9]. Approaches the problem in a very different manner to the other tutoring systems mentioned here. It does not look for mistakes in the student’s code. It creates C++ code with bugs and tests whether the student can identify the errors. This is done by taking a template piece of code and creating variations of it. The system knows which errors will be created with each variation, so the questions for the student can be generated by the system as well.

Almost all of the systems mentioned here require the instructor to provide some documentation. The content and format varies greatly, but all are for providing guidance to the system about what to teach.

2.4 Program Narration

A system called NOPRON [3] processes C++ code and provides a detailed, syntax-free explanation of the algorithm followed. The system does this by decomposing the program into its most basic operations and identifying the plans that are represented by those operations. These plans are stored in an abstract manner and are later expressed using natural language with pre-defined templates.

2.5 Basis for a New Intelligent Tutoring System

Many students fail their first programming course [8]. While there are systems designed to assist novice programmers [10], there is no system that specifically guides a student in translating tasks to their syntax.

Using natural language descriptions of the algorithm, as seen in NOPRON [3], could provide simple step-by-step sets of instructions to students for writing a novice program. The abstracted structure representing these plans could also function as the template for error checking that a system such as PROUST [7] requires. This reduces the work that an instructor needs to do to set up a programming exercise with the tutoring system.

3 Problem Statement

For novice programmers, two basic skills are language constructs (i.e., writing valid code) and plan composition (i.e., designing algorithms) [5]. These skills are considered key to novices learning to program [5, 6, 8]. Can we develop a system that helps students learn the syntax of C++ as well as reinforce the connection between its plan and implementation?

Natural language instruction shows promise as an effective technique for interacting with the student [10]. Many Intelligent Tutoring Systems require instructors to write the documentation for these applications to produce instructions and feedback to students [7, 10]. However, the technique of narration allows natural language explanations for a program to be produced exclusively from its source code. It is possible for us to create an Intelligent Tutoring System that provides natural language instruction and feedback based entirely on the provided code?

We require that the student write code that achieves the same result as the instructor’s solution i.e. follows the same plan. We, however, do not require that they produce the exact same code. The name of a variable for instance has no bearing on its function. Can we then create a system that evaluates whether the code written by a student is equivalent to that of the instructor?

4 The Tutoring System

In this section we discuss the structure of our Intelligent Tutoring System. We will refer to our program as Code Pathfinder. This name refers to how the student is guided through the writing of programs. Figure 1 provides an outline of the flow of Code Pathfinder.

Fig. 1.
figure 1

Structure of Code Pathfinder

Full size image

4.1 Role of Narrations

The method for producing narrations from source code is the same as was done in NOPRON [3]. Some pre-processing is done first before the actual narration process.

We require that each line of code to perform only one action. Multiple tasks on one line can happen when declaring variables or sending values to the standard output. We can identify cases where the lecturer has done this and split those into multiple lines.

We also strip all leading and trailing white space, as well as comments from the code.

We have written regular expressions that can identify the various tokens of the C++ language. Combinations of these tokens are used to define regular expressions that are used to identify lines of code that match a particular plan. These regular expressions can also be used to extract information from a line of code, such as the variable names used.

4.2 Plan Mirroring

Code Pathfinder gives the student instructions for individual lines of code. The student must give an equivalent line of code before they can proceed. If the line that they write is not correct then feedback is provided to them based on their mistakes. The instructions use the variable names that the students have declared.

4.3 Equivalent Variables

We do not specify to the student what to name the variable that we ask them to declare. We do this to provide some element of choice to the student. Since the system works on line-by-line plan mirroring, variables declared on the same line of the student’s code and the lecturer’s code will be equivalent. We keep track of which variable names are equivalent between the student and instructor’s programs. This is essential for the creation of instructions and feedback, both are discussed below.

4.4 Creating Instructions

We wish to give instructions that use the variable names that the student declared. This is done by determining the code that the student is required to write. The narration of this line of code is used as our instruction to the student. To determine the required line of code we replace all the variable names from the lecturer’s line with the equivalent variables from the student’s code. This is not needed for variable declarations, as explained above. The narration of this line is used as the instruction given to the student. The system uses regular expressions to identify the plan for that line as well as other relevant information, such as variable names, to incorporate in the instruction. For each line plan we have written a function that describes the action of that plan as a task for the student to perform.

4.5 Checking the Line

To check whether the student’s line of code is correct we run several checks. First we check for syntax errors. This is done by checking if a plan is recognised by one of our regular expressions. All valid lines of code with structures within the scope of our system are recognised. We then check if the student’s and instructor’s codes have similar plans. By our definition, this means that the two lines of code are using the same plan, but possibly have different tokens. We substitute all variable name tokens in the instructor’s line of code with the equivalent ones from the student’s program. We then normalise the lines of code to have the same spacing. These lines should match character-to-character for the student’s code to be equivalent to the instructor’s.

4.6 Producing Feedback

Feedback for the student is generated based on which of the above checks succeed. Failure to identify a matching plan indicates a syntax error; while being similar to the instructor’s code, but not equivalent, suggests the use of the wrong variable name. Using regular expressions we can check specific parts of the lines of code to identify the location of the mismatch. For example, we can check the type of variable declared and whether it is the same between the student and instructor. We have functions written for each type of error that will inform the student of what they have done wrongs.

4.7 Available Problems

We have included the following problems for demonstration: next integer, average problem, factorial problem, and Fahrenheit to Celsius conversion. Some of these programming problems have been used to demonstrate functionality of Intelligent Tutoring Systems in the past [3, 5, 7].

5 Implementation and Results

Code Pathfinder has been implemented as a Windows application using regular expression and text-to-speech libraries of the .Net Framework provided by Microsoft Corporation. We have also used the Fast Colored TextBox component [12] to display the instructions and provide the student’s code editing space.

For each problem below we display two listings: one which contains the source code for a problem, and the other which contains the complete set of instructions provided to the student to produce the associated code.

5.1 User Interface

A screenshot of Code Pathfinder can be seen in Fig. 2. The left text box provides instructions and feedback to the student. The right text box is the space where the student writes their code. The code editor provides syntax highlighting and auto-indentation. Feedback on errors is provided in bars below the instruction for the current action. All feedback is available in audio form using text-to-speech. This can be disabled by the user if they want to do so. The user cannot go back to change already written code.

Fig. 2.
figure 2

Screenshot of Code Pathfinder

Full size image

5.2 Next Integer

Code Pathfinder has successfully provided instructions and feedback for the next integer problem. The program should take in an integer and output the next number. Listing 7.1 shows the instructions given by Code Pathfinder. Listing 7.2 gives the complete code written to satisfy the program. Note that the instructions use the variable name chosen by the student.

figure a
figure b

5.3 Average of Two Numbers

In Listings 7.3 and 7.4 we can see similar functionality to that in the next integer problem. What is clear as well is that the instructions for declare a variable get repetitive and give no indication of what the variable does. It is possible that future variations of this system could suggest variable names or even specify what the variable is for. This possibility is discussed in Sect. 7.2.

figure c
figure d

5.4 Factorial Problem

The factorial problem requires the program to calculate the factorial of a given positive integer. The factorial is the product of all integers from one to the number. There are many ways in which this could be programmed. Our example uses a while loop.

figure e
figure f

5.5 Fahrenheit to Celsius Conversion

Converting between units of temperature is a classic task for students taking their first course in programming. The program is given a temperature in degrees Fahrenheit and must return the equivalent value measured in degrees Celsius.

figure g
figure h

5.6 Feedback Examples

If the student is given the instruction declare a variable of type real, then the following lines of code from the student will elicit their corresponding responses as seen in Table 1. Similarly, if the student is given the instruction read value for b, then their answers will be given responses as per Table 2.

Table 1. Examples of feedback for declare a variable of type real
Full size table
Table 2. Examples of feedback for read value for b
Full size table

6 Limitations

In this section we discuss what Code Pathfinder cannot do. In the previous section we demonstrated that Code Pathfinder can handle the basic structures of the C++ language. Code Pathfinder does not handle all structures provided by C++. For example, Code Pathfinder cannot handle multiple operations on one line, nor can it process definitions of new data structures. There are many cases where the order of tokens in a line of code would not change its result, adding two values for example. Code Pathfinder does not allow for this type of line equivalence. Code Pathfinder requires that only one thing is done per line of code. Common practices like declaring multiple variables on one line or initialising a variable on its declaration line cannot be done by the user. The instructor’s code may use these methods, but our preprocessing breaks these into multiple lines. Code Pathfinder provides no explanation as to the purpose of a variable. This means that the student does not know what the variable will be used for, so they cannot give it a descriptive variable name.

7 Conclusion and Future Work

We have demonstrated that our system, Code Pathfinder, is capable of providing instructions and feedback to novice programmers for simple C++ programming problems.

7.1 Key Contributions

  • Feedback generation: We have developed a method where we can use the source code of a problem to generate feedback for a tutoring system.

  • Plan mirroring: We have developed a system that can convert the variable names of one program to those of another program that follows the same plan.

7.2 Future Work

  • Multi-line instruction: We have demonstrated that we can provide line-by-line instructions for novice programs. Can we generate instructions and feedback for code segments spanning multiple lines? Could this handle variations of the order of lines and the number of operations done on a line? Could variations of high-level plans be handled? For example, different sorting algorithms being accepted as a sort plan.

  • Variable name guidance: Can we provide a description of what a variable is used for by analysing the instructor’s code? This could be done by using the lecturer’s variable name as a suggestion, or using comments from the instructor’s code. Alternatively, could Program Comprehension be used to identify the purpose of the variable? Counter variables and flags could possibly be identified by the operations used on them.