Introduction

Most of a student’s day-to-day educational experiences at school occur in the classroom, in which teachers deliver curriculum materials and students formulate knowledge, skills, and attitudes by interacting with teachers and with their own peers. The way teachers behave in the classroom and the effects of these practices on student learning have received significant attention from policy makers and researchers in the field of education, because the quality of instruction is considered one of the most critical factors closely tied to students’ learning experiences. Over the past decade, many researchers have probed into teachers’ behaviors in mathematics classrooms to identify dimensions that describe the characteristics of insightful and effective instruction (e.g., Baumert et al. 2010; Brown et al. 2014; Seidel and Shavelson 2007). The study by Klieme et al. (2009) proposed a theoretical model of the basic dimensions of instructional quality, building on the classical process–product model and the more recent constructivist view of instructional effectiveness. The three dimensions are cognitive activation with challenging content, a supportive and student-oriented classroom climate, and clear and well-structured classroom management. The basic framework of the so-called “triarchic model” proposed by Klieme et al. (2009) was based on the TIMSS 1995 video study, and it has been adopted as a behavior-oriented framework to measure the instructional quality of mathematics teachers in the TALIS and PISA studies (OECD 2013).

The triarchic model suggests that the three dimensions would affect student learning both individually as well as collectively (Klieme et al. 2009). According to Brophy (2000), effective teachers provide challenging tasks, use questions to stimulate students to process and reflect on content, and encourage them to develop an elaborated, content-related knowledge base. However, as addressed in Stefanou et al. (2004) and Turner et al. (1998), simply providing students with challenging tasks is not sufficient to encourage students to engage in insightful learning processes. In other words, cognitive activation strategies can promote student learning given that students experience supportive teacher–student relationships, positive and constructive feedback on errors and misconceptions, and individual learner support (Brophy 2000; Klieme et al. 2009). In addition, clear and well-structured classroom management ensures sufficient learning time by establishing and maintaining structure and order in the classroom (Baumert et al. 2010).

Along with the theoretical conceptualization proposed by the researchers mentioned above, empirical evidence exploring the relationships among one or more elements of instructional quality and student outcomes has accumulated in the literature. For instance, Baumert et al. (2010) found that the cognitive level of math tasks provided to students and the quality of classroom management were significant predictors of math performance of 10th graders and emphasized that clear and well-structured classroom management had been a robust predictor of the quality of instruction. Based on content analyses and meta-analytic methods, Wang et al. (1993) observed that a variety of instructional variables such as instructional and classroom management techniques and student–teacher social and academic interactions exerted significant influence on school learning.

However, teaching effectiveness studies over the past decades were mostly centered on cognitive outcomes (Seidel and Shavelson 2007), despite the fact that student outcomes could encompass achievement, learning process, motivation, and attitudes, from the perspective of the process–product model. The PISA 2012 report (OECD 2013) also addressed that the relationships between learning environment and affective outcomes, such as students’ interest and enjoyment in mathematics, were not fully covered in the PISA 2003 studies. Although fewer studies focus on affective or motivational outcomes compared to those dealing with cognitive outcomes, recent meta-analysis results by Seidel and Shavelson (2007) imply that instructional quality may be more closely related to affective outcomes than cognitive outcomes. Therefore, a closer look at the instructional quality and a global view of the relationships between all facets of instructional quality and students’ affective characteristics as well as cognitive outcomes will provide empirical evidence of the feasibility of the framework of instructional quality employed in the present study.

Hence, the PISA 2012 mathematics assessment, an international survey with abundant context variables encompassing cognitive and non-cognitive student outcomes, was used for this study. The survey provides concrete measures of student-perceived instructional quality and teacher behaviors in math classrooms based on the triarchic model as well as a variety of students’ math-related affective characteristics. Among the top performing countries on the PISA 2012 math assessment, Singapore and Korea were selected for this study. The two countries have received attention from many countries seeking to identify the reasons for the high performance. Interestingly, however, international comparison studies such as PISA and TIMSS have shown that interest in math and math-related self-concepts are generally high for Singaporean students, whereas they are relatively low for Korean students, despite the overall high performance of both countries. One approach to investigating this phenomenon could be taking a closer look at teachers’ classroom behaviors, to find differential effects, if any, of instructional quality on students’ affective and cognitive outcomes.

Therefore, the main purpose of the present study is to closely examine instructional quality of math teachers based on the PISA 2012 math results of Korea and Singapore, and to explore relationships between instructional quality and students’ affective and cognitive outcomes. Specific research questions are as follows: (1) What is the overall pattern of instructional quality of math teachers of Korea and Singapore? (2) How does the level of instructional quality differ between Korea and Singapore and across the level of student math performance? (3) How many latent subpopulations may exist in the profiles of instructional quality and how would they differ, if any, in terms of cognitive and affective student outcomes? and (4) Does instructional quality have direct and mediating effects on students’ affective and cognitive outcomes?

In order to answer these questions, overall levels of instructional quality were explored over the six different yet closely related dimensions comprising instructional quality. Possible existence of subpopulations was also explored to investigate whether interpretable patterns could be found across subpopulations. Diverse measures of student characteristics ranging from student background information to math-related cognitive and non-cognitive outcomes were compared across latent subpopulations. Finally, based on the theoretical framework of the triarchic model defining the basic dimensions of instructional quality and their effects on student learning, a structural equation model was specified and analyzed, reflecting relationships between the major dimensions of instructional quality and student outcomes. Student’s math performance on PISA was used as the final outcome measure, with non-cognitive math-related characteristics being specified as mediators.

Literature review

Many researchers have explored factors describing insightful and effective instruction over the past decades (e.g., Baumert et al. 2010; Bolhuis 2003; Scheerens and Bosker 1997). Since the research is diverse and complex in conceptualizing the quality of instruction reflected by teachers’ classroom behaviors (Seidel and Shavelson 2007) and individuals may benefit from different instructional practices (Brown et al. 2014), identifying universal factors explaining effective learning is not an easy task. Despite the variety of terminology and categorization defining effective instruction, however, major characteristics or behaviors of teachers that would serve as crucial factors of high-quality math instruction can boil down to a few common dimensions (Baumert et al. 2010; Klieme et al. 2009).

Building on the process–product model and the constructivist paradigm, Klieme et al. (2009) proposed a triarchic model defining the basic dimensions of instructional quality and their effects on student learning. The model is depicted in Fig. 1. Among the three major components comprising the quality of instruction, cognitive activation is deemed essential in encouraging students to engage in constructive and reflective higher level thinking and thus develop an elaborated knowledge base. According to Hiebert and Grouws (2007), mathematics instruction that promotes conceptual understanding attends explicitly to concepts and connections among mathematical facts, procedures, ideas, and representations and prompts high levels of cognitive functioning. This process is facilitated by a supportive classroom climate, which includes student-oriented instruction, constructive feedback, and care for individual learners (Brophy 2000; Stefanou et al. 2004; Turner et al. 1998). In addition to these two dimensions, clear and well-structured classroom management is suggested as a precondition for time on task engagement of students, which in turn, improves students’ understanding of knowledge (Baumert et al. 2010; Klieme et al. 2009). The triarchic model has been implemented in OECD TALIS study as behavior-oriented items, some of which have been employed in PISA 2012. In TALIS, teachers’ instructional behaviors were asked in three dimensions: structuring practices, student-oriented practices, and enhanced activities (OECD 2009, 2013). The TALIS teaching practices scale was adapted for use in PISA, tailored to measure students’ perceptions about their math teachers’ instructional behaviors, including cognitive activation, student-oriented or teacher-directed instruction, use of feedback, and teachers’ learning support (OECD 2013). TALIS survey results reported that a great majority of teachers indicated that they held views consistent with active teaching practices aimed at higher order thinking skills and pedagogy that supports these skills in a more student-centered approach, although teachers were less likely to report that they engaged in practices consistent with these views (Burns and Darling-Hammond 2014).

Fig. 1
figure 1

Theoretical model of the dimensions of instructional quality and their effects on learning (reproduced from Klieme et al. 2009)

Full size image

Empirical research has explored the relationships between instructional quality and student learning. For example, based on content analyses and meta-analytic methods, Wang et al. (1993) showed that efficient classroom management enables teachers to focus more on instruction and has positive effects on enhanced student achievement. In addition, academic and social interactions between teachers and students allowed teachers to tailor their instruction to meet the specific needs of students, so that students can receive instruction that matches their prior knowledge, addresses their misconceptions, and organizes knowledge in meaningful ways. A classroom climate created by supportive student–teacher and student–student relations, achievement orientation, and a disciplinary atmosphere also has been perceived as a vital factor in predicting effective learning (OECD 2013). Use of formative assessment and feedback is another important factor that helps students check their learning status and brings about qualitative changes in their performance (Ellery 2008; Kingston and Brooke 2011). However, the effects of each component of instructional quality on student performance have not been always significant. Lipowsky et al. (2009) and Baumert et al. (2010) reported that cognitive activation and classroom management were positively associated with student achievement, while supportive climate in terms of positive teacher–student relationships and constructive teacher feedback had no direct effects. Research on different types of feedback suggests that process-oriented, descriptive, and specific feedback has more positive effects (Davis and Carson 2005; Fluckiger et al. 2010; Harks et al. 2014).

Some authors have focused on non-cognitive student outcomes in addition to the cognitive outcomes. For example, Yair (2000) observed that authentic learning experiences and the sense of voluntary participation in class were significant predictors of intrinsic motivation and the sense of accomplishment, whereas challenging instruction did not have a significant effect on intrinsic motivation and even had a negative effect on the sense of accomplishment. Klieme et al. (2009) and Seidel et al. (2005) emphasized that more cognitively activating instruction and facilitating a more supportive classroom climate have the power to encourage students and to transform their interests into advanced math achievement. Vieluf et al. (2009) also showed that teacher support was positively linked to students’ interest in mathematics. Although there are fewer studies focusing on affective or motivational outcomes compared to those dealing with cognitive outcomes, and some studies report findings contrary to our expectations, recent meta-analysis by Seidel and Shavelson (2007) implies that instructional quality may be more closely related to affective outcomes than cognitive outcomes. They reported that the effects of teaching on learning yielded small effect sizes in general, but disentangling the effects by the type of outcomes revealed that the low effects were mainly due to cognitive outcomes.

Recently, several empirical researches based on large-scale survey such as TIMSS or PISA have extensively investigated the relationships between teacher characteristics and cognitive/non-cognitive outcomes. For example, Kim and Ham (2014) investigated effects of school-level variables on Korean students’ non-cognitive outcomes in math using the PISA 2012 data. They found that, as process variables in the classical process–product model, teacher characteristics such as student-oriented teacher behaviors and disciplinary climate of math classroom had positive effects on students’ self-efficacy in math, which in turn served as a major non-cognitive factor that drives students to higher math performance. Based on the theoretical framework of classroom assessment by Brookhart (1997), Sohn et al. (2014) investigated and compared relationships between teachers’ classroom assessment practices and math achievement, confidence, and enjoyment using TIMSS 2011 data for Korea, Singapore, and Finland. The authors highlighted differential effects of teachers’ feedback types on student math performance. In Korea, higher math performance was associated with practices of allowing students to correct their homework on their own or spending more time on homework discussion, while negative relationships were found between these variables for Singapore students. Ku et al. (2015) explored and compared the relationships between math-related educational context variables and mathematics achievement among the four high-performing countries on PISA 2012 math assessment. An observation that contrasted South Korea most from other countries was the positive effects of teacher’s support in math lessons, which were not significant in other countries. However, as opposed to anticipations formed by literature review, student-oriented and teacher-directed instruction did not show any significant effects on math performance for Korea. Student-oriented instruction had even negative effects on math performance for Singapore.

Methodology

Data

PISA 2012 math assessment and survey questionnaires were used to explore the relationships between instructional quality and student outcomes in this study. In PISA, schools in each country are randomly selected and the assessment is given to students who are between age 15 years 3 months and age 16 years 2 months at the time of the test, rather than to students in a specific grade level (OECD 2013). The analyses drew on the samples of 5546 Singaporean students of 172 schools and 5033 Korean students of 156 schools who have taken the PISA 2012 mathematics assessment. Some analyses were conducted using subsets of the samples, because PISA 2012 employed a rotated test design that resulted in each item being administered to approximately two-thirds of the entire sample. The final student weight (W_FSTUWT) was employed to obtain generalizable estimates for the target population. Demographic information on the participating students in Korea and Singapore as well as general information about math teachers of the participating schools is summarized in Table 1. It should be noted that PISA does not collect data at the teacher level, and thus demographics about teachers are not available.

Table 1 Demographic information of participating students and school information related to math instruction
Full size table

Variables

Partly based on the triarchic model of instructional quality employed in the present study, the survey provides concrete measures of student-perceived instructional quality and teacher behaviors in math classrooms as well as a variety of students’ math-related affective and motivational characteristics. In PISA, standardized indices are usually constructed from students’ responses for each item to represent the overall level of each variable of interest. In this study, standardized PISA indices were mainly used in the latent profile analyses as well as the analyses exploring the overall patterns of instructional quality, while item-level information was used in the structural equation model analyses. The six PISA indices reflecting math teachers’ instructional quality relevant to the three dimensions of the triarchic model include teachers’ use of cognitive activation strategies (COGACT), teacher-directed and student-oriented instructional behavior (TCHBEHTD and TCHBEHSO), teachers’ use of formative assessment (TCHBEHFA), classroom management (CLSMAN), and learning support (TEACHSUP). The student-perceived teacher characteristics are used for the analysis without aggregating into the school level, mainly because teachers’ classroom behaviors are believed to be more accurately and credibly evaluated by students as opposed to teachers’ self-reports, and also because aggregation bias has been reported as a major source in the literature (Hanushek 1997).

Scores on the PISA math assessment were used to represent the cognitive outcomes of students’ learning. Math-related affective characteristics collected on the student questionnaire were used as non-cognitive outcomes. They include interest in mathematics (INTMAT), instrumental motivation for studying mathematics (INSTMOT), self-efficacy (MATHEFF), and self-concept in math (SCMAT). Variables reflecting schools’ learning environment and climate, such as disciplinary climate (DISCLIMA), teacher morale (TCMORALE), and student–teacher relations (STUDREL), were examined in order to find any patterns explaining different levels of instructional quality. Students’ experience of pure math (EXPUREM) and applied math (EXAPPLM) was also examined. Finally, an index of socio-economic status (ESCS) was used as a control variable in exploring the effects of the variables measuring instructional quality on student outcomes.

For the SEM analysis, item-level variables were used as observed indicators for each latent variable. In order to select items that would make reliable measurement model, correlation coefficients were examined and a series of factor analyses was run to see whether there was any evidence of collinearity or items with low factor loadings. As a result, one item measuring the student-oriented teacher behavior (ST79Q04) and two items measuring class management (ST83Q03 and ST83Q04) were excluded from the SEM analyses due to low factor loading. Nine items measuring cognitive activation were parceled to make three indicators to obtain more stable estimates. In PISA, math performance is measured in four subdomains: space and shape, change and relationship, uncertainty and data, and quantity. Composite score as well as subscale scores is reported for each student in the form of five plausible values. First plausible value was used for the analyses in this study (PV1MATH, PV1MACC, PV1MACQ, PV1MACS, PV1MACU). Descriptive statistics of the PISA indices used in the present study are summarized in Table 2. Due to limited space, items measuring each variable could not be reported in the table. Interested readers should refer to OECD (2013) in order to get information about items for each PISA index.

Table 2 Descriptive statistics of PISA indices used in the present study
Full size table

Statistical analyses

In order to examine the instructional quality of math teachers in both countries closely, two statistical analyses were conducted: a latent profile analysis and structural equation modeling. All analyses were conducted in Mplus 7.0 (Muthen and Muthen 2012) under missing data theory using full information maximum likelihood (FIML) estimation.

First, the latent profile analysis (LPA) was applied to explore the overall profiles of each component of instructional quality and the profiles of the latent subpopulations that may exist. The six PISA indices representing the student-perceived instructional quality of math teachers were used for the LPA. An exploratory approach was taken to identify the number of latent classes. Several models were fit to the data, specifying one through five latent classes for each country, respectively. For each model, replication of the best log-likelihood was verified to avoid local maxima, by using a sufficient number of different start values and initial stage iterations (1000 in the first step and 100 in the second step of the optimization; 100 initial stage iterations). To determine the most optimal number of latent classes, the AIC, BIC, and adjusted BIC (aBIC) were reviewed. A bootstrap likelihood ratio test and Lo–Mendell–Rubin test were used to compare a model with k latent classes against a model with k-1 classes. Interpretability based on the proportion of members belonging to each latent class was also considered, coupled with the model-fit indices. After the number of latent classes was determined from the LPA, diverse measures of student characteristics ranging from student background information to math-related cognitive and non-cognitive outcomes were compared across the latent classes.

Second, a structural equation model (SEM) was specified, reflecting the relationships between the dimensions of instructional quality and student outcomes as depicted in the triarchic model. Four out of the six variables used in the LPA were used in the SEM analysis. They include cognitive activation, student-oriented instructional behavior, classroom management, and learning support. Teacher-directed instruction was excluded from the SEM model because preliminary analyses, which will be provided in the next section, showed that it did not change as the level of math performance increased. Use of formative assessment was also excluded because it was highly correlated with the student-oriented instruction, possibly resulting in collinearity problems. Student-oriented instruction showed negative relationships with math performance for both countries in our preliminary analyses, which was consistent with Ku et al. (2015). However, the authors decided to include the variable in the research model, not only because literature implied possible differential effects between cognitive activation and mere student–teacher interaction without facilitating students’ cognitive activation (Klieme et al. 2009) but also because we were interested in the effects on non-cognitive outcomes as well as cognitive outcomes. Among the mediating variables introduced in the triarchic model, depth of processing, reflection, and time on task are not collected in the PISA 2012. Self-concept and interest in math were selected as measures of emotions/affects and intrinsic motivation, respectively. Scores on the PISA math assessment were used as the final outcome measures, with interest in mathematics and mathematics self-concept being specified as mediators. All variables except the index of socioeconomic status were treated as latent variables, each of which was measured by associated items as indicators. Figure 2 represents the path diagram of the SEM model proposed in this study. The full model depiction including observed indicators for each latent variable and error terms is provided in the “Appendix”.

Fig. 2
figure 2

Structural equation model of instructional quality and student outcomes

Full size image

Results

Instructional quality of math teachers in Korea and Singapore

In order to see how student-perceived instructional quality of math teachers differs along the students’ math performance, six PISA indices representing each component of instructional quality were selected and compared across Singapore and Korea using PISA proficiency scale. The PISA proficiency scale is a “described proficiency scale” developed to provide substantive meaning to the PISA scale. It is a criterion-referenced scale, which is composed of six levels (Level 1–Level 6) with performance descriptors describing expectations that students classified into each level can typically do in math. There is a level below Level 1 for scores that fall below the lowest score point of Level 1, but no descriptors are attached to the level. The analysis results are shown in Fig. 3, in which L1–L6 represents Level 1–Level 6 of the PISA proficiency scale, and L0 represents performance below Level 1. Overall, Singapore was higher than Korea in every index of instructional quality at all levels. Particularly, the differences in cognitive activation and use of formative assessment were larger than those of the other four indices. Class management and learning support tended to increase as the level of math achievement increased for both countries. Student-oriented instruction and the use of formative assessment had a generally decreasing pattern, while the use of formative assessment slightly went up for the highest two levels for Korean teachers. It was an interesting observation that students at the lower levels of math achievement tended to perceive that their math teachers employed a more student-oriented approach and used more strategies of formative assessment.

Fig. 3
figure 3

Instructional quality of math teachers at each level of math performance

Full size image

Latent profiles of instructional quality of math teachers and comparisons of characteristics across latent classes

Latent profile analyses were conducted for Korea and Singapore using the six PISA variables representing student-perceived instructional quality of math teachers in order to determine the number of latent classes and the membership of each individual in one of the latent classes. To find a model with the optimal number of latent classes that closely fits the data and provides us with meaningful interpretations, the LPA was repeatedly run by increasing the number of classes from one to five. Table 3 shows the fit statistics of each LPA model with a varying number of classes. The AIC, BIC, and aBIC decreased as the number of latent classes increased, as anticipated, due to increased model complexity, which affects the Chi-square-based statistics. However, the bootstrap LRT and Lo-Mendel-Rubin LRT results suggest that the model with four latent classes might be the best-fitting yet parsimonious model for both Korea and Singapore. Examining graphical representations of each estimated profiles also suggested that the four-class model is reasonable in terms of interpretability of latent classes.

Table 3 Fit statistics of latent profile analysis models
Full size table

The estimated means of the LPA with four latent classes are depicted in Fig. 4. For both countries, the profile of Class A shows very high level of instructional quality for all six components with a dip on the class management. Part of the reasons for showing lower level of class management within Class A might be due to relatively lower standard deviation of the class management variable for Korean and Singaporean samples, although it deserves further explorations to figure out whether there are specific reasons for this phenomenon. The level of class management is still higher than those in other classes. Therefore, we can interpret the Class A as teachers practicing highest level of instructional quality. The profile of Class B in Singaporean sample shows fairly high level of instructional quality with slightly lower level of student-oriented instructional behaviors. The profile of Class B in Korean sample and that of Class C in Singaporean sample is almost flat around the zero line, which implies that teachers in this class use intermediate level of instructional quality overall. The profile of Class C for Korean sample and that of Class D for Singaporean sample both show generally lower level of instructional quality. Teachers in this class tend to use less formative assessment and cognitive activation strategies compared to other components of instructional quality, and this pattern was more obvious in Korean sample. The Class D in Korean sample shows very low level of instructional quality.

Fig. 4
figure 4

Estimated means of latent profiles of instructional quality

Full size image

Overall, Korean teachers showed lower profiles than Singaporean teachers. All four profiles were distributed in the range from −1 to 2 for Singapore, while only three profiles were in the same range for Korea. About 5.63 % of Korean samples were classified into Class D, which had the lowest profile from approximately −3 to −1. Students classified into this profile tended to perceive that their teachers use a generally lower level of instructional strategies. Relatively speaking, these teachers used more student-oriented classroom behaviors, effectively managed classrooms, and supported student learning, but did not use cognitive activation strategies as much. The majority of Korean samples were classified into the third highest profile, the Class C (50.98 %), which was approximately around the standardized scores of −1. The majority of Singaporean samples were also classified into the third highest profile, the Class C (50.48 %), but the profile was at a higher level of the standardized score, which was approximately zero. Korean students in Class C perceived that their math teachers use less formative assessment and cognitive activation strategies compared to the use of other instructional strategies. The profile at the top looked similar for both countries, but the percentage was higher for Singapore (7.25 %) than Korea (2.38 %). Teachers in this latent class showed a higher level of classroom behaviors and cognitive activation around the standardized score of 2 and showed markedly low standardized scores for the class management around −1. Korean students who belong to the top latent class perceived that their teachers used more student-oriented classroom behaviors and less cognitive activation strategies than those Singaporean students perceived. Learning support was not as high as classroom behaviors or cognitive activation strategies for both countries.

Having decided on the four-class model for both countries, characteristics that members of each latent class would possess were examined and compared with respect to students’ cognitive and non-cognitive characteristics as well as the general characteristics of the schools to which the members belong. Descriptive statistics and the results of the analyses of variance (ANOVA) across the latent classes with respect to the student-level and school-level variables are summarized in Table 4. The Brown-Forsyth test was used to adjust for unequal variances, which might have been caused by noticeably different sample sizes across latent classes. Likewise, a Games-Howell adjustment was made for post hoc tests.

Table 4 Analysis of variance results comparing means across latent profiles
Full size table

An interesting pattern was observed with respect to math performance. As shown in Table 4, math performance was higher for the second and the third profiles compared to the top and the lowest profiles. This indicates that low-performing students tended to perceive the instructional quality of their math teachers as either very high or very low. The pattern was consistent in both countries. Students’ SES was higher for the top and second profiles than the other two profiles in the case of Korea, while the differences were nonsignificant for Singapore. Patterns with respect to students’ affective outcomes were more consistent across variables. In general, the average level of math-related affective characteristics of students increased as the profile of student-perceived instructional quality went up, for both countries. That is, students in the higher profiles of instructional quality tended to have more interest in math and be more instrumentally motivated than those in the lower profiles. The students in the higher profiles showed a higher level of self-efficacy and more positive self-concepts in math than those in the lower profiles. An exception was found for mathematics anxiety. In the case of Korea, mathematics anxiety increased as the profile of instructional quality moved from the top to the second and the third profiles, with an exception that the students in the lowest profile showed the lowest level of math anxiety.

Variables related to school environment and climate showed mixed results. In general, the student–teacher ratio and student–math–teacher ratio were higher for the schools in the lower profiles in Korea, while significant differences were not found for Singapore. In the Korean samples, the student–math–teacher ratio was very large for the lowest profile. Students’ disciplinary climate and student–teacher relations were more favorable for the higher profiles than lower profiles for both countries, although disciplinary climate was highest for the second profile. Teacher morale did not make any difference among the profiles of instructional quality for both countries. Students in the higher profiles reported that they had more frequent experiences with applied and pure mathematics than those in the lower profiles. The same patterns were observed for both Korea and Singapore.

Results of structural equation modeling analysis

To estimate the path coefficients of the relationships between instructional quality and student outcomes proposed in Fig. 2, an SEM analysis was conducted. Prior to analyzing the research model, stability of measurement model was checked using an exploratory and a confirmatory factor analysis. As described in the method section, items with low factor loadings were excluded. After removing such items, the model-data fit indices showed reasonably good fit for both Korea and Singapore, and all factor loadings were larger than 0.6, indicating stable measurement structure at the item level as well as at the entire model level. Since the measurement model was determined to be stable, we continued our analysis for the research model. The CFI and TLI indices of the research model were higher than 0.95, and the RMSEA indices were lower than 0.05, suggesting a reasonable fit for both countries. The model-data fit indices for the CFA and the research model are reported in Table 5. The unstandardized and standardized estimates of path coefficients and associations of exogenous latent variables are reported in Table 6. Standardized estimates of path coefficients are also shown in Figs. 5 and 6 for Korea and Singapore, respectively.

Table 5 Model-data fit of the SEM model
Full size table
Table 6 Estimates of path coefficients of the SEM model
Full size table
Fig. 5
figure 5

Standardized estimates of path coefficients of the SEM model (Korea)

Full size image
Fig. 6
figure 6

Standardized estimates of path coefficients of the SEM model (Singapore)

Full size image

Somewhat inconsistent patterns were observed between Korea and Singapore. First, cognitive activation had positive effects on interest in math, self-concept, as well as math performance for the Korean sample. However, for the Singaporean sample, it did not have significant effects on the interest in math and self-concept, although a direct positive effect on the math performance was observed. Second, student-oriented teacher behavior also had differential effects between the two countries. In the Korean sample, the student-oriented teacher behavior had negative effects on students’ interest in math as well as the math performance and did not have a significant effect on the mathematics self-concept. In contrast, it had significant positive effects on both affective characteristics, but had a negative direct effect on the math performance, for the Singaporean sample. Student-oriented teacher behavior had a negative indirect effect with math interest as a mediator for Korean students, while a positive indirect effect with math self-concept as a mediator for Singaporean students. It did not show any significant indirect effects on math performance with math self-concept as a mediator, for both countries. Third, class management had a positive effect on interest in math and a slight negative effect on math performance for the Singaporean students. It had a positive indirect effect on math performance with the interest in math as a mediator. However, none of those effects associated with class management was significant for the Korean students. Fourth, teachers’ learning support had a positive effect on interest in math as well as on the math performance for both countries. It also had a positive effect on the mathematics self-concept for the Singaporean sample. For Singapore students, learning support had a positive indirect effect on math performance with both interest in math and math self-concept as mediators, while it only had a positive indirect effect through interest in math for Korean students. Fifth, interest in math and mathematics self-concept had positive effects on math performance for both countries, although the effect of interest in math had a much larger effect for Korea than Singapore. Lastly, students’ SES had a positive effect on math performance for both countries. All correlations between exogenous latent variables were significant.

Discussion and conclusion

For many decades, policy makers and educational practitioners have applied extensive efforts to determine ways to improve education. Among the many facets that could affect the quality of education, a large body of research has focused on the characteristics of teachers who have proximal relationships with students in school. In this study, the instructional quality of math teachers was explored for Korea and Singapore based on the PISA 2012 survey, and the relationships between instructional quality and student outcomes were investigated using LPA and SEM analyses. The major findings and implications from the present study are as follows.

First, it was an interesting observation that math teachers tended to employ a more student-oriented approach and use more strategies of formative assessment for students at lower levels of math performance than for those at higher levels. In contrast, class management and learning support were more positively endorsed by high-performing students. This observation may be due to the tendency that low-performing students need more individualized and low-paced instruction suitable for their state of learning and may be more benefited by small group activities to generate solutions jointly in order to keep their interest. Also, teachers may use formative assessment strategies to students with lower performance more often in order to provide information on students’ strengths and weaknesses and to monitor progress. A well-managed classroom and supportive learning climate associated with a higher level of math performance may be related to the disciplinary climate of classrooms. Use of more student-oriented instruction and less strategies of cognitive activation associated with lower performance in math, especially for Korean teachers, corroborates the notion that student-oriented instruction itself without cognitive challenges that promote deep understanding may not be effective (Klieme et al. 2009). This implies that student-oriented instruction may not be effective if it fails to engage students in learning but only makes them work on math-related activities in a superficial way. Thus, teachers need to put more efforts to help low-performing students engage in cognitively challenging tasks.

Second, the level of instructional quality was generally higher for Singapore than Korea in every index at all math achievement levels. This pattern might be a reason for keeping interest and self-concept in math higher for Singaporean students than other high-performing countries including Korea, as observed in international comparisons such as PISA and TIMSS. Cognitive activation and use of formative assessment were particularly more different between the two countries. A closer look at the differences at an item level revealed that responses such as “The teacher tells us what is expected of us when we get a test, quiz, or assignment” and “The teacher presents problems for which there is no immediately obvious method of solution” had the largest differences between Korea and Singapore for the use of formative assessment and cognitive activation, respectively. We have seen from the literature that students tend to find descriptive and contextualized feedback more helpful (e.g., Davis and Carson 2005; Fluckiger et al. 2010) and that providing students with highly challenging, novel, diverse, and authentic activities that can stimulate divergent thinking may facilitate students to be more engaged in math (Klieme et al. 2009; Yair 2000). Therefore, Korean educators should look for reasons for the relatively low level of instructional quality in general and find ways to encourage their teachers to be more specific and clear about their expectations when doing formative assessment activities and to allow their students to be more exposed to thought-provoking problems or tasks.

Third, a series of comparisons made for the latent classes from the LPA showed that most of the affective characteristics and math teachers’ instructional focus considered in this study were positively associated with higher profiles of instructional quality. Interestingly, however, lower performance in math was associated with either very high or very low profiles of instructional quality, with the majority of students perceiving the level of instructional quality to be intermediate. This finding is somewhat promising because adopting more instructionally desirable strategies in math classrooms may have positive effects on students’ affective characteristics, although it may not play a crucial role in making differences in effective learning from the perspective of cognitive outcomes. This is consistent with the findings from previous research that instructional quality is more closely related to affective and motivational outcomes than cognitive outcomes (Seidel and Shavelson 2007). As educators, we have experienced that students who struggle with math often possess negative self-concepts and anxiety in math, which hinder their progress in learning. Therefore, motivating students and fostering positive self-concepts and efficacy in math is as important as enhancing the math performance of students. In this regard, encouraging teachers to employ more desirable instructional strategies seems to be a crucial element for successful education.

Fourth, building on the provisional association of instructional quality and affective characteristics found from the LPA, an investigation of the individual effects of each component comprising instructional quality through the SEM analysis presented differential effects on student outcomes across countries as well as individual components. Discrepant findings were observed for cognitive activation and student-oriented instruction between the two countries. Cognitive activation had positive effects on math performance and two affective mediators for Korean students, while it only had a positive effect on math performance for Singaporean students. In contrast, student-oriented instruction had negative effects on interest in math as well as math performance in Korea, but had a positive effect on interest in math in Singapore. Klieme (2013) addressed the issue of negative relationship between student-oriented instruction and supportive teaching behaviors and student learning outcomes found in certain countries, suspecting that this phenomenon might be related to the tendency that teachers tended to be more supportive and less demanding to low-performing students. The positive effect of cognitive activation and the negative effect of student-oriented teacher behavior on math performance observed in both countries provides empirical evidence for the notion that cognitive activation should not be confused with student-oriented teacher behavior because student-oriented instruction alone may not contribute to effective learning unless deep learning occurs as a form of cognitive activating strategies (Klieme et al. 2009). Positive effects of cognitive activation on interest and self-concept in math observed in Korea are encouraging in that providing challenging tasks enhances students’ motivation and self-concept as well as math performance, although the nonsignificance of cognitive activation on affective characteristics in Singapore and the negative effect of student-oriented instruction on interest in math in Korea are discouraging. The mixed results deserve further research.

Finally, it was observed for both countries that teachers’ learning support was a positive predictor of interest in math and math performance. This is inconsistent with previous empirical findings that supportive learning climate did not play a significant role in predicting math performance (Baumert et al. 2010; Lipowsky et al. 2009). However, the finding of the present study suggests that, as Anderman et al. (2001) addressed, teachers who create a learning environment where all students can feel successful and feel a sense of improvement emphasize effort, improvement, and challenges, and these are essential elements of supportive learning that may lead to effective learning.

There are limitations to this study. First, although the basic framework of our research model was the triarchic model, some mediation variables such as depth of processing and time on task represented as part of the model were not included in our research model, because those variables were not available in PISA survey data. Availability of those variables might have provided more extensive empirical evidence for the effects of instructional quality based on an international survey data. Second, as mentioned earlier, the student-perceived teacher characteristics were used to represent teachers’ instructional quality in this study, mainly because there is no teacher-level questionnaire in PISA survey. Besides, teachers’ classroom behaviors were believed to be more accurately and credibly evaluated by students as opposed to teachers’ self-reports. However, drawing implications on teachers’ instructional behaviors when the unit of analysis is student may also have limitations in that they are based on students’ perception about teachers’ behaviors, not the teachers’ behaviors themselves. Limitations due to employing the secondary data such as PISA and using student-perceived teacher behaviors may call for future study investigating the relationship between instructional quality and students’ cognitive and non-cognitive outcomes based on first-hand data collected directly from teachers.

Some educators would lament that today’s students are not motivated enough to engage in math, and they often feel depressed in math classrooms. However, as Yair (2000) indicates, students do not have a general tendency to be emotionally depressed in school. Rather, they perceive their experiences to be highly influenced by the specific characteristics of instruction. Instruction that lacks authenticity and performance-oriented instruction that makes normative comparisons with others will produce low motivation, a sense of failure, and a detrimental mood for learning (Yair 2000). The goal of teaching is to support student learning. One essential competence that every teacher should possess is the ability to analyze his or her own teaching in terms of its effects on student learning (Hiebert and Grouws 2007). Setting clear learning goals for students, assessing whether the goals are being achieved during the lesson, specifying hypotheses for why the lesson did or did not work well, and using the hypotheses to revise the lesson are crucial elements for improving instructional quality. Since the way in which a teacher and student interact about the subject in a classroom can be more powerful than any other factor, investing efforts toward the improvement of teaching will yield greater returns.