Introduction

The rapid advancement of technology over the past decades has enabled innovations to considerably influence daily life and drive key shifts in the manner by which education is delivered (Jonassen 2013; Tan et al. 2017). Research on education has extensively focused on integrating technologies in teaching and learning and developing teachers’ professional knowledge (Lawless and Pellegrino 2007; Mishra and Koehler 2006). Mishra and Koehler (2006) proposed the technological pedagogical content knowledge (TPACK) framework, which comprises theoretical constructs that can be used to help teachers carry out self-assessments and comprehend the development of their professional knowledge with technology-based teaching practices. Meanwhile, Angeli and Valanides (2009) argued that TPACK is a holistic, integrated, and transformative form of teachers’ knowledge to be assessed by the five criteria.

Previous studies suggest that a well-developed TPACK instrument can facilitate teachers’ understanding of how technologies can be used to enhance student learning in the 21st century (Koehler and Mishra 2008; Niess 2008; Shih and Chuang 2013; Tseng 2014). Most TPACK-related instruments were developed from teacher-centered perspectives (e.g., Hofer and Grandgenett 2012; Schmidt et al. 2009; Yurdakul et al. 2012; Zelkowski et al. 2013); few studies have explored students’ perceptions regarding the knowledge of teachers who handle technology-supported classes (e.g., Shih and Chuang 2013; Tseng 2014). Studies have also articulated that what teachers believe they know, what they implement in their teaching practices, or how they evaluate their knowledge via a self-assessment instrument may be inconsistent with actual teaching principles (Lawless and Pellegrino 2007; Tseng 2014). These challenges motivated us to develop and validate an instrument that measures teachers’ professional knowledge as perceived by students and not merely as determined by self-assessment (Tseng, 2014; Wearn 2008).

Computer-assisted language learning (CALL) refers to a variety of technology applications for language learning to cultivate teaching and language learners’ skills (Aryadoust et al. 2016). CALL is illuminated as the development, discovery, selection, use, and evaluation of language learning activities that draw on technologies (Chapelle 2010). Rahmany et al. (2014) found that technological knowledge affects the development of teachers’ knowledge under CALL activities during a course. Liu and Kleinsasser (2015) reported that language teachers’ TPK, TCK, and TPACK ratings, as well as their computer self-efficacy, increase after participation in CALL training courses. The vigorousness with which technology integration in language education has been pursued reinforces the critical role of technology in the language learning process. The majority of studies related to the impact of CALL teacher education programs have focused on how these courses influenced teachers’ belief, confidence, perceptions, and attitudes towards CALL and confidence in integrating CALL into their teaching (e.g., Kamhi-Stein 2000; Lam 2000; Peters 2006; Van Olphen 2007; Yildirim 2000). However, little has been done to examine how students perceived their teachers’ knowledge in technology-supported language classes.

With these considerations in mind, we sought an alternative means of developing and validating an instrument for assessing high school students’ perceptions of the knowledge possessed by English teachers who work in technology-supported environments. This rigorously developed and scientifically based instrument is expected to enable evaluators to thoroughly examine the perceptions of high school students and identify the factors that they regard as critical to teachers’ knowledge. This work contributes to the literature by expanding our understanding of student perceptions of teachers’ knowledge and could further triangulate those findings with teachers’ self-evaluations of their TPACK. We view the development and validation of an assessment instrument as essential given the dominance and prevalence of technology in today’s high school campuses. The categories of high school students’ perceptions of teachers’ knowledge instrument were derived from the SPFK (students’ perceptions of faculty knowledge) scale proposed by Shih and Chuang (2013). The data obtained with the instrument can be used to identify and evaluate high school students’ perceptions of their English teacher knowledge in determining strategies by which teachers can improve their English language teaching competencies.

CALL, TPACK, and Students’ Perceptions of Teachers’ Knowledge

Researchers proposed that CALL teacher training interventions should aim at placing teachers in the authentic context while learning to integrate technology (Chapelle and Hegelheimer 2004; Hampel and Stickler 2005; Slaouti and Motteram 2006) and equipping teachers with more than just technology knowledge (Hegelheimer 2006; Kessler and Plakans 2008). Most studies on the effects of CALL teacher training programs have focused on how these courses influence teachers’ attitudes and confidence in integrating CALL into their teaching. CALL training courses are generally deemed helpful in encouraging L2 teachers to develop positive attitudes towards technology integration in classrooms and build confidence in implementing this measure (Liu and Kleinsasser 2015; Peters 2006; Van Olphen 2007).

For the past three decades, researchers have studied students’ attitudes towards the use of CALL and also developed CALL instruments to measure EFL (English as a foreign language) learners’ attitude towards CALL (Aryadoust et al. 2016). Several of students’ perception studies in CALL reported mostly about students positive attitudes towards CALL activities instead of their perception towards teachers’ knowledge. For example, Lin et al. (2011) explored EFL students’ perceptions of learning vocabulary collaboratively with computers and found that more than 70% of the participants in the computer group reported a positive attitude and anticipation to learning vocabulary in such an environment. Hsu et al. (2008) reported that English learners enjoyed the ease of using audioblogs and believed that audioblogs assisted their language learning experiences.

In addition, the animated pedagogical agent has been a powerful addition in CALL (Carlotto and Jaques 2016). The embodied agent, or talking-head animation, becomes a noticeable virtual aid for teaching pronunciation, vocabulary, articulation in language learning (Wik and Hjalmarsson 2009). The 3D talking-heads may be characterized as virtual instructors in language learning applications, supporting language acquisition especially practicing pronunciation (Chen and Massaro 2011). Mohamad Ali et al. (2015) investigated the benefit of inclusion of various verbal elements in 3D talking-head on pronunciation learning among non-native speakers. They found that 3D talking-head with spoken text and on-screen text mobile-assisted language learning has contributed significantly in retaining the correct pronunciation acquisition in comparison with 3D talking-head with spoken text alone and spoken text with on-screen text.

Meanwhile, classroom learning environment research values a student cognition paradigm that focuses on the ways in which students perceive their learning environments and react to their learning tasks. Without feedback from students, teachers might have difficulty reflecting their teaching practices. Using students’ perceptions will enable teachers to appreciate the perceived instructional influences on students’ learning processes and tasks (Senocak 2009). Previous research has recognized the significance of investigating students’ perceptions of teachers’ teaching as a means of enabling researchers and teachers to appreciate the perceived instructional and environmental influences on students’ learning processes and outcomes. Knight and Waxman (1991), and Tuan et al. (2000) specifically advocated for investigating students’ perceptions of teacher knowledge. Jang (2010) developed a method for assessing college students’ perceptions of teachers’ PCK in an empirical study of college professional development programs. However, these techniques were only PCK-bounded.

Technological advancements and the establishment of a foundation for pedagogical content knowledge (PCK) (Shulman 1986, 1987) motivated Mishra and Koehler (2006) to put forward the TPACK framework, which consists of seven knowledge areas: (i) content knowledge (CK), (ii) pedagogical knowledge (PK), (iii) technological knowledge (TK), (iv) PCK, (v) technological content knowledge (TCK), (vi) technological pedagogical knowledge (TPK), and (vii) technological pedagogical knowledge content TPACK. However, Tseng (2016) also found difficulties distinguishing some of the seven components of the TPACK construct proposed by Mishra and Koehler (2006) when trying to develop an instrument for assessing technological pedagogical content knowledge as perceived by EFL students. Meanwhile, Angeli and Valanides (2009) developed an argument describing TPACK as a distinct body of assessable knowledge and proposed teachers’ (TPACK) a holistic, integrated, and transformative form of knowledge, to be assessed by the five criteria. Shih and Chuang (2013) used these five criteria for assessing TPACK to develop items to assess student perceptions of teachers’ TPACK, one of the four categories in their students’ perceptions of faculty knowledge (SPFK) instrument. According to Shih and Chuang (2013), they have also chosen from two validated PCK related instruments by Tuan et al. (2000) and Jang (2011) the two common constructs of subject matter knowledge (SMK) and knowledge of students’ understanding (KSU). Viewing technological content as an added dimension to the original teachers’ PCK in a technology-mediated learning environment, they included technological knowledge (TK) as another category.

Thus, given the dominant force and prevailing presence of technology on college campuses and teaching in a digital age, Shih and Chuang (2013) stepped beyond the PCK boundary to develop a SPFK instrument that included SMK (Subject Matter Knowledge), TK (technological knowledge), KSU (Knowledge of Student Understanding), and TPACK, to assess the SPFK for college students, in response to the challenge of improving the learning experiences of college students, as well as the advancement of university faculty’s professional development in teaching.

Previous CALL studies discuss educators’ knowledge, particularly their TPACK, only from the perspective of teachers; they often highlight positive student feedback and use faculty observations as supplementary support for findings (Liu and Kleinsasser 2015; Rahmany et al. 2014). Exploring students’ perceptions illuminates the reality that students and their peers confront in the classroom, as well as enables researchers and teachers to appreciate the influence of perceived instruction on student learning processes and outcomes. Teachers should therefore have access to how their students perceive or evaluate the knowledge that they possess.

Research Objectives

The research objectives of this study are to develop and validate an instrument for assessing high school students’ perceptions of the knowledge possessed by English teachers who work in technology-supported environments. With this rigorously developed and scientifically based instrument, we seek to further examine and identify the factors that students regard as critical to teachers’ knowledge in the technology-supported environments.

Methods

Development of the SPETK Scale

We developed a paper-based, five-point Likert scale SPETK (students’ perceptions of their English teachers’ knowledge) questionnaire that was administered to high school students, who were asked to rate their English teachers’ knowledge on a range of 1 (the level of least agreement) to 5 (the level of greatest agreement).

Questionnaire development progressed as follows: (a) The literature on PCK, TPACK and students’ perceptions of teachers’ knowledge was reviewed. (b) The four categories of Shih and Chuang’s (2013) SPFK scale were used as reference in framing the dimensions reflected by SPETK. (c) The survey items under each dimension were conceptualized and revised for suitability to the English subject and the high school context. (d) Two high school students and two experts were consulted to confirm the readability and comprehensibility of the items and to correct ambiguities in the wording and phrasing of each item for the purpose of ensuring content validity. (e) The items were analyzed, and the validity, reliability, and structural soundness of the instrument were confirmed.

CFA of the Measurement Model

The first draft of the survey (SPETK) comprised four categories and 38 items: 5 on subject matter knowledge (SMK), 10 on TK, 6 on knowledge of students’ understanding (KSU), and 17 on TPACK. The construct validity of the scale was examined by confirmatory factor analysis (CFA) using Mplus Version 7 (Muthén and Muthén 1998–2012) with 287 valid high school student questionnaires. While the questionnaire contained the Likert-type scale that was ordinal in nature, previous studies have shown that in general, parametric tests, like factor analysis, are robust to the violation of normality assumption and therefore appropriate even with ordinal data (Norman 2010; Sullivan and Artino 2013). With the number of response categories on the current Likert scale being less than six (Rhemtulla et al. 2012), however, we used the MLM estimator in Mplus based on Satorra–Bentler corrections (Satorra and Bentler 1994), such that both Chi-square test statistic and standard errors produced accordingly would be robust to non-normality.

A four-factor model was first estimated with the original scale comprising 38 items, and two goodness-of-fit indices [i.e., root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR)] were used to examine model–data fitness. On the basis of the initial CFA results, 17 items were excluded from the draft, thus yielding a final scale that contained 21 items: 5 on SMK, 6 on TK, 4 on KSU, and 6 on TPACK (see Appendix). The measurement model based on the final scale had χ2 = 464.07, χ2/df = 2.54, p < 0.001, TLI = 0.928, CFI = 0.937, RMSEA = 0.073, SRMR = 0.048, indicating an adequate fit to the data (Hu and Bentler 1999). The summary statistics of the measurement model are presented in Table 1.

Table 1 Summary statistics of the measurement model (CFA)
Full size table

We calculated Cronbach’s alphas, composite reliability, and average variance extracted (AVE) to evaluate the internal consistency, convergent validity, and discriminant validity for the four major constructs measured by the instrument of SPETK. As shown in Table 1, the subscale reliability values of SMK, TK, KSU, and TPACK were 0.91, 0.86., 0.90, and 0.91, respectively, all of which met the conventional criterion of 0.7 for adequate internal consistency. In addition, the convergent validity for each subscale was overall satisfactory, since the values of composite reliability were all above 0.7 (ranging from 0.86 to 0.92), and those of average variance extracted (AVE) were largely above 0.5 (Fornell and Larcker 1981; Hair et al. 2006). Finally, the discriminant validity was also acceptable, as evidence by the fact that the square root of the AVE for a specific construct was to a large extent greater than the correlations between the construct in question and other constructs (Fornell and Larcker 1981), and none of the 95% CIs for the correlations among the constructs covered 1 (Anderson and Gerbing 1988).

Given the high correlations among the four major constructs as shown in Table 1, we conducted a supplemental analysis (not shown) where TPACK was regressed on SMK, TK, and KSU, simultaneously. While the result showed that all predictors were significant at α = 0.001, VIFs for the predictors ranged from 2.68 to 2.80, with the corresponding values of tolerance ranging from 0.36 to 0.37. Essentially, none of the VIFs exceeded 10 and none of the values of tolerance was less than 0.1, thus indicating no serious threat of multicollinearity (cf. Kutner et al. 2004).

Research Hypotheses

Using teachers’ self-report data as bases, Koh et al. (2013) examined the effects of teachers’ PK, TK, and CK on their TPACK. The authors found that the PK, TK, and CK of preservice teachers significantly predict TPACK, with the strongest influence exerted by PK. In particular, TK and PK exert positive direct effects on teachers’ TPACK. We argue that if the TPACK knowledge is critical to teachers’ professional development in the 21st century and this would affect the students’ learning, we seek to further understand the predicting power of TK, KSU(an alternative for PK), and SMK (an alternative for CK), on the overall TPACK from the students’ perception.

Therefore, we formulated the four following hypotheses:

H1

Students’ perceptions of teachers’ SMK, KSU, and TK are significantly and directly related to TPACK.

H2

Students’ perceptions of teachers’ SMK and KSU are significantly and directly related to TK.

H3

Part of the overall association between students’ perceptions of teachers’ KSU and TPACK is mediated by TK.

H4

Part of the overall association between students’ perceptions of teachers’ SMK and TPACK is also mediated by TK.

Participants, Data Collection, and Data Analysis

Students of four English teachers were recruited to investigate the learners’ perceptions of the teachers’ knowledge. The teachers were completing five-week CALL workshops into which the TPACK framework was integrated. A total of 287 valid questionnaires were collected after the end of the semester in January 2015. Among the respondents who returned the valid surveys, 95 were male and 192 were female. In terms of grade level, 201 were in the tenth grade and 86 were in the eleventh grade.

Descriptive statistics on the respondents’ personal profiles were obtained, and correlation analysis was conducted to further examine the unique effects of the SMK, TK, and KSU constructs on TPACK, as determined using a structural equation model; the model enables us to simultaneously validate the research questions and account for measurement errors (Bollen 1989; Kline 2005).

Results

Structural Equation Modeling: Hypothesized Mediation Model of SPETK

Model Specification, Estimation, and Evaluation

The mediation model of SPETK was specified according to the four research hypotheses presented above (Fig. 1). Notably, the hypothesized mediation model was fully recursive, in that the causal flows all went in one direction without feedback loops, and all possible paths were specified. With the same measurement structure, essentially, the fully recursive (or just-identified) structural model (i.e., the hypothesized mediation model) was identical in overall model fit to the confirmatory factor model with all latent constructs free to covary with one another (Maruyama 1998).

Fig. 1
figure 1

SEM: Hypothesized mediation model of SPETK. \(\chi_{(df\, = \,183)}^{2}\) = 464.071, χ2/df = 2.536, CFI = 0.937, TLI = 0.928, RMSEA = 0.073, SRMR = 0.048. All path coefficients were standardized, with asymptotic z values in the parentheses. ***p < 0.001, **p < 0.01

Full size image

Table 2 presents the total, direct, and indirect effects on the two endogenous latent variables (i.e., TK and TPACK). In essence, 80% of the variance in TK and 70% of the variance in TPACK, respectively, were accounted for by the entire model. Specifically, KSU exerted a significant direct effect on TK (β = 0.56, p < 0.001) and TPACK (β = 0.31, p < 0.01). While SMK was significantly directly related to TK (β = 0.39, p < 0.001), its direct association with TPACK was non-significant (β = 0.05, p > 0.05).

Table 2 Decomposition of standardized effects for the model
Full size table

The indirect effects of SMK and KSU, respectively, on TPACK through TK were tested with two strategies. First, the Sobel test (Sobel 1982) indicates that the standardized indirect paths of SMK → TK → TPACK (z = 3.14, p < 0.01) and KSU → TK → TPACK (z = 3.72, p < 0.001) were both significant. Moreover, using the bootstrap procedure (Shrout and Bolger 2002), we generated 10,000 bootstrap samples to compute bias-corrected 95% confidence intervals for the indirect effects. Again, both indirect effects were considered significant, since neither of the bias-corrected 95% CIs covered 0.

Last but not least, both SMK (z = 3.06, p < 0.01) and KSU (z = 7.63 p < 0.001) had significant total effects on TPACK. Taken together, H2 and H3 were fully supported by our data. That is, despite a significant direct effect, part of the association between KSU and TPACK was mediated by TK. On the other hand, H1 and H4 were partially supported, in that the effect of SMK on TPACK was fully mediated by TK, without a significant direct effect.

Discussion

English teachers’ TK is regarded as the pivotal mediator of high school students’ perceptions of their teachers’ knowledge. The results of the structural equation model indicate that the student-perceived teachers’ TK significantly and directly affects TPACK (β = 0.51, p < 0.001). Meanwhile, SMK and KSU exert a significant indirect effect on TPACK via the mediating TK. Given the affordance characteristics of technologies, the manner by which teachers operate technologies can be easily observed by students during a class. Related research reveals that teachers admit to having moderately high confidence in their TK; such confidence directly and positively affects their TPACK (Koh et al. 2010, 2013). Other studies derived similar results; that is, TK is a valid construct through which student perceptions of teacher knowledge can be understood (Shih and Chuang 2013; Tseng 2014). In the English teaching domain, technology inevitably plays a key role (Liu et al. 2014; Rahmany et al. 2014). Results from this study provide another evidence from the students’ perspective to confirm the important direct association of TK and TPACK.

The results obtained in the current work may be attributed to the demographics of the participating English teachers. They are young digital natives and therefore tend to easily accept new technologies; they are willing to integrate advanced innovations into their teaching. Such findings echo the perspectives of Orlando and Attard (2015), who maintain that the generation of hi-tech novice teachers freely exhibit their technological abilities to their students. This behavior reinforces their confidence in diffusing technologies into teaching practices. Teachers integrate technologies in their teaching practices and immerse students in an environment abundantly supported by technology; these approaches prompt students to regard technologies as part of their learning processes (Conole et al. 2008) and encourage them to make a connection to their teachers’ knowledge, with TK as a vehicle. It also reflects the importance that students attach to TK when evaluating teachers’ knowledge. Such importance drives students to establish a connection to teachers’ TPACK through TK as a mediator.

Studies indicate that KSU is one of the principal components of PCK (Halim et al. 2014; Jang et al. 2009; Park and Chen 2012) and regard it as a significant construct in the evaluation of students’ perceptions of teachers’ knowledge (Halim et al. 2014; Jang et al. 2009; Tuan et al. 2000). The results of the present study point to a direct association of KSU and TPACK, which corresponds with Tuan et al.’s (2000) argument that students expect teachers to possess students’ perceptions of how the teacher evaluates student understanding during interactive teaching and at the end of lessons and units. This expectation, in turn, contributes to teachers’ TPACK. Halim et al. (2014) revealed that low-achieving students, in particular, perceived the value and benefits when teachers pay attention to their needs, such as being sensitive to students’ reactions and preparing additional learning materials. In determining students’ perceptions of teachers’ knowledge, teachers must be sensitive to students’ needs, analyze their misconceptions and difficulties in learning and appropriately evaluate students’ comprehension of specific subjects (Jang et al. 2009; Park and Chen 2012; Park and Oliver 2008). Research on such perceptions in technology-supported environments reveals that students expect teachers to suitably assess their understanding during the course of teaching and learning. The findings of the present research suggest that students value teachers’ KSU and that such valuation directly influences their perceptions of teachers’ TPACK.

Conclusion

This study was intended to design an instrument of SPETK to determine students’ perceptions of English teachers’ knowledge and to investigate how high school students respond to their English teachers’ instructions in technology-supported class environments. Students’ perceptions could further enable teachers to reflect the impact of their instructions on student learning processes and outcomes.

The four constructs of this developed and validated SPETK instrument, the SMK, KSU, TK, and TPACK of the SPETK scale, correlated with one another with the relationship being significantly positive. Further analysis based on the structural equation modeling reflects an indirect association between students’ perceptions of their teachers’ SMK and TPACK and KSU and TPACK with TK as a mediator. These results may be attributed to changes in teacher demographics; that is, the teachers of today belong to a generation of young digital natives with technological knowledge and traits easily observed or perceived by students when these are integrated into teaching. KSU is also directly associated with TPACK, thus indicating that the construct is essential to high school students’ perceptions of teachers’ knowledge. This result implies that teachers should adopt adequate assessment methods in understanding students learning situations and correcting misconceptions during learning.

A notable limitation of the current study is that we used a purposive sample consisted of 287 high school students from four English teachers. Therefore, the results may not be generalizable to other populations with different education levels or in different cultures or/and geographical regions. Future research can revolve around investigating and comparing the knowledge levels of novice and experienced teachers from the standpoint of students to exhaustively determine the relationships among knowledge factors (e.g., SMK, KSU, TK, and TPACK). Researchers can also analyze different teaching experiences and teachers’ knowledge constructs to ascertain the factors that are viewed by students as the most crucial as they provide feedback on teachers’ professional development and technologically oriented teaching design.