Introduction

Classroom assessment is essential to predict students’ academic development and success (Wilson 2018), and academic engagement is one of its best predictors, hence, it should be effectively evaluated. As Sinatra et al. (2015) stated, students’ engagement could be described as the holy grail of learning, due to its relationship with positive educational outcomes, both inside, and outside of school. However, despite the importance of this construct for education, the challenges with its conceptualization and its measurement still remain (Alrashidi et al. 2016; Bae and Han 2019).

Alrashidi et al. (2016, p.41) pointed out that, in current literature, there are two major approaches to engagement: The North American Model entails students’ cognitive, behavioral, and emotional engagement (Fredricks et al. 2004), while the European Approach of Engagement, is characterized by vigor, dedication, and absorption (Schaufeli et al. 2002; Upadyaya and Salmela-Aro 2013). Therefore, in this paper we display a comparative analysis of two measurement scales representative of each of the two models of student engagement to verify their convergent and discriminant validity (a psychometric objective), which translates into an important substantive one: if the two theoretical conceptualizations converge in such a central construct for the educational context, and particularly, in higher education.

Literature Review

Conceptualizations and Dimensions of Engagement

The term “engagement” is used to describe the participation of students at school; it was established in North American educational research at the end of the 80s (Appleton et al. 2008; Baron and Corbin 2012; Fredricks et al. 2004). As we have noted above, there are two substantially different traditions in the conceptualization of engagement. One of them, (American Model) originates from the educational context, it has a motivational perspective and conceives engagement as a multidimensional construct (Fredricks et al. 2004; Wang et al. 2019). The other tradition, (European Model) has its roots in organizational psychology (Schaufeli et al. 2002).

From an educational point of view, scientific literature conceives engagement as a multidimensional construct. However, researchers do not end up agreeing on how many and which are its dimensions. Thus, Fredricks et al. (2004) considered that school engagement is composed by three dimensions: cognitive, emotional and behavioral. The cognitive dimension refers to mental operations and learning strategies that students use while learning or completing a school task (Fredricks et al. 2004; Walker et al. 2006). The emotional dimension may be defined as the interest and positive feelings a student experiences during learning tasks, and the absence of negative emotions, anxiety or stress (Fredricks et al. 2004; Skinner et al. 2009). The behavioral dimension refers to the effort, persistence, and actions taken by the students in order to achieve academic objectives (Fredricks et al. 2004; Skinner et al. 2009). Later, Reeve and Tseng (2011) added a fourth dimension to the overall construct of engagement: agentic engagement. Agentic engagement occurs when students contribute constructively to the teaching and learning process (Reeve 2013; Reeve and Tseng 2011).

From the work and organizational perspective, school engagement is structured in three dimensions directly related to those found in the work context. These dimensions have been found to predict positive work functioning: energy or vigor, dedication, and absorption (Schaufeli et al. 2002). Initially, the notion of engagement was conceptualized as work-related engagement, however, this concept was later expanded to include the notion of study engagement (Ouweneel et al. 2014; Schaufeli et al. 2002; Upadyaya and Salmela-Aro 2013). This is so, because from a psychological point of view, students’ tasks and activities can be considered ‘work’ (Salmela-Aro and Upadyaya 2012). Analogously to work engagement, study engagement is characterized by students’ feelings of vigor, their dedication to their studies, and their absorption in their academic-related tasks and activities (Schaufeli et al. 2002). Schools are settings in which students work, attend classes, and carry out projects and assignments, so that the concept of engagement can reasonably be extended to the context of schoolwork (Salmela-Aro and Upadaya 2012). Indeed, as Salmela-Aro and Upadyaya (2012) pointed out, the three dimensions of the Utretch Work Engagement Scale exactly correspond to EDA dimensions with the only difference being that of the referent to which the scales are answered: work vs. school. Energy/vigor refers to the mental capacity for studying, and the positive approach and willingness to invest efforts in school work (Ouweneel et al. 2014; Salmela-Aro and Upadyaya 2014; Upadyaya and Salmela-Aro 2013). Dedication is characterized by enthusiasm and identification with the academic task, in addition to perceiving schoolwork as significant (Ouweneel et al. 2014; Salmela-Aro and Upadyaya 2014). Finally, absorption is characterized by so high levels of concentration and productivity, that the perception of time is lost, and the feeling that the activity in itself is rewarding (Ouweneel et al. 2014; Salmela-Aro and Upadyaya 2014; Upadyaya and Salmela-Aro 2013). This last dimension of engagement is close to the concept of flow, an involving experience which is so pleasant that people tend to desire it purely for the sake of the experience itself, as Csikszentmihalyi (1990) pointed out.

Both the American model of engagement and the European model were initially created to be applied in the context of Primary and Secondary education, however, with increasing frequency they have also been applied in university settings, due to its evident relationship with social, emotional, motivational and academic performance aspects, as well as satisfaction with life and with the educational center, and persistence in studies (e.g., Bae and Han 2019; Beasley 2020; Bosselut et al. 2019; Garn et al. 2017; Gehrke and Kezar 2019; Gutiérrez and Tomás 2019; Loscalzo and Giannini 2019; Martínez et al. 2019; Matos et al. 2018; Orio-Granado et al. 2017; Parra and Pérez 2010; Schaufeli and Bakker 2003; Tuominen-Soini and Salmela-Aro 2014).

Comparison between Fredricks et al.’s (2004) and Schaufeli et al.’s (2002) Models of Engagement

Alrashidi et al. (2016) stated that the American and the European models (e.g., Fredricks et al. 2004; Schaufeli et al. 2002) of engagement are taping different features of students’ underlying academic engagement, although both emphasize the link between engagement and successful academic outcomes (Phan 2014; Upadyaya and Salmela-Aro 2013). Therefore, these two approaches present both, similarities and differences.

Both approaches include willingness to make an effort, persistence in tasks, preference for challenge, resilience, or a positive approach towards learning in their definitions of the different dimensions of engagement (Alrashidi et al. 2016). Fredricks et al.’s (2004) American model includes items such as investment in school-related activities or willingness to learn in the dimension of cognitive engagement, all very close to the enthusiasm and inspiration mentioned by Schaufeli et al. (2002) in the dedication dimension (Tuominen-Soini and Salmela-Aro 2014; Upadyaya and Salmela-Aro 2013). Furthermore, in Schaufeli et al.’s (2002) model, vigor includes a positive approach to school-related activities, again, very close to positive feelings about school and enjoyment, present as well, in the emotional engagement of the American model (Tuominen-Soini and Salmela-Aro 2014; Upadyaya and Salmela-Aro 2013). In the same vein, the absorption factor in the European model shares features with the behavioral dimension in the Fredricks et al.’s (2004) model, in which, as well as in the European model, it is described the focus on school tasks and duties, and behavioral performance (Tuominen-Soini and Salmela-Aro 2014; Upadyaya and Salmela-Aro 2013).

Nevertheless, academics also point out some differences between the two models, because dedication, absorption, and vigor in Schaufeli et al. (2002) describes students’ psychological engagement more than their actual behavior at school (Upadyaya and Salmela-Aro 2013; Schaufeli et al. 2002). Therefore, Schaufeli et al.’s (2002) model lacks indicators, for example, on school attendance, adherence to norms and rules, or adequate behavior with teachers and other staff members (Upadyaya and Salmela-Aro 2013). Emotional engagement in the American model includes feelings about classmates and teachers and the school as a whole, whereas the European model only considers feelings towards the studies. Despite these differences, the detailed description of students’ psychological engagement of the Schaufeli et al.’s (2002) model may provide new information from a motivational perspective of engagement (Phan and Ngu 2014a; Upadyaya and Salmela-Aro 2013).

Measurement of Student Engagement

Several instruments to measure student engagement have consistently appeared in specialized literature. For example, the Student Engagement Instrument (SEI; Appleton et al. 2006), the Student Engagement in Schools Questionnaire (SESQ; Hart et al. 2011), the Australasian Survey of Student Engagement (AUSSE; Hagel et al. 2012), the Student School Engagement Measure (SSEM; Hazel et al. 2013), the Student Engagement Scale (SES; Lam et al. 2014), the Classroom Engagement Inventory (CEI; Wang et al. 2014), the Student Engagement in School Success Skills (SESSS; Brigman et al. 2015), the Multidimensional School Engagement Scale (MSES; Wang et al. 2019), the School Engagement Scale (SES; Reeve 2013), the Utretch Work Engagement Scale (UWES; Schaufeli et al. 2002), and the Schoolwork Engagement Inventory (EDA; Salmela-Aro and Upadyaya 2012).

In this study, we will focus on two scales which represent the central axis of each approach to student engagement: “The School Engagement Scale (SES)” by Reeve (2013) representative of the American Model, and “The Schoolwork Engagement Inventory (EDA)” by Salmela-Aro and Upadyaya (2012), representative of the European Model of student engagement.

The School Engagement Scale

(SES; Reeve 2013) is based on Reeve and Tseng’s model of academic engagement, which incorporates the three “classical” dimensions of engagement (cognitive, behavioral and emotional) plus the “new” dimension of agentic engagement. Reeve and Tseng (2011) defined agentic engagement as “students’ constructive contribution into the flow of the instruction they receive” (p. 258). In this way they created a scale of 22 items that measured the aforementioned four factors: behavioral engagement (5 items), agentic engagement (5 items), cognitive engagement (8 items) and emotional engagement (4 items). In their study, Reeve and Tseng (2011) concluded that agentic engagement was conceptually distinct from the three other dimensions of engagement.

Reeve (2013) tried to refine this scale to create a brief, construct-congruent, and psychometrically sound self-report questionnaire; which he later validated and refined. In this new, shortened, and refined scale, a CFA of the 17 remaining items had a good fit to a four-factor model, as hypothesized. The four factors were reliable, with alphas ranging from .72 (cognitive engagement) to .91 (emotional engagement). Additionally, Reeve (2013) found that the correlations among the four dimensions of engagement and academic achievement were statistically significant; he also found a positive relationship between engagement and autonomous motivation, and a negative relation with controlled motivation. Finally, Reeve (2013) longitudinally studied the effect of agentic engagement on teacher’s support of autonomy and found significant positive effects. He concluded that agentic engagement plays a new role among the measures of engagement as it allows educators to fully appreciate how students actually engage themselves in learning activities, trying to create a motivationally supportive learning environment for themselves. Recently, Matos et al. (2018) have developed a scale based on the work of Reeve (2013) and Jang et al. (2016) with the same four factors but including more items. This scale was validated in Peruvian college students.

The Schoolwork Engagement Inventory: Energy, Dedication, and Absorption

(EDA; Salmela-Aro and Upadyaya, 2012) is based on the works by Schaufeli et al. (2006). Salmela-Aro and Upadyaya (2012) analyzed the psychometric properties of the EDA and found that a one-factor solution had the highest reliability and fitted the best among younger students, whereas a three-factor solution was the most reliable and fitted the best among older students.

Some years back, Schaufeli et al. (2002) and Schaufeli and Bakker (2003) developed an instrument to measure work engagement: The Utretch Work Engagement Scale (UWES). This scale was mainly used in the organizational psychology arena. Parallelly, a version for students was developed (UWES-S), this version was composed by the same three factors found in the work engagement version: (a) energy/vigor (6 items); (b) dedication (5 items); and (c) absorption (6 items). After successive depurations, several items were excluded and a 9-item version (UWES-9; Schaufeli et al. 2006) was established, with three items per dimension. Internal consistencies of the dimensions ranged between .89 and .97.

Although the three-factor theoretical structure of work engagement seems to be the most common in literature (e.g. Balducci et al. 2010; Loscalzo and Giannini 2019; Spontón et al. 2012), there is still a lack of agreement on its structure. Some authors found a one-factor solution to fit the data better than the three factors (e.g., Souza et al. 2015). Additionally, a two-factor solution has also been proposed (Parra and Pérez 2010). Portalanza-Chavarria et al. (2017) validated the UWES-9 in Ecuadorian college students with the best solution being a two-factor structure (vigor-absorption and dedication). García-Ros et al. (2018) found that, using the Student Engagement Inventory (EDA) with teenage students, one, two and three factor models adequately fitted the data, and therefore, they retained the one-factor solution for the sake of parsimony.

Relationships among Engagement, its Precursors and Students’ Achievement

School or student engagement has shown a great potential for research in educational problems in the last decades, as well as in positive educational outcomes such as academic success (Gutiérrez et al. 2017; Wang and Fredricks 2014). Regarding the constructs that constitute the nomological net of student engagement, the literature has shown strong links with educational variables such as school adaptation (Fredricks et al. 2016), subjective well-being (Upadyaya and Salmela-Aro 2013), teachers’ autonomy support (Jang et al. 2010), students resilience (Skinner and Pitzer 2012), academic self-efficacy (Zhen et al. 2019), life satisfaction (Hazel et al. 2014), gratitude (Zhen et al. 2019), school satisfaction (Curran et al. 2016; Gutiérrez et al. 2017), satisfaction of basic psychological needs (Jang et al. 2016; Molinari and Mameli 2018), academic achievement (Carmona-Halty et al. 2019; Jelas et al. 2016), as well as the development of leadership skills, and networking (Gehrke and Kezar 2019). On the contrary, the lack of student engagement, is also related to boredom in the classroom (Tzé et al. 2014), high crime rates, substance abuse, and depression in students (Wang and Fredricks 2014).

Studies dedicated to analyzing the relationships between engagement and other variables in the educational context (both precursors of engagement and students’ achievement) have been frequently carried out in primary and secondary educational levels. However, in the last years, the number of studies at the university context has increased. For example, based on the Fredricks et al.’s (2004) model of engagement, Garn et al. (2017) found that emotional engagement was a positive predictor of enjoyment, while enjoyment was a positive predictor of change in students’ behavioral and emotional engagement; Matos et al. (2018) found that autonomy-supportive teaching systematically predicted all types of engagement (the four factors), and that agentic engagement, in turn, had an effect on autonomy-supportive teaching; Gutiérrez and Tomás (2019) found that teacher’s autonomy had important direct effects on engagement and self-efficacy, and these, in turn, had direct effects on subjective well-being. In the same line, engagement has been identified as a key component to enhancing the academic outcomes of college students (Beasley 2020). In sum, many researchers have stated the value of this construct to understand academic outcomes in university context. For example, Bosselut et al. (2019) believes that increasing our knowledge on what variables affect engagement can assist educational agents in maximizing student engagement within the university setting, while Kahu (2013) stated that engagement is widely recognized as an important influence on achievement and learning in higher education.

Phan and Ngu (2014b) analyzed the dimensions of engagement in the European model and found that the three dimensions, when combined as engagement, were significant predictors of students’ performance. Similarly, Casuso-Holgado et al. (2013) found strong associations among students’ GPA and the three dimensions (vigor, dedication, and absorption). Oriol-Granado et al. (2017) found that positive emotions, autonomy support and self-efficacy predicted academic engagement, and the latter improved academic performance, while Martínez et al. (2019) found a positive relationship between academic engagement, and academic performance. Moreover, there is longitudinal evidence of the positive impact of vigor, dedication, and absorption on students’ academic performance over time. For example, Phan and Ngu (2014a) found that absorption, vigor, and dedication significantly explained grades in a panel study.

Present Study

There are modest variations in the definitions and operationalizations of the dimensions in both approaches (Fredricks et al. 2004; Schaufeli et al. 2002), although there is clear evidence for a positive short-term and/or long-term influence of engagement on academic achievement. This fact supports the argument stated by Appleton et al. (2008) that, no matter which approach to engagement is used, there is strong empirical support for the link between engagement and academic performance. Based on this statement, and given that scientists lack empirical comparisons between the two conceptualizations of engagement, the objective of this paper is to compare the result of applying two engagement measurement scales in the same population of university students, the School Engagement Scale (SES; Reeve 2013) and the Schoolwork Engagement Inventory (EDA; Salmela-Aro and Upadyaya, 2012), corresponding to each of the theoretical engagement models explained above, to better understand their similarities and differences. Therefore, this research has two objectives: Firstly, to study the factor structure of the SES and the EDA, and secondly, to study the nomological validity of the two traditions of engagement.

Method

Sample and Procedure

The sample was composed of 870 Psychology and Education students from the Universidad Autónoma de Santo Domingo (Dominican Republic). Their age ranged from 18 to 45 years old (M = 25.99, SD = 5.09). Women were 49.4% of the sample. Students of Psychology were 35% of the sample, while 69.5% were studying Education. Sampling was made by convenience. Participants knew the purpose of the study, participation was voluntary and they gave informed consent. Anonymity was guaranteed. Participants took a self-administered survey in their classroom settings with the presence of a researcher willing to answer any questions they may had. The average time taken to fulfill the questionnaire was 20 min. The research met the standards contemplated in the ethical code of the American Psychological Association. Both, the Ministry of Education of the Dominican Republic, and the academic authorities of the educational center gave their permission for the research.

Measures

The Student Engagement Scale

(SES; Reeve 2013) taps four dimensions (see Table 1): Behavioral Engagement (4 items), Agentic Engagement (5 items), Cognitive Engagement (4 items), and Emotional Engagement (4 items). We used the Spanish adapted and validated version of the SES for university students (Gutiérrez and Tomás 2019).

Table 1 Alternative models fit estimates
Full size table

The Schoolwork Engagement Inventory

(EDA; Salmela-Aro and Upadyaya, 2012) measures three dimensions of engagement (see Table 1): Energy (3 items), Dedication (3 items), and Absorption (3 items). We used the Spanish version of the EDA scale adapted and validated by Gutiérrez et al. (2018).

Teachers’ Autonomy Support

was assessed with Jang et al.’s (2012) short version of the Learning Climate Questionnaire (LCQ, Williams and Deci 1996). This is a six-item scale measuring teachers’ behaviors to encourage students’ autonomy. Examples of items are “My teachers listen to how I would like to do things” or “My teachers encourage me to ask questions”. This scale has been widely used in academic settings (Jang et al. 2009; Jang et al. 2012). Alpha in this sample was .85 and Composite Reliability Index (CRI) was .87. This scale was adapted and validated into Spanish in the study by Gutiérrez et al. (2018).

Academic Self-Efficacy

was measured with the Academic Self-Efficacy Scale (Midgley et al. 1998), a 6-item unidimensional instrument. It measures students’ beliefs about their possibility of mastering certain material or subject given enough time and dedication. Examples of items are “If I try, I can do even the hardest work in this course” or “If I have enough time, I can do a good job with my homework”. Alpha in this sample was .83 and CRI was .83. We used the adaptation and validation of this scale into Spanish from Tomás et al. (2020) study.

Life Satisfaction

was measured with the Satisfaction with Life Scale by Diener et al. (1985). It is a unidimensional scale composed by five items. Examples of items are: “I am satisfied with my life”. The alpha in this sample was .77 with a CRI of .78. This scale has been validated into Spanish several times, among others in the study by Galiana et al. (2015).

Satisfaction with the Center

was measured by the School Connectedness Scale (Nearchou et al. 2014). It is a scale composed by 5 items which measure the satisfaction of students with their center, with items such as “I feel happy to be in this school” or “I feel as I am part of this school”. The alpha of this scale was .79 with CRI of .83. This scale was adapted and validated into Spanish in the study by Gutiérrez et al. (2018).

All the scales used a Likert-type format ranging from 1 “totally disagree” to 5 “totally agree”.

Statistical Analysis

Several Structural Equation Models, specifically Confirmatory Factor Analyses (CFA), Bifactor CFA, and Exploratory Structural Equation Models (ESEM) were estimated and tested. Whereas CFA places strict hypotheses on which indicators (items) should load on each latent variable, usually constraining to zero cross-loadings, while ESEM are more flexible by allowing non-zero cross-loadings, a situation quite plausible when two scales measuring the same broad construct are analyzed together (Marsh et al. 2014; Asparouhov and Muthén 2009). Syntax code from De Beer and Van Zyl (2019) was used to estimate ESEM models. Robust estimation was used for all the models to overcome the non-normality and ordinal nature of the items (Hancock and Mueller 2013). Fit of the alternative or competing models was assessed by means of several indices and statistics from different families (Tanaka 1993): a) chi-square; b) Comparative Fit Index (CFI); b) the Standardized Root Mean Residual (SRMR); c) Root Mean Square Error of Approximation (RMSEA) with its 90% confidence interval; and d) several information criteria, specifically Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC) and Adjusted Bayesian Information Criterion (ABIC). Criteria for acceptable model fit were CFI above .90 (better fit above .95), and RMSEA and SRMR below .08 (Marsh et al. 2004). Lower levels of Information criteria are indicative of better relative fit. Internal consistency was estimated with Cronbach’s alpha and also with Composite Reliability Indexes (CRI), Explained Common Variance (ECV), and omega estimate of reliability (Dueber 2017) in order to address some of the limitations of Cronbach’s’ alpha (Hancock and An 2018). Validity of scales was assessed correlating both scales dimensions with other constructs. Alphas and correlations were calculated in SPSS v.24 and Structural Equation Models with Mplus v.8 (Muthén and Muthén 1998-2017).

Results

Factor Structure

Nine competitive Structural Equation Models (SEM) were specified and tested. All of them were specified to better understand the convergent and divergent validity of the two traditions of engagement measurement. For a graphical representation of all SEM models see Fig. 1.

  • Model 1. One-factor CFA model. If this model fits the data better than any other model implies that both scales are unidimensional and the construct of school engagement they pretend to measure is the same, without differences.

  • Model 2. Seven correlated dimensions CFA: energy, absorption and energy, and behavioral, emotional, agentic and cognitive engagement. This model follows the theoretical dimensions of the two scales.

  • Model 3. Two factors CFA of school engagement, each one associated to the items of one of the two scales. If this model fits the data better than the others, it implies that both scales are unidimensional but the constructs of school engagement they measure are different (distinguishable).

  • Model 4. Hierarchical CFA model with seven first order factors (the same as in model 3) plus two correlated second order factors, one associated to each scale. If this model fits the data better than the other models implies that theoretical dimensions in the two scales are adequate, but they measure two different constructs of school engagement.

  • Model 5. Hierarchical CFA model with seven first order factors (the same as in model 3) plus one second order factor, school engagement. If this model fits the data better than any other model implies that the theoretical dimensions in both scales are present but they also measure a single supra-factor of school engagement.

  • Model 6. Bifactor CFA model, in which each indicator (item) is both explained by a general factor of engagement and a specific factor (each one of the seven original dimensions of the two scales).

  • Model 7. Seven factor ESEM model. The same seven correlated dimensions as in Model 2, but non-zero cross-loadings are specified for all items (indicators).

  • Model 8. ESEM Bifactor model.

  • Model 9. Hierarchical ESEM model. Seven first order factors and a single second order factor.

Fig. 1
figure 1figure 1figure 1

Competing (alternative) structural models

Full size image

Table 1 shows the goodness-of-fit indexes for all the SEM tested. All CFA models (models 1 to 5) showed poor fit.

The bifactor CFA model also had a poor fit. In other words, unless cross-loadings are posited by means of different ESEM structures (models 7 to 8) fit indexes did not reach the cut-off criteria for adequate fit. In the three cases of the ESEM models CFI was larger than .90, although only the bifactor-ESEM was very close to the .95 cut-off. Again, both RMSEA and the SRMR were in all three cases well below .08. Chi-square statistic was lowest in the case of model 8. Information criteria allow to better compare competitive models, and all of them (AIC, BIC and ABIC) were lower for model 8 (Bifactor-ESEM).

These three models were also assessed in terms of interpretability of the loadings in the different factors and we consider the most interpretable was model 8, which divides variance of each indicator into a general factor of engagement and the seven specific dimensions. Standardized factor loadings can be seen in Table 2. A general view at the table shows firstly that all indicators load moderate to large and significantly in the general engagement factor. Secondly, we can also see that there is a non-negligible amount of specific variance present in most indicators. Behavioral, agentic and emotional indicators all loaded significantly in their corresponding specific factors and had respectively 6, 7, and 8, significant cross-loadings. Regarding cognitive engagement, energy, dedication and absorption indicators only had one indicator in each, that did not load significantly in their corresponding factor, and these dimensions also had 4, 4, 6, and 7 cross-loadings with other specific factors.

Table 2 Original dimension, and items’ means, standard deviations, standardized factor loadings (λ), and residuals
Full size table

Reliability

Internal consistencies for all factors in the two scales were calculated with both Cronbach’s alpha and Composite Reliability Indexes (CRI), although, the CRIs were calculated with standardized coefficients from the Bifactor-ESEM model that had the best fit. Alpha for items of the two scales together was .94, and for the specific dimensions were: .71 for energy; .74 for dedication, .64 for absorption; .82 for behavioral; .81 for agentic; and .77 for cognitive and emotional engagement. The CRI for the general factor was .93 and for the specific factors were: .25 for energy; .26 for dedication; .10 for absorption; .39 for behavioral engagement; .49 for agentic; .33 for cognitive; and .31 for emotional engagement. Additionally, ECVs have been calculated for the general factor (.649), as well as for the specific dimensions: .036 for energy; .040 for dedication; .107 for absorption; .066 for behavioral engagement; .071 for agentic; .056 for cognitive; and .058 for emotional engagement. Finally, omega estimates were calculated for the general factor with a result of .952, and for the specific seven dimensions of engagement. The omega estimate for specific factors represents the proportion of reliable variance in the subscale composite that is independent of the general factor. The estimates were: .001 for energy; .003 for dedication; .001 for absorption; .011 for behavioral engagement; .017 for agentic; .008 for cognitive; and .008 for emotional engagement.

Validity

In order to explore nomological validity, we have estimated several structural models in which the seven dimensions of the two scales of school engagement are modeled together with each construct that the literature has consistently found related to school engagement (autonomy support, academic self-efficacy, life satisfaction, and satisfaction with the school) plus age and gender. Goodness-of-fit indices are presented in Table 3. All models had a reasonable fit, and therefore, allowed to estimate relationships among latent variables.

Table 3 Fit of models to test for criterion-related validity
Full size table

All correlations among the seven latent dimensions of engagement and criteria are shown in Table 4. All correlations are statistically significant (p < .01), positive as expected and large. Regarding age, correlations were positive, low and significant with all dimensions of engagement except emotional. Finally, women had slightly lower means in dedication, and behavioral, cognitive and emotional engagements.

Table 4 Correlations of the seven dimensions of school engagement with autonomy support, self-efficacy, life satisfaction, and school engagement
Full size table

Discussion and Conclusions

It is not infrequent in Psychology to find a flourishing of different definitions of broad concepts that lead to different operationalizations, and therefore, to problems with generalization of findings. This is a topic that taps the extremely important issue of measurement adequacy. School or academic engagement is one of these constructs, that has presented such operational problems. Although engagement is diverse in its definitions and dimensionality, there is consensus in the scientific community about its multidimensionality (Alrashidi et al. 2016). However, there are still discrepancies regarding the number and nature of the dimensions which compose school engagement. Hence, it is then of outstanding interest to establish if two of the most widely adopted operationalizations of traditions in the academic engagement arena converge on similar results and structure, or on the contrary, significantly differ. This was the aim of this research.

In order to fulfill this aim, a first step was evaluating if both scales present the dimensions their authors first defined. To prove this, several CFAs, bifactor CFAs, and ESEM models were estimated with the purpose of finding the model which optimally represents data for both scales. From the nine estimated models, model eight had the best fitting indexes, thus, a bifactor ESEM model in which items explained variance was divided between a general underlying factor and the dimensions to which each item were presumed to belong. This may be considered a relatively expected result considering two scales which intend to measure the same broad construct. In other words, the total of seven dimensions are empirically distinguishable and show discriminant validity, although there is a big portion of variance explained by an underlying general factor in which all items converge. These results are difficult to compare with previous findings because no one has tested the two scales together before. However, given that the theoretical structure of both scales has been in part supported in our results, the previous research on each scale alone can be compared.

Regarding the factor structure of the SES scale, our best-fitting model is congruent with previous literature by the authors who found support for the same four dimensions in the SES scale (Jang et al. 2016; Reeve and Tseng 2011; Reeve 2013). There is also evidence for a four-factor stable solution in Peruvian (and therefore Spanish speaking) population, even though in a version with more items (Matos et al. 2018). All items loaded significantly on their corresponding dimension, besides the non-negligible amount of variance on other factors, except item 10 “When I study for my classes, I try to connect to my own experiences”, which belonged to cognitive engagement dimension. This item loads significantly of course on the general factor, and also on the agentic engagement dimension, this might be because a student sharing their own experience in class regarding a certain topic it is still interfering with the flow of instructional process by changing the course of a topic, it might open a debate about that specific topic and slightly modify teachers’ program for the class.

Regarding the EDA scale, our results are congruent with all the authors who have found support for the theoretical three-factor structure, model results indeed indicated a three-dimensional solution, although not all items loaded significantly on their corresponding factor: item 2 (“I feel strong and vigorous when I am studying”), for example, had a significant loading on behavioral engagement dimension from SES scale; and item 4 (“I find the schoolwork full of meaning and purpose”) corresponding to dedication, loaded on absorption dimension. It is important to note that the support for this three-factor solution is far from universal, as many other authors have found other (mainly unidimensional) solutions. Indeed, several studies found a three-factor structure (e.g. Balducci et al. 2010; Loscalzo and Giannini 2019; Spontón et al. 2012). However, authors as Portalanza-Chavarria et al. (2017) found that the best solution was a two-factor structure (vigor-absorption and dedication) in Ecuadorian students. Two factors were also found by Parra and Pérez (2010). In the same vein, García-Ros et al. (2018) using the EDA with teenage students found that one, two and three factor models adequately fitted the data but that the one-factor solution was the best solution based on parsimony considerations. A one factor solution was again found by Souza et al. (2015). A potential explanation for this confusing structure may lay in the age and/or educational level of the samples studied. Salmela-Aro and Upadyaya (2012) after conducting several CFAs, found that a one-factor solution better fitted the data from younger students while the three-factor structure was better for older (university) students. A very recent study, for example, with Italian college students found again a three-factor solution (Loscalzo and Giannini 2019). In sum, being our sample composed of university student we may say that our results basically agree with most of the already offered evidence. Overall, these results indicate that both definitions of these constructs have a very similar background but potentially could relate slightly different with other variables.

In addition to the basic factor structure of both scales, it is necessary to comment on dimensions’, convergent and discriminant validity. The fact that there is a big general factor underlying all items indicates that if there are two engagement constructs, they are extremely overlapped and they share a huge amount of common variance. This might be because the underlying mechanism of feeling engaged with school might be the same, although the manifestation of it differs by different dimensions. In other words, although the two traditions look quite different, the construct seems well tapped by both of them. This is interesting because the research results coming from these two traditions may well been generalized.

A different type of evidence on the nomological validity of the seven dimensions comes from the correlations with other constructs that are related in a nomological net with engagement. All dimensions showed similar overall patterns of correlations with autonomy support, self-efficacy, school and life satisfaction and satisfaction with the center. There are a few quantitative differences in the amount of correlation among the seven dimensions of engagement and these constructs. Among the EDA factors, seems that all three dimensions correlate positive and high with these variables. Additionally, all three were positively related with age of students, and only dedication had a correlation with gender, indicating that women had a slightly lower score on this dimension. Regarding SES scale, all correlations were high and positive, except autonomy support with behavioral and cognitive engagement, which were slightly lower than the rest. Age was positively related to all, except emotional engagement, while gender was slightly negatively related to all, except agentic engagement, which might be due to the fact this type of engagement relates to more stable traits of a student rather than their age. These correlations were already supported by previous research relating engagement and teachers’ autonomy support (Jang et al. 2010), for instance, in Matos et al.’ (2018) results, autonomy-supportive teaching systematically predicted all types of engagement using the SES scale in university students. Additionally, the relationship between school engagement and academic self-efficacy is also congruent with previous literature (e.g. Oriol-Granado et al. 2017; Zhen et al. 2019), so it is engagement with life satisfaction (Hazel et al. 2014), and finally, engagement with school satisfaction (e.g.: Curran et al. 2016; Elmore and Huebner 2010; Gutiérrez et al. 2017).

A final note on the reliability of the measures is that results from previous literature were quite similar to the ones in this study in the SES scale, with a slightly slower internal consistency in this case (Reeve 2013). Alphas of EDA were clearly lower in this sample than in previous studies (Salmela-Aro and Upadyaya 2012).

All considered, as far as we are concerned, this is the first study that examines the psychometric properties and validity of these two academic engagement scales together. It can be concluded that there are seven distinguishable dimensions, but there is a big general factor of academic engagement which underlies all dimensions in a way that both scales behave in very similar ways, that is, regardless of the variety of conceptualization of school engagement, there is strong empirical evidence supporting that they have similar impact on external variables. Thus, we agree with Alrashidi et al. (2016) in that, academic engagement at last is a term that captures students’ quality learning. Although findings from this study are relevant to shed light on the conceptualization and measurement of such a core construct involved in academic success, results should be taken with caution since there are several limitations: variables should have been measured in several moments in order to compare evolution over time of all dimensions, sample could be selected randomly in order to better represent the population, and convergent and divergent validity could be evaluated with a wider range of different variables. Future research should overcome these limitations to conclude with a consistent conceptualization of the construct of school engagement.