Introduction

With the inauguration of 12-year Basic Education, free tuition in vocational high schools, and promotion of major technical and vocational policies in Taiwan, the government has devoted a large sum of its budget and resources. Thus, school management effectiveness has received more public concern than in the past. In this study, a Ministry of Science and Technology research project was conducted: “A Study on the Constructions of Vocational High School Principals’ Competing Values Leadership Effectiveness Indicators, and Their Relationships with Behavior Change Intention Based on the 360 degree evaluation system.” The study surveyed a total of 69 principals as the evaluates. In addition to self-evaluation, each principal was evaluated by another 20 people, including the educational administration supervisors, other schools’ principals and the teachers and parents of their schools. The statistical results were given to the principals as feedback, and principals’ attitudes toward 360 degree evaluation feedback system were investigated.

The above study reached the following conclusions. (1) Vocational high school principals’ leadership effectiveness achieved a high-intermediate level, with rational goal leadership effectiveness being the highest among all. (2) Principals’ self-evaluation of leadership effectiveness scored higher than other-evaluation. (3) The principals highly accepted the use of 360 degree feedback as the reference for their leadership behavior change intentions. (4) Vocational high school principals’ attitudes toward 360 degree evaluation feedback had a direct influence on their leadership behavior change intentions.

This pilot study indicated that most vocational high school principals accepted the use of the 360 degree evaluation feedback system as a mechanism for evaluating leadership and strongly agreed with making it a reference for leadership behavior change intention. This study provides innovative research that expands the scope of 360 degree feedback for the appraisal of school principals’ leadership effectiveness in Taiwan. To extend the findings of a previous study, the current study was designed to investigate the following questions.

First, does the implementation of 360 degree evaluation feedback help improve vocational high school principals’ leadership effectiveness? The researcher adopts the statistics in the previous study as the pretest and the follow-up research conducted in this study as the post-test to examine differences in principals’ leadership effectiveness before and after receiving 360 degree evaluation feedback.

Second, what are vocational high school principals’ perceptions and acceptance level of 360 degree feedback? Can strong leadership behavior change intentions truly prompt positive leading performance and elevate leadership effectiveness? For this aspect, principals were subjected to an interview survey.

The 360 degree feedback system, a multiple appraisal method, can reflect the diversity and complexity of current leadership (London and Smither 1995). The 360 degree feedback has special functions and benefits in terms of organizational goal fulfillment, outcome evaluation, member development, and leadership effectiveness (Fleenor et al. 2008; Karkoulian et al. 2016; Markham et al. 2017). The 360 degree feedback is regarded with increasing attention and even explored with the use of big data (Markham et al. 2015; Brown et al. 2017). Through 360 degree feedback, evaluatees can acquire multidimensional evaluation feedback from their and others’ angles, and even more precisely comprehend the abilities they are supposed to improve (Edwards and Ewen 1996). When organizations implement the 360 degree feedback system, the ultimate goal is to encourage leaders to set goals and further improve their effectiveness (Deshpande et al. 2015). Consequently, as indicated in the statement by London and Smither (2002), behavior change is not necessarily achieved solely with only 360 degree feedback; individual attitudes toward the feedback and the holistic acceptance level are key factors in behavior change, that is, 360 degree evaluation feedback can offer vocational high school principals a more diverse understanding of leadership effectiveness, but the key to implementation is principals’ viewpoints and acceptance level of this system.

Hence, it is necessary to further understand and explore principals’ opinions regarding 360 degree leadership effectiveness evaluation feedback.

Principal leadership behavior has been demonstrated to be a key element of the success of education. A school with good performance definitely has a principal with good leadership effectiveness. Principals’ personalities and leadership behaviors are crucial factors in leadership effectiveness (Dobewall et al. 2013; Hirschmüller et al. 2013; McKee et al. 2015; Bergner et al. 2016). Not all principals’ leadership abilities are natural gifts, but many can be improved by learning and receiving multiple information and feedback (McKee et al. 2015; Bergner et al. 2016). In a pilot study (Cheng 2012), it was revealed that after receiving evaluation feedback, most vocational high school principals could face the results comfortably, contemplate improving the evaluation results, and generate strong leadership behavior change intentions. However, whether this change in intention transforms into an actual change in leadership behavior has remained unknown. Many research findings (Amundsen and Martinsen 2014; Day et al. 2014; Day and Dragoni 2015) suggest that if leaders are willing to accept 360 degree feedback and improve leadership behavior, leadership effectiveness can be improved. In addition to leadership behaviors of leaders, even attitudes of organization members have changed 1 year after the intervention of 360 degree (Deshpande et al. 2015; Karkoulian et al. 2016; Markham et al. 2017). Nevertheless, no relevant empirical study has been conducted in Taiwan. Hence, based on the findings of the pilot study, the current study provides a further analysis of vocational high school principals’ opinions of the 360 degree evaluation feedback system, and it traces and analyzes their changes in leadership effectiveness.

The purposes of the study are as follows:

  1. (1)

    To analyze the current situation of vocational high school principals’ leadership effectiveness and the differences in their leadership effectiveness before and after implementing 360 degree leadership effectiveness evaluation feedback.

  2. (2)

    To analyze the relationship between vocational high school principals’ leadership behavior change intentions and leadership effectiveness.

  3. (3)

    To analyze vocational high school principals’ viewpoints of using 360 degree evaluation feedback as leadership behavior change intentions.

  4. (4)

    To analyze vocational high school principals’ perceptions of the influence of 360 degree evaluation feedback on their leadership effectiveness.

  5. (5)

    To propose suggestions based on the results of this study as a reference for future academic studies on improving principals’ leadership effectiveness or implementing 360 degree evaluation feedback, and for applications in principal leadership practice.

Literature review

Principals’ leadership and evaluation

Leadership is a process by which a person influences others to accomplish an objective and directs the organization in a way that makes it more cohesive. Regarding the development of the school, as well as students’ learning achievements, which are also deeply influenced by school principals’ leadership styles and behavior (Engin et al. 2015), it can be seen that the influence of the principal’s leadership on the school’s effectiveness is deep and far-reaching, and school principals can play a very important leadership roles during school development and affect students’ learning achievement (Orr and Orphanos 2011; Alammar 2015). There are roughly three ways to evaluate the leadership effectiveness of vocational school principals in Taiwan. (1) Each year, in evaluation of principals by the government office of educational administration, principals with excellent performance can obtain a one to two-month bonus. (2) Vocational high schools receive the external evaluation every four to five years (external evaluation, university professors may be appointed as the evaluation committee). (3) The principals are assigned to other schools or retain the post of principal in the original school before expiration of the term of office for four years or the sixth, seventh or eighth year of service at the school. They should accept the school performance assessments (external evaluation, evaluation committees including scholars, principals with excellent academic performance, and representatives from the government office of educational administration, teacher organization, and parent associations) (Cheng 2012). The composition of the principal’s performance assessment committee members has almost the spirit of the 360 degree evaluation; however, in fact producing the assessment committees (not the teacher or parent from that school) and the running time for assessment (4–5 years) is irregular. A gap still exists in the execution of the 360 degree evaluation.

Basic concept of the competing values framework

The competing values framework (CVF) is a meta-theory, and its thinking model is based on the existing contradictory and paradoxical phenomena in the human mind (Quinn and Rohrbaugh 1983; Cheng and Wu 2009). This framework was derived from the integration by Quinn and Rohrbaugh (1983) of organizational theorists’ and researchers’ opinions on effectiveness indicators. They proposed a multidimensional cognitive system, including organizational focus on “micro internal or macro external” and emphasis on “stability or flexibility.” The four quadrants were structured through two values integrating four models for organizational analysis. This framework is shown in Fig. 1 (Quinn and McGrath 1985):

Fig. 1
figure 1

Organization theory model of competing values framework. Source Quinn and McGrath (1985, p. 324)

Full size image
  1. (1)

    Rational goal model In the lower-right quadrant, this model emphasizes task-based leadership effectiveness. Its organizational values are competitiveness and productivity.

  2. (2)

    Open systems model In the upper-right quadrant, this model emphasizes adaption-based leadership effectiveness. The main goals include adaption, growth, external support and resource acquisition, stressing flexibility and external focus, such as innovation and creativity.

  3. (3)

    Human relations model In the upper-left quadrant, this model emphasizes the leadership effectiveness of trust and belonging. The main goals are cohesion, participation, morale and communication, stressing flexibility and internal focus.

  4. (4)

    Internal process model In the lower-left quadrant, this model emphasizes the leadership effectiveness of stability and control. The individual is assigned a certain role and is expected to act by the rules.

Measure dimension of competing values leadership effectiveness

With different focuses by researchers, the definitions of leadership effectiveness also vary. This study, on the basis of CVF, defines leadership effectiveness as the process and results by which leaders apply their leadership strategies to achieve organizational ends and improve organizational competitive advantages (rational goal), emphasize organizational innovation and adjustment and respond to external needs (open systems), raise organization members’ satisfaction and human resource development (human relations), and maintain organizations’ stable operation and good internal integration (internal process).

Hooijberg and Denison (2002) presented four dimensions and 12 subdimensions of leadership effectiveness based on CVF, including the mission (defining direction, defining goals, creating shared vision), adaptability (creating change, emphasizing customer focus, promoting organizational learning), involvement (empowering, building team orientation, developing organizational capability), and consistency (defining core values, working hard to reach agreement, coordinating and integrating). Lynch (2001) also focused on CVF and generalized an indicator system to measure leadership effectiveness according to the required abilities of every leadership role. The eight abilities of leadership roles are establishing goals, productivity, adaption, gaining resources, solving conflicts, human resources development, stability, and monitoring assessment, which are further divided into 16 leadership effectiveness indicators.

To effectively and completely construct the indicators of vocational high school principals’ leadership effectiveness, Cheng’s (2012) study, based on the four organizational models of CVF, constructed “principals’ competing values leadership effectiveness indicators,” divided into 4 major domains and 12 subdimensions, including rational goal (confirming direction, rational promotion, goal accomplishment), open systems (external relationship, innovation transcendence, change adaptation), human relations (taking advice widely, harmonious consensus, satisfied morale), and internal process (internal integration, work assignment, evaluation, and feedback). In summary, the research related to the construction of leadership effectiveness indicators based on the theoretical basis of CVF is listed in Table 1.

Table 1 CVF leadership effectiveness-related research.
Full size table

Definition and advantages of 360 degree evaluation feedback

The 360 degree feedback provides a comprehensive evaluation of target objects, including the individual (self-evaluation), superiors, peers, subordinates and external customers (other evaluators), and supplies feedback to the target objects after evaluation. The basic concept is shown in Fig. 2.

Fig. 2
figure 2

360 degree feedback diagram. Source The researcher’s adaption

Full size image

This feedback method is considered to offer more advantages compared with single-source evaluations, for instance, by providing new angles of judgment for individual behaviors or performances and mitigating the shortages of up-to-down, single-source evaluations (Bracken and Rose 2011). This system has an appreciable impact on both leaders’ and members’ organizational behaviors (Hammerly et al. 2014; Deshpande et al. 2015; Karkoulian et al. 2016; Markham et al. 2017).

Content and influence of effectiveness feedback

Generally, how feedback is perceived and responded to by receivers is the key for evaluation feedback research (Kinicki et al. 2004). The results of evaluation should be offered to members as feedback after their effectiveness is evaluated and to help them advance. However, can the feedback encourage members to produce positive behavior change? While it has been demonstrated in many studies (e.g., Fleenor et al. 2008; Tosti and Addison 2009; Gumustekin et al. 2010), the key is whether evaluatees can perceive the accuracy of evaluation because when feedback is provided from reliable sources, members will have more positive reactions (Brett and Atwater 2001).

In the evaluation of 360 degree feedback, the attitudes of members in the organization toward the evaluation system represent a critical factor. The receivers of 360 degree feedback may dedicate themselves to behavior change, so the feedback must be directly connected to receivers’ development plans (Liviu et al. 2009; Mishra 2014). The research by Atwater and Brett (2005) indicated that when evaluatees have higher acceptance of the evaluation system and results, they are more inclined to use the evaluation results. Furthermore, one of the main purposes for organizations to conduct effectiveness evaluation feedback is to provide feedback to members and make members realize their strengths and weaknesses through feedback from various different angles (O’Keefe 2018).

Effectiveness feedback and behavior change intention

Behavior change intention means that the individual desires to become better and closer to the status that should be or is expected to achieve (Taylor and Todd 1995). Behavior change intention may be positive or defensive. The correctness of feedback not only makes evaluatees perceive useful relevant information provided in feedback but also affects evaluatees’ reaction of developing intentions. If evaluatees believe the feedback is useful, an intention of change will be produced (Folger 1987). Support through reliable measures and the establishment of continuous behavior change are the foundations for effectiveness feedback to perform the expected functions (Bracken and Rose 2011).

Evaluatees’ perceptions of attitudes toward evaluation feedback (including acceptance and fairness) are closely connected to their subsequent leadership behavior change intentions. Similar findings have been obtained in many studies (Dobewall et al. 2013; Braddy et al. 2014; McKee et al. 2015; Bergner et al. 2016).

When effectiveness feedback is offered to members, they will compare the evaluation outcome given to them by other people with their own evaluation outcome. In this comparative process, the issue of discrepancy between self-evaluation and other-evaluation emerges. The difference can be divided into four types: overestimate, consistency/high, consistency/low, and underestimate, which can have varied degrees of influence on individuals, organizations and human resources leadership (Atwater and Yammarino 1997), as shown in Fig. 3.

Fig. 3
figure 3

Influence process model of consistency of self- and other-evaluation. Source Atwater and Yammarino (1997, p. 37)

Full size image

The research findings by London and Smither (1995) suggest that individuals will strongly question the accuracy of the feedback results if they differ from their self-evaluation; however, evaluatees will perceive the accuracy of evaluation if feedback results from multiple evaluation sources are consistent. The research by Brett and Atwater (2001) also demonstrated that higher evaluation scores given by evaluatees themselves than by their supervisors and subordinates lead to evaluatees’ perception that the evaluation from their supervisors and subordinates is less accurate.

Correlation analysis of leadership behavior change intentions and leadership effectiveness

Behavioral intention has been shown to have a close connection with actual behavior (Venkatesh and Agarwal 2006). Sheeran’s (2002) meta-analysis also discloses a strong correlation between intention and behavior. To predict a person’s behavior, behavioral intention is the key indicator; that is, behavioral intention is a must process in any behavior (Ajzen and Driver 1991). Research findings by Armitage and Conner (2001) and Trafimow et al. (2002) show that intention and perceived behavioral control can explain 20 to 30% of behavior. Ajzen (1991) claims that to make an accurate prediction about behavior, intention has to remain stable with time until the behavior is performed. In many studies, the stability of intention is described as a prominent factor that may account for the inconsistency between intention and behavior (Sheeran 2002).

Literature pertaining to the connection between behavior change intention and actual behavior mostly adopts a cross-sectional or retrospective approach via statistical methods of correlation or regression analysis to yield research results. The interview method is also certainly a fundamental way to obtain data. Regarding the correlation of principal leadership behavior change intentions and leadership effectiveness, due to a lack of literature for discussion, the current study is conducted to provide an understanding of the connections between these variables. This research is of importance and value for both theoretical verification and empirical application.

Methods

Sample and data collection methods

The population comprises 69 principals in the pilot study. Excluding 29 principals who have retired or transferred from original schools, this study uses 40 principals in public or private vocational high schools as research subjects. Each school samples the principal, the school administrative directors, chiefs or homeroom teachers, and parent representatives, while those who serve at the school less than 2 years are excluded from the research. Sampling in each school ranges from 15 to 30 people based on the scale of the school. A total of 1030 questionnaires are mailed, 876 copies sent back, and 751 copies found to be valid, with a response rate 72.9%. The sampled number in every district is shown in Table 2.

Table 2 The number of vocational high schools sampled in districts
Full size table

From the 40 principals in the questionnaire survey, the interview survey samples 12 principals (7 from public schools and 5 from private schools) based on the district. After an explanation of the research purpose, procedure, method and confidential agreement and signing of the permission and consent forms by the principals, the principals receive a face-to-face interview (9 principals) or telephone interview (3 principals), depending on their preference. Each interview lasts one and a half hour to two hours. The interview survey was conducted from November 15th to December 23rd in 2013. Table 3 displays the details of the interview sample.

Table 3 Interview subjects
Full size table

Measures

Questionnaire of the current situation of principals’ leadership effectiveness

The present study utilizes the population of the principals surveyed in the pilot study “A Study on the Constructions of Vocational High School Principals’ Competing Values Leadership Effectiveness Indicators, and Their Relationships with Behavior Change Intention Based on 360 Degree Evaluation System”. The principals who remain unchanged in position represent the research range, and relevant people are sampled. A questionnaire survey is conducted in each school to investigate the perceptions of the actual performances of principals’ leadership effectiveness after implementing 360 degree leadership effectiveness evaluation feedback. Then, the data are compared with the statistics from the pilot study to analyze the relationship between the perceptions, leadership behavior change intentions and leadership effectiveness performance.

Interview of principals

After the questionnaire survey, 12 principals are selected from the 40 schools and interviewed after their permission. Through the interview method, the aim is to understand principals’ opinions of using 360 degree feedback results as their leadership behavior change intentions and the effects on their leadership effectiveness before and after implementing 360 degree evaluation feedback. The interview is semi-open to acquire in-depth research data. The interview outline is also available provided to the principals to read while they agree to participate. The following is the interview framework:

  1. (1)

    What do you think of using the evaluation results of 360 degree feedback system as leadership behavior change intentions?

  2. (2)

    What is your opinion on the influence of carrying out 360 degree leadership effectiveness evaluation feedback on leadership effectiveness?

  3. (3)

    How do you think of using 360 degree feedback as principals’ effectiveness evaluation?

Data analysis

The present study is revised from “The Questionnaire of Competing Values Leadership Effectiveness” constructed in the researcher’s Ministry of Science and Technology research project as a research tool. The internal consistency of the reliability and construct validity are examined to ensure the reliability and validity of the questionnaire used in the study; moreover, factor analysis is also conducted to identify conceptually linked factor scales in the questionnaire.

The current situation of principals’ leadership effectiveness is analyzed with descriptive and inferential statistics. Pearson correlation and one-way ANOVA are used to analyze the relationship between principals’ leadership behavior change intentions and their leadership effectiveness and the difference in the performance of principals’ leadership effectiveness before and after implementing 360 degree evaluation feedback.

The interviewed principals express their opinions by the outline of the interview, which is recorded after obtaining the principals’ consent. The interview data are transcribed into written scripts, proofread in multiple ways to increase validity, confirmed by the interviewed principals, and then analyzed inductively by the attribute of the question. Two scholars are invited to verify the transcripts of the original interview content and the integrated drafts. Numeral coding of the interview data is shown in Table 3. For example, “001-1126-F” represents the content of the face-to-face interview with the principal numbered 001 on November 26, and “006-1115-T” represents the content of the telephone interview with the principal numbered 006 on November 15.

Results and discussion

Reliability and validity

In the determination of the reliability for the four subscales of leadership effectiveness in this study, items with an inter-item correlation coefficient of < .5 are deleted from each subscale. In the subscale of rational goal, the inter-item correlation coefficients are between .594 and .796; in the subscales of open systems, human relations, and internal process, the inter-item correlation coefficients are between .619 and .799, .714 and .857, .689, and .817, respectively; all items are found to be internally consistent within the four subscales.

Furthermore, Table 4 presents the Item-total correlation and Cronbach’s alpha if an Item is deleted for the subscale of the rational goal, open systems, human relations, and internal process. Item-total statistics present the value that Cronbach’s alpha would be if that particular item is deleted from the scale. Analysis of the internal consistency for the 24 items reveals an overall Cronbach’s alpha of .981, indicating good reliability. It should also be noted that while a high value for Cronbach’s alpha indicates good internal consistency of the items in the scale, it does not mean that the scale is unidimensional.

Table 4 Item-total correlation and Cronbach’s alpha if item is deleted for the subscales of rational goal, open systems, human relations, and internal process (N = 751)
Full size table

Factor analysis is a method to determine the dimensionality of a scale, that is, factor analysis can help determine the construct validity of the measurement. Factor loadings and cumulative explained variation of the topics in each subscale are shown in Table 5. From the evidence data, it is obvious that the research tool has good validity and high internal consistency.

Table 5 Factor analysis and reliability analysis of the questionnaire on vocational high school principal leadership effectiveness (N = 751)
Full size table

Analysis of the current situation of school principals’ leadership effectiveness

The analysis of the current situation of vocational high school principals’ leadership effectiveness is shown in Table 6. In overall leadership effectiveness, the average score of the perceptions of school educational staff and parents, hereafter referred to as school members, toward principals’ leadership effectiveness is 5.007, indicative of “mostly meet” on a 6-point scale and reaching a high-intermediate level. Every dimension is tested by repeated measures ANOVA, and all four domains of principals’ leadership effectiveness achieve a significant difference (F = 44.246, p < .01). The post hoc comparison reveals that the order of the domains from high to low is rational goal, open systems, internal process, and human relations. Thus, school members perceive that principals have the best performance on rational goal leadership effectiveness and the lowest score on human relations leadership effectiveness. This finding is consistent with the research reported by Cheng (2012).

Table 6 The mean and standard deviation of the current situation of vocational high school principals’ leadership effectiveness (N = 751)
Full size table

Analysis of the difference in leadership effectiveness performance before and after implementing 360 degree leadership effectiveness evaluation feedback

Table 7 presents the results of the analysis of the difference in leadership effectiveness performance before and after vocational high school principals receive 360 degree leadership effectiveness evaluation feedback. The analysis discovers significant differences in vocational high school principals’ leadership effectiveness in rational goal, open systems, human relations and internal process, as well as in overall leadership effectiveness, between 2012 and 2013. Confidence intervals are − .3680, − .2008 for rational goal, − .4082, − .2276 for open systems, − .2305, −.0528 for human relations, − .1890, −.0209 for internal process, and − .2941, − .1297 for overall leadership effectiveness. The critical confidence level is between .000 and .014, which is much greater than 5%. In the intensity of leadership effectiveness measure, excluding the small effect size of “human relations” and “internal process”, the effect sizes of the remaining leadership effectiveness belong to the medium intensity measure. Thus, there is a substantial difference in leadership effectiveness before and after vocational high school principals receive 360 degree leadership effectiveness evaluation feedback. All performances of leadership effectiveness are higher after feedback is received, which is consistent with the related research results (Tosti and Addison 2009; Gumustekin et al. 2010; Bracken and Rose 2011; Amundsen and Martinsen 2014; Day and Dragoni 2015). To position these findings in another way, after implementing 360 degree leadership effectiveness evaluation feedback, vocational high school principals will produce behavior change intentions and change their leadership behavior to improve leadership effectiveness.

Table 7 The difference of vocational high school principals’ leadership effectiveness between before and after 360 degree leadership effectiveness evaluation feedback (N = 751)
Full size table

Analysis of differences in principals’ leadership behavior change intentions and leadership effectiveness

This study divides vocational high school principals’ behavior change intentions into a high score group and low score group based on the mean score and analyzes their differences in leadership effectiveness. The statistical analysis points out significant differences between the two groups in rational goal (F = 6.649, p = .014 < .05), open systems (F = 5.645, p = .023 < .05), and overall leadership effectiveness (F = 5.221, p = 0.028 < .05) (see Table 8). Principals with high behavior change intentions are superior to principals with low behavior change intentions in rational goal, open systems, and overall leadership effectiveness. This result is consistent with several research findings (Day et al. 2014; Karkoulian et al. 2016): leadership effectiveness performances will vary with the degree of leadership behavior change intentions. Nonetheless, the strength of the association (ω2) between the above two domains of leadership effectiveness and overall leadership effectiveness are .149, .129, and .121, which belong to the medium to high level; the statistical powers are .710, .639, and .605, which do not meet the basic standard. Thus, the results should be interpreted with caution.

Table 8 The difference in every domain of vocational high school principals’ leadership effectiveness between high and low behavior change intentions
Full size table

Analysis of principals’ viewpoints concerning the use of 360 degree evaluation feedback as leadership behavior change intentions

The 360 degree feedback can be closely associated with vocational high school principals’ leadership effectiveness, as shown in this study and in relevant literature (Bracken and Rose 2011; Cheng 2012; Day and Dragoni 2015). Principals’ opinions regarding this issue and their basic supporting points are compiled and listed below:

360 degree evaluation feedback can provide a clear mirror to assist principals in personally understanding the blocks of which they are not aware

The 360 degree evaluation is an assessment system itself, which can serve as references for bettering one’s leadership behavior through multiple information sources. Some principals point out the following:

With a complete evaluation system, through a complete course, from impartial third parties, the results are always more convincing. Moreover, through the guide of a benevolent and clear mirror, leadership behavior can be demonstrated more effectively (006-1115-T).

Significant differences are found before and after implementation by the principals of 360 degree evaluation feedback, and leadership effectiveness after implementation is significantly improved, as described by the following two interviewed principals:


I can highly accept the 360 degree principal leadership effectiveness evaluation system because normally no one would give you feedback…. 360 degree evaluation feedback is a very comprehensive leadership effectiveness evaluation method. It is beneficial for the development of principals’ effectiveness…. This kind of evaluation can help principals see their blind spots and make improvements from thinking to behavior. (002-1211-F)


If someone can point out my fault… then that is a mirror. If the fault does exist, correct it. If not, just encourage yourself. As for me, I will take it as advice for introspection and correction. We are unable to see some parts of ourselves clearly on our own, and sometimes we are too self-complacent. Whatever the situation, as a leader, we should correct it if there is any fault; if there is not, cheer ourselves on. This is the demeanor a leader should minimally possess (005-1119-F).

360 degree evaluation feedback can enhance the transparency of principals’ decision making concerning school affairs from the perspective of different roles

Some principals believe that the openness and transparency of 360 degree feedback and the multidimensional collection of information can help leaders make better decisions.

If principals only manage schools based on their own opinions or perspectives, the decisions may not get supported. The 360 degree evaluation can integrate others’ opinions… so [school management] can be connected with the 360 degree evaluation (012-1217-T).

With the popularity of post-modernism, “decentralization” has also become the mainstream of power operation in current school organizations. The standard of school affairs operation and methods has been transformed from the “principal system” to the “committee system”. Through 360 degree evaluation feedback, information from different sources would be provided to the principal as a reference for school decision making. The interviewed principal stated the following:

The promotion of school affairs is to pool the wisdom and efforts of everyone…. Generally speaking, the questions raised by the first-line executors are the most direct questions…. Through the evaluation results, we adjust our own decisions and take into account the feedback from colleagues, and quality decisions can be made (004-1126-T and 010-1202-F).

360 degree evaluation feedback can assist in pooling everyone’s efforts toward school visions

The evaluation results can be given to evaluated leaders as feedback. The leaders can realize their own strengths and weaknesses through the evaluation results. The principal is the “pilot” of the school. School visions and goals must be specific and feasible. The 360 degree feedback has a great impact on strengthening schools’ visions and values (Bracken and Rose 2011), as indicated by one principal as follows:

My behavior will be affected by parents’ and colleagues’ satisfaction at present. Moreover, being an ideal leader, I also pay attention to the children… whether the children still have the ability to dream. In leading the whole school team, we should manage to not only satisfy the current situation but also fulfill the vision. So speaking of behavior change, maybe it is fine to maintain the superficial peacefulness for now, but if we want to advance actively, we need to take into consideration students’ ability to dream and the ability of the whole school to achieve the vision… (001-1126-F).

Thus, it is believed that leaders can meet schools’ needs more if they utilize feedback from evaluation results, clarify the future blueprint and adjust their leadership behavior.

It means reflecting on our own actions and behavior. The feedback from others can cause retrospection and self-review. It is like a mirror though which one can correct oneself. In particular, all of us working at the educational site will affect the future direction of the school (003-1223-F).

360 degree evaluation feedback can strengthen interactive mechanism and inspire the mutual growth of principals and members

Gumustekin et al. (2010) deemed that a deeper mutual trust system is built during the process of evaluation. Even a culture can be formed after the system is constructed. Mutual growth through the interaction is also an important outcome of 360 degree feedback process. As stated by one principal,

Actually, I do this sometimes. That is, I usually take some survey and ask my colleagues to fill out the forms. Then, I will make some adjustments based on the suggestions from my colleagues or peers. I believe this is beneficial for the promotion of the school. In fact, a principal’s duty is the operation of the whole school…. Therefore, I can accept this, which I believe is a great means (006-1115-T).

Another principal (011-1203-F) narrated the following:

The so-called leading must comprise a team of two or more people, indicating interpersonal interactions. There must be some communication and negotiation in the interaction. The so-called “altitude of the principal” doesn’t mean an altitude of principal’s self-awareness, but principals need to take “the future development of the school as altitude” and integrate and balance opinions from everywhere with holistic and macroscopic leadership behavior. It is by doing so that “a real team” can be formed and “leading” can have its existing value.

If the leader is willing to be the first to show good intentions, it is believed that those who are led will also respond with more opportunities for interactive growth.

360 degree evaluation feedback can catalyze the development of leadership and help principals improve their leadership behavior, and it is conducive to the principal’s own leadership behavior for self-reflection

The development of the school cannot be shouldered solely by the leader. It should be able to be built and passed on. A more proactive perspective is required to gain sustainable development of the school, as indicated in the following citations expatiated by the principals.

The development of leadership effectiveness and evaluation should complement each other. Leading without evaluation is “blind” because leaders have no way to know if they are on the right path of leading. With the help of evaluation feedback, leaders should maintain the good parts in the feedback results and introspect about those needing improvement. Leadership behavior is supposed to be fully believed by people who are led. In this way, it can be called the display of leadership effectiveness (012-1217-T).

The competent authority of public schools is the educational administrative agency, and the competent authority of private schools is not only the educational administrative agency but also the board of directors. Therefore, the Board’s opinion on the principal’s school management is also important evaluation information.

Leadership behavior depends on the conditions of the organization, including the board of directors. The board of directors sets their limits. I will try my best to combine the needs of all stakeholders and achieve the highest effectiveness. It is essential to form the best atmosphere and culture of the organization (009-1211-F).

Carry out the evaluation from broader and different angles and figure out the problems. There is always space for me to improve in managing the school…. Basically, I think evaluation from different angles is better than self-evaluation (011-1203-F).

Therefore, 360 degree evaluation feedback supplies a benefit in promoting leadership change and improving leadership effectiveness.

Subjectively, the errors present in the evaluation lead to principals’ doubts about implementing the 360-degree evaluation

Only when evaluatees consider the feedback provided by reliable sources can more positive reactions from members be effectively activated (Brett and Atwater 2001).The opposition of principals with a conservative attitude mainly lies in doubt of the evaluators and the process. They assume that there are subjective errors in the 360 degree evaluation (004-1126-T, 005-1119-F, 008-1119-F, 011-1203-F). One principal argued the following:

I will hold a conservative attitude if I need to adjust my leadership effectiveness based on the results. It should only for my reference. If the results make me feel I should improve, I will do it, and it will be the basis of the adjustment. Different evaluations provide different results. Everyone differs in views, and evaluators have different subjective ideas. I will make judgement on my own. If the opinion they provide is something the school needs to adjust, we will make the adjustment at once (004-1126-T).

There is no denying that evaluators possess a certain degree of subjectivity, but compared with a single-source evaluation, multiple-source evaluation is also a means and a process of pursuing “true fairness.” As conservatives (005-1119-F for example) express that they are willing to make corrections if the opinions from others make sense, this view suggests that the 360 degree evaluation still has value and feasibility; however, by demonstrating to the principals that the feedback is objective and fair, their acceptance will be increased.

Analysis of principals’ perceptions of the influence of 360 degree evaluation feedback on their leadership effectiveness

Some related research (Cheng 2012; Markham et al. 2017) indicates that 360 degree evaluation feedback has a close relationship with leadership effectiveness, but the premise lies in the perception of 360 degree evaluation feedback (Karkoulian et al. 2016). Vocational high school principals’ opinions on this issue are summarized below.

The acceptance of the 360 degree evaluation feedback system is an evaluation method of high fairness

For principal’s leadership effectiveness evaluation, most principals can accept the objectivity and fairness of the feedback results provided by 360 degree evaluation for leaders (001-1126-F, 002-1211-F, 004-1126-T, 005-1119-F, 006-1115-T, 007-1203-F, 009-1211-F, 010-1202-F, 011-1203-F). Only when leaders believe the evaluation tool can they have faith in the results. Only when evaluation results are accepted can their functionality exist. For leaders who have achieved a certain altitude in their position, only trust and faith can bring about the motive for change (Latham and Wexley 1994). In particular, many people observe principals’ leadership from different angles, but they all bring forward the same opinion. Its fairness and impartiality have greatly improved. For example,

The proverb goes, “A repeated rumor makes others believe it.” When only one or two people tell us there is something that needs improvement in our leadership model, we will doubt it at first. However, when a third or a fourth person says the same thing, the one who receives the information will begin to introspect and ponder that maybe the issue everybody is mentioning does exist (002-1211-F).

In fact, the foundation of the 360 degree evaluation is crowd power, diffusing errors caused by single or a few evaluators. The application of a multiple-angle and multisource evaluation method will, in other words, produce a kind of equality similar to average. The concept of average is an important one in logics, and its greatest advantage is to reduce error.

Principals generally believe that 360 degree feedback results can improve leadership behavior and even supply answers to leadership-related issues

In the interview as well as the question survey, 9 of the 12 interviewees (001-1126-F, 002-1211-F, 004-1126-T, 005-1119-F, 006-1115-T, 008-1119-F, 010-1202-F, 011-1203-F and 012-1217-T) believe their own leadership behaviors have improved after receiving the complete 360 degree evaluation. Rosti and Shipper (1998) indicate that feedback results certainly contain evaluators’ expectations to leaders, as well as the unsatisfied parts of all stakeholders. The so-called display of leadership effectiveness is that leaders manage to seek balance between exerting leadership desires and meeting the needs of those being led during the practice of leading (London and Beatty 1993; Murphy and Cleveland 1995).

Much information related to leadership can be acquired from the feedback results of the 360 degree evaluation (Edwards and Ewen 1996; Bracken and Rose 2011; Karkoulian et al. 2016). As several principals impart, they have obtained some answers to the following questions through 360 degree feedback:

What aspects can those who are led approve (008-1119-F)?; what are their expectations to leaders in the workplace (004-1126-T, 007-1203-F, 012-1217-T)?; and which parts do those who are led hope to be satisfied (004-1126-T and 009-1211-F)? If leaders can be acquainted with the information regarding these dimensions and make good use of it, it will be easy to determine where to exert force (001-1126-F, 003-1223-F and 008-1119-F).

One principal (011-1203-F) added the following:

Archimedes once said, “Give me a place to stand on, and I will move the Earth.” Whether the force can be exerted well or not, the force point is very important. Find the right point, and you can do half the job. The 360 degree evaluation feedback is a tool that works in this way. The 360 degree evaluation feedback can be considered a good management tool.

If the principal can make good use of the 360-degree feedback information as a reference for improving leadership behavior, then the gap between “expectation” and “will” for members and leaders will be reduced.

The evaluators of 360 degree feedback include internal and external stakeholders, and the 360 degree feedback results can be applied to assist the development of individuals and schools

This study found that principals’ leadership performance is still the highest on the “rational goal” and lowest on “human relations”. In competing values framework, the above two items are in conflicting positions (refer to Fig. 1). The 360 degree evaluation feedback can be used to improve this phenomenon. The interviewed principals also thought that the 360 degree evaluation collects opinions not only outside schools but inside, including the voice of parents and teachers (001-1126-F, 003-1223-F, 004-1126-T, 005-1119-F, 006-1115-T). Consequently, it can strengthen the performance management of the administrative operation (Cipolla 2009). However, principals must focus on not only the achievement of the instrumental goals of schools, such as progression to school, enrollment, and certification but also the satisfaction of member needs, because becoming a principal with high leadership effectiveness requires both “internal and external” and “flexibility and control”. Additionally, an open mind and acceptance of change are also necessary. The interview principal stated the following:

The school environment has transformed from a closed to an open system… Once it becomes open, you cannot go back to closed. Since transparent inspection is an inevitable trend, why not make the crisis a turning point? Take advantage of 360 degree feedback to collect opinions outside and inside the school and turn it into an aiding force for the promotion of school affairs (003-1223-F).

In this study, the principals need to pay attention to leadership effectiveness performance of “human relations”, which is the worst, because most of the principals spent a large amount of time on solving personnel problems. The interview principal (005-1119-F) stated the following:

Discarding the in-school model and including the evaluation forces outside school makes the whole process “more serious.” Besides, school-leaders, or principals, spend most of their school management time coordinating human resources inside the school; it is beyond their compass to hear the voice outside the school.

Evaluation in the past has mainly focused on the opinions inside schools. Although outside committees come into schools, they have to reach the conclusions of all evaluations within one day. It is inevitable that the evaluation results are difficult to accept (002–1211-F, 04-1126-T, 010-1202-F). Some principals indicated the following:

Such a more just and objective access as the 360 degree evaluation does not provide hearsay from unidentified sources but information presented by evaluators sampled from those related to schools. Only those who really care about the school will pay attention to the conditions of the school on normal days and give advice to the school (011-1203-F).

Although the 360 degree evaluation method can extend evaluators to those involved in all aspects, which should be helpful for principal and school development, principals’ personal attitude is a very important factor.

The proverb goes, “When we see a man of virtue and talent, we should think of equaling them; when we see a man of a contrary character, we should turn inwards and examine ourselves.” The great Mencius introspected himself on three points every day. Although principals are busy managing the school and handling a host of issues, introspection can never be omitted (001-1126-F).

This additional benevolent eye can reduce principals’ self-interested bias and demonstrate important leadership effectiveness

Some principals regard 360 degree evaluation feedback as assistance in seeing their blind spots (002-1211-F and 005-1119-F), which corresponds to several research findings (Cacioppe and Albrecht 2000; O’Keefe 2018). Like Gardner’s theory of multiple intelligences, there are multiple modalities of human intelligence, each of which differs in strength. The same point is made by principals:

There are diverse aspects that need to be taken into account in leadership, and every leader has their own strengths and weaknesses. In the process of promoting school affairs, it is inevitable that leaders have some blind spots because of their own “inertia in leadership” (012-1217-T).

The 360 degree evaluation feedback is like a health check that can provide diverse physical health information through different categories and perspectives. Although some information is previously unknown or unconfirmed, it can also be used as an important source of information to clarify ourselves. One principal stated as the following (006-1115-T):

If 360 degree evaluation feedback can be made good use of, principals should be able to accept this benevolent eye. It can help principals see the blind spots in their school management. After all, this is not the observations of a few people of a few aspects but a comprehensive consideration. It is believed that this kind of consideration can substantially reduce leaders’ blind spots.

Contributions and recommendations

This study first revealed that educational staff and students’ parents of vocational high schools perceive that principals generally show a high-intermediate level of leadership effectiveness in rational goal, open systems, human relations and internal process. Overall, rational goal leadership effectiveness has the best performance. In other words, vocational high school principals emphasize task-oriented leadership in their leadership effectiveness performance. Second, after 360 leadership effectiveness evaluation feedback is carried out among vocational high school principals, their leadership effectiveness performance is noticeably higher than before. Thus, it is evident that 360 degree evaluation is an applicable multisource evaluation method, which can avoid personal subjectivity and can have positive effects on principals’ leadership effectiveness. Third, vocational high school principals who display high behavior change intentions in overall leadership effectiveness, rational goal, open systems, and others show better leadership effectiveness than those with lower intentions. This finding also clarifies that principals with high behavior change intentions tend to care about others’ opinions or comments about themselves and change their leadership behavior in an endeavor to demonstrate better leading effectiveness. Fourth, the interview survey reveals that principals use 360 degree evaluation feedback results as an important source of leadership behavior change intentions because they believe it has the benefit of acting as a mirror to assist individuals and schools with high-quality development, enhancing the transparency of school decisions through different perspectives of roles, building more proper visions and pooling everyone’s efforts toward school visions, strengthening the interactive mechanism and inspiring the mutual growth of principals and members, and catalyzing the development of leadership and helping principals improve their leadership behavior and effectiveness. However, a small number of principals hold conservative attitudes toward the system because they doubt the subjectivity of the evaluation and believe there are subjective errors of evaluators. Fifth, based on the interview with principals, it can be discerned that most vocational high school principals positively perceive and supportively the influence of implementing 360 degree leadership effectiveness evaluation feedback for leadership effectiveness. This mechanism is recognized as an evaluation method with a high degree of fairness; the feedback results are commonly believed to improve leadership behavior; the feedback results can be applied to assist the development of individuals and schools; and this additional benevolent eye can aid in detecting one’s blind spots in management.

According to the findings of this study, some suggestions are provided as a reference. The first suggestion is that education administration authorities may confer regarding the feasibility of adopting the 360 degree multiple feedback method. However, the education field continues to lack a sufficient understanding of 360 degree evaluation feedback. It is advised that seminars be held relating to 360 degree leadership effectiveness evaluation feedback and encourage participation by principals and teachers of all-level schools to learn more about the content and application of 360 degree leadership effectiveness. The second suggestion, as indicated in this study, 360 degree leadership effectiveness feedback can offer suggestions and feedback to principals to modify the developmental orientation of self-evaluation and to impact their behavior change intentions. Thus, principals should maintain an open mind to feedback from different evaluators, inspire their own change intentions on leadership behavior and further elevate leadership effectiveness. In addition, the third suggestion is that vocational high school principals’ leadership effectiveness can still be refined, in particular the human relations model, which strengthens trust and belonging and the orientation of participation, coherence, morale and communication. If principals can not only maintain rational goal achievement but also take care of human relations leadership effectiveness in helming schools, they can definitely adapt to the paradoxical and complicated leading environment. Furthermore, how can we establish a consensus on the aspects of the implementation process, the representativeness of the selected evaluators, the number of evaluators and the content of evaluation? How can we refine this system through persistent research to further enhance the functions and effectiveness of implementing this evaluation feedback system? These are goals that future research should strive to address.