Abstract
Performance judgment is a situation of incomplete information where raters' inference would play an important role. Consequently, the schematic nature of human cognition may introduce implicit personality theory bias in performance judgment. To demonstrate this, a causal model of performance rating judgment was framed from the theories of person perception and social cognition. The model yielded a good fit to the data obtained from a performance rating task where the availability of performance information was manipulated. The results supported the hypotheses that student raters' inferences are partly contaminated by their implicit theories of a good instructor. Student raters inferred traits and behaviors and provided ratings for corresponding items even when the instructor behavior was limited to a subset of performance data only. The findings imply that one aspect of invalidity in student ratings of instructors is the bias in human inference due to the implicit theories of effective instructional behavior.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Abrami, P. C., d'Apollonia, S., and Cohen, P. A. (1990). Validity of student ratings of instruction: What we know and what we do not.Journal of Educational Psychology 82(2): 219–231.
Anderson, J. C., and Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach.Psychological Bulletin 103(3): 411–423.
Anderson, N. H. (1981).Foundations of Information Integration Theory. New York: Academic Press.
Bollen, K. A. (1989).Structural Equations with Latent Variables. New York: Wiley.
Breckler, S. J. (1990). Applications of covariance structure modeling in psychology: Cause for concern.Psychological Bulletin 107(2): 260–273.
Cadwell, J., and Jenkins, J. (1985). The effects of semantic similarity of items on student ratings of instructors.Journal of Educational Psychology 77(4): 383–393.
Cantor, N., and Mischel, W. (1979). Prototypes in person perception. In L. Berkowitz (ed.),Advances in Experimental Social Psychology, vol. 12 (pp. 3–52). New York: Academic Press.
Carlson, M., and Muliak, S. (1993). Trait ratings from descriptions of behavior as mediated by components of meaning.Multivariate Behavioral Research 28(1): 111–159.
Cliff, N. (1983). Some cautions concerning the application of causal modeling methods.Multivariate Behavioral Research 18: 115–126.
Cook, S. S. (1989). Improving the quality of student ratings of instruction: A look at two strategies.Research in Higher Education 30(1): 31–45.
Cooper, W. H. (1981). Ubiquitous halo.Psychological Bulletin 90: 218–244.
DiNisi, A. S., Cafferty, T. P., and Meglino, B. M. (1984). A cognitive view of the performance appraisal processes.Organizational Behavior and Human Performance 33: 360–396.
Doyle, K. O., Jr. (1981). Validity and perplexity: An incomplete list of disturbing issues in instructional evaluation research.Instructional Evaluation 6(1): 23–25.
Feldman, K. A. (1989). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers.Research in Higher Education 30(2): 137–194.
Gaski, J. F. (1987). On construct validity of measures of college teaching effectiveness.Journal of Educational Psychology 79(3): 326–330.
Hastie, R. (1981). Schematic principles in human memory. In E. Higgins, C. Herman, and M. Zanna (eds.),Social Cognition: The Ontario Symposium, vol. 1, (pp. 39–88). Hillsdale, NJ: Erlbaum.
Hayduk, L. A. (1987).Structural Equational Modelling with LISREL: Essentials and Advances. Balitmore: The John Hopkins University Press.
Holland, J. H., Holyoak, K. J., Nisbett, R. E., and Thagard, P. R. (1986).Induction: Processes of Inference Learning and Discovery. Cambridge, MA: MIT Press.
Howard, G. S., and Maxwell, S. E. (1982). Do grades contaminate student evaluations of instruction?Research in Higher Education 6: 175–188.
Ilgen, D. R., and Feldman, J. M. (1983). Performance appraisal: A process focus. In B. M. Staw and L. Cummings (eds.),Research in Organizational Behavior, vol. 5 (pp. 141–197). Greenwich, CT: JAI Press.
Jenkins, J. (1987). Implicit theories and semantic similarities: Reply to Marsh and Groves (1987).Journal of Educational Psychology 79(4): 490–493.
Joreskog, K. G., and Sorbom, D. (1989).LISREL 7 User's Reference Guide. Moresville, IN: Scientific Software Inc.
Kishor, N. (1990). The effect of cognitive complexity on halo in performance judgment.Journal of Personnel Evaluation in Education 3: 377–386.
Kishor, N. (1992). Compensatory and noncompensatory information integration and halo error in performance rating judgments.Journal of Personnel Evaluation in Education 5: 257–268.
Krzystofiak, F., Cardy, R., and Newman, J. (1988). Implicit personality and performance appraisal: The influence of trait inferences on evaluations of behavior.Journal of Applied Psychology 73(3): 515–521.
Landy, F. J., and Farr, J. L. (1980). Performance rating.Psychological Bulletin 87: 72–107.
Larson, J. R., Jr. (1979). The limited utility of factor analytic techniques for the study of implicit theories in student ratings of teacher behavior.American Educational Research Journal 16(2): 201–211.
L'Hommedieu, R., Menges, R. J., and Brinko, K. T. (1990). Methodological explanations for the modest effects of feedback from student ratings.Journal of Educational Psychology 82(2): 232–241.
Marsh, H. W. (1984). Students' evaluation of university teaching: Dimensionality, reliability, validity, potential biases, and utility.Journal of Educational Psychology 76(5): 707–754.
Marsh, H. W. (1987). Students' evaluations of university teaching: Research findings, methodological issues, and directions for future research.International Journal of Educational Research 11(3): 253–388.
Marsh, H. W., and Groves, M. A. (1987). Students' evaluation of teaching effectiveness and implicit theories: A critique of Cadwell and Jenkins (1985).Journal of Educational Psychology 79(4): 483–489.
McCallum, L. W. (1984). A meta-analysis of course evaluation data and its use in the tenure decision.Research in Higher Education 21: 150–158.
Medley, D. (1979). The effectiveness of teachers. In P. Paterson and H. Walberg (eds.),Research on Teaching: Concepts, Findings and Implications. Berkeley, CA: McCutchan.
Mook, D. J. (1983). In defense of external validity.American Psychologist 38: 379–387.
Nisbett, R., and Ross, L. (1980).Human Inference: Strategies and Shortcomings of Social Judgment. Englewood Cliffs, NJ: Prentice Hall.
Rotem, A., and Glasman, N. S. (1979). On the effectiveness on students' evaluative feedback to university instructors.Review of Educational Research 49: 497–511.
Simon, H. (1978). Rationality as process and as product of thought.American Economic Review: Papers and Proceedings 68: 1–16.
Sobel, M. E., and Bohmstedt, G. W. (1985). Use of null models in evaluating the fit of covariance structure models. In N. B. Tuma (ed.),Sociological Methodology (pp. 152–178). San Francisco: Jossey-Bass.
Srull, T. K., and Wyer, R. S. (1989). Person memory and judgment.Psychological Review 96: 58–83.
Sulman, L. S. (1986). Paradigms and research programs in the study of teaching: A contemporary perspective. In M. C. Wittrock (ed.).Handbook of Research on Teaching, 3rd ed. (pp. 3–37). New York: Macmillan.
Taylor, S. E., and Crocker, J. (1981). Schematic bases of social information processing. In E. Higgins, C. Herman, and M. Zanna (eds.),Social Cognition: The Ontario Symposium, vol. 1 (pp. 89–134). Hillsdale, NJ: Erlbaum.
Trzebinski, J. (1985). Action-oriented representations of implicit personality theories.Journal of Personality and Social Psychology 48(5): 1266–1278.
Trzebinski, J., McGlynn, R. P., Gray, G., and Tubbs, D. (1985). The role of categories of an actor's goals in organizing inferences about a person.Journal of Personality and Social Psychology 48(6): 1387–1397.
Whitely, S. E., and Doyle, K. O. (1976). Implicit theories in student ratings.American Journal of Educational Research 13(4): 241–253.
Wyer, R. S., and Martin, L. L. (1986). Person memory: The role of traits, group stereotypes and specific behaviors in the cognitive representation of persons.Journal of Personality and Social Psychology 50: 661–675.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kishor, N. The effect of implicit theories on raters' inference in performance judgment: Consequences for the validity of student ratings of instruction. Res High Educ 36, 177–195 (1995). https://doi.org/10.1007/BF02207787
Issue Date:
DOI: https://doi.org/10.1007/BF02207787