saog. New for ve An ‘in, rin, 000 Yo deree satis na of dt A META-ANALYSIS OF SELF—SUPERVISOR, SELF-PEER, AND PEER-SUPERVISOR RATINGS MICHAEL M. HARRIS, JOHN SCHAUBROECK ‘Kranet Graduate School of Management, Prdue University Reviews of self-supervisor, sef-peer, and peer-supervisor ratings have ssenerally concluded that there is at best # modest correlation between sifferent rating sources. Nevertheles, there has been much inconsistency across studies. Accordingly, a meta-analysis was conducted. The resulie indicated a relatively high correlation between peer and supervisor atings (= .62) but only a moderate correlation between self-supervisor (p 35) and self-peer ratings (p= .36). While rating format (dimensional versus global) and rating scale (ait versus behavioral) had litle impact as "moderators, job type (manageril/ professional versus blue- HARRIS AND SCHAUBROECK 49 Journal of Vocational Behavior (1970-1986), Organizational Behavior and Human Decision Processes (Formerly Organizational Behavior and Human Performance) (1966-1986), and Personnel Psychology (1956-1986), Con- ference papers from the Midwest and National proceedings of the Academy of Management were also examined. Computer-based searches were con- ducted on Psyc-Info (Psychological Abstracts) for the period since 1967 and the ABI Inform data base for the period since 1975, References for ‘other studies were obtained from these sources to expand the research base. ‘Three judgments were made conceming the collection and analysis of data. First, only those reliability estimates based on accepted formulas ‘were selected 10 determine the mean reliabilities actoss studies. ‘This ex- cluded, among other estimates, unadjusted average inter-item correlation coefficients. Second, laboratory studies were deliberately excluded from the analysis. This was done because the conditions under which laboratory ratings occur grossly exaggerate the observation time factor. More impor- tantly, assigned roles cannot be taken to reflect true perspectives of super- visors and peers (Ilgen & Favero, 1985). Finally, some studies contained ‘multiple measures of performance. Since these measures were typically not independent, the effect sizes were averaged and the mean effect size was used in the meta-analysis. Results ‘The literature search produced a total of 36 independent self—supervisor correlations, 23 independent peer-supervisor correlations, and 11 indepen- dent self-peer correlations (approximately 5 studies did not provide either ‘means or correlations). Table 1 shows the average correlations, reliabil ties, and range restriction for the three rater combinations. The reliabilities are based on intemal consistency (alpha) measures. It is noteworthy that the average reliabilities are substantially higher than the .60 figure used by ‘Schmidt and his colleagues (e.g., see Pearlman, Schmit, & Hunter, 1980). ‘There are three reasons for using the more conservative figures found here. First, Schmidt and his colleagues based their reliability estimates on the correlations between ratings from different supervisors at different points in time. In the present study, only intemal consistency measures. were found. Second, the .60 figure used by other researchers assumes that one source of ertor is the use of different raters. In the present case, at least for self-ratings, the rater did not change. Hence, a reliability of .60 would bbe an underestimate. Finally, there has been considerable debate over the accuracy of .60 as an estimate of reliability (Sackett, Tenopyr, Schmitt, & Kehoe, 1985; Schmitt & Schneider, 1983). Use of the higher, and hence ‘more conservative, figures therefore seemed in order.2 2 suo us usucanssou 30) 8 es 20) 5 Saison 2 Bones an pee g o we Gro Oe OLE Doe wv APS & wr eae WN Glew (SL x snostasodnss334 a s ww wise tne (Woe of aostniadns 196 Z punoq —punog si ° 7 ” z ‘bdd) mop 2 suownja0.) 40f synsox s186}oUD-DI2y vatavae, i HARRIS AND SCHAUBROECK st Because there was only one self-peer study reporting reliabilities, es- timates of peer-raing reliability were obtained from the peer-supervisor studies; likewise, estimates of self-rating reliability were taken from self supervisor studies. ‘As shown in Table 1, all three rater combinations had lower 90% conti dence intervals that were greater than zero, Hence, the correlation between ratings from different raters is statistically significant. However, there were marked differences between rater combinations: The peer supervisor cor. relation (p = .62) was substantially higher than either the self-supervisor (o = 38) or the selt-peer correlation (p = .36). Finally, all three rater combinations contained more than 25% unexplained variance even after sampling error, measurement error, and range restition were taken into account; accordingly, moderator variables were examined. Table 2 summarizes the moderator analysis for self-supervisor corre- lations. Consistent withthe hypothesis, there was slightly more agreement for dimensional (p = .36) than for global (p = .29) ratings. However, there was no decrease in variance across the subgroups. In accord with the rating-scale hypothesis, there was also a small difference between trait and behavioral scales, wherein greater agreement occured for the latter While the ratio of unexplained to total variance was less than 25% (15%) for behavioral ratings, there was no decrease in sample-size weighted vari- ance averaged across subgroups compared with the whole set. Thus, rating sale and rating format were not significant moderators for self-supervisor correlations. However, job type appeared to be a significant moderator. ‘When sampling error and other artifacts were taken into account, no vari ance remained for managerial professional employees, Although substan tial variance remained for blue-collar’ service workers, the average cor- rected variance across the subgroups was lower than forthe set as @ whole. Moreover, there was lower agreement for managerial professional employ. ees (p = 21) than for blue-ollar/ service employees (p = .42). Hence, jb type did seem to moderate self-supervisor comelations Table 3 summarizes the results of the moderator analysis for peer supervisor correlations. Contrary to expectations, slighty higher agreement as observed for global than for dimensional ratings; nonetheless, there was no decrease in variance across subsets, More surprising was the finding that greater agreement was obtained for trait (p = .64) than for behavioral ratings (p = .53). There was only a slight reduction in the sample-size weighted average variance across subsets as compared with the variance in the whole set (9 = 23 across subsets as compared with .24 for the whole set), Finally, job type did not seem to affect the amount of agreement. Due to the small number of studies (2) using either behaviorally based rating seales or dimensional formats, these variables were not investigatedBY PERSONNEL PSYCHOL 2 popu 2 Bue eo eu eur yo zou us sere sasiod WAP ila Seaqiun of ws Square Bus ys) w 31 wr air = Fea « (rome Me 0 os o wer os ws a renee 87 se a sez e e ast wmnoq—punog one 7a ToREPOR fsa Samo i aes na s2quen, reo amp) 106, ouso.) s0spsodns- fas ‘sins>y sIsSjouy AOI,33 HAUBROECK HARRIS AND ojsod ye Joy pov roswaue “Sompous See 5 ‘won co SORApORY sequins, us ony pa soil tod pau SUOUI 209 s9>4- HES “sHMED HESPOUY soMDLpoW patava 13 pu oun oa gus uns my 2 ads pauguo>, inant fi sgn rs Bad yn Sas eae du Sagan 7 on a ray st wer Sa roo of ovr We (er 8 6 ox ow 0 ars s 0s or we rns a at os w we (os 90 oF se 6 (onus ree ‘onan a ahr sve satiny, aL swoop Updns-12eq “Spey SIS fOHY s09D23po yy eaTavLst PERSONNEL PSYCHOLOGY fas moderators for self-peer correlations. Thus, only job type was consi ered; the results are shown in Table 4 'As in the self-supervisor category, little or no variance remained for either subgroup of self-peer correlations when sampling error and other aifacts were taken into account. Accordingly. there was a reduction in average variance across subsets as compared with the whole set, white the average corelation was higher for blue-colla/ service workers (p = .40) than for managerial/professional employees (p = .31)- Thus, job type did ‘meet the criteria for acceptance as a moderator. {A total of 18 independent self-supervisor means, 7 independent peer supervisor means, and 4 independent self-peer means were located. AS predicted by the egocentrie-bias explanation, there was @ fairly large dif ference between self- and supervisor ratings (d = 70). However, the true variance was also large (07 = .25). Thus, the difference in means between self- and supervisor ratings was not statistically significant, Simi- larly, while on average self-ratings were somewhat higher than peer ratings (d= .28), a substantial amount of variance remained even after correcting for sampling exor (2? = .09). Hence, the difference in mean self=peer ratings was nonsignificant. Finally, while peers provided somewhat higher ratings than did supervisors (d = .28), the difference was not significant given the large variance (0? = .11).* Discussion The first question of interest concemed the average correlation and ‘mean difference between rater pairs. Overall, the data suggest that after correcting for measurement error and range restriction, moderate agree- ment between self=peer (p = .36) and self-supervisor (p = .35) ratings ‘exists, There was much higher agreement between peers and supervisors (o = .62), Nonetheless, the lower 90% confidence interval was greater than zero for all three rater combinations. The results of this study then are somewhat contrary to conclusions by Landy and Farr (1980). As noted earlier, they concluded that different raters exhibit low to moderate agree ‘ment. The present results indicate that while this is true for self-ratings ‘compared with ratings by others, peer-supervisor ratings demonstrate a far higher average correlation. This discrepancy between a meta-analytic-based literature review and a traditional narrative review is not surprising, given the differences between the two procedures. The present results indicate the value of conducting quantitative reviews. 44, moderator analysis subgrouping by job type for self-supervisor means revealed no sedition n variance,HARRIS AND SCHAUBROECK 3s In terms of mean differences, on average, self-ratings were over a half standard deviation higher than supervisor ratings and approximately one- {quarter of a standard deviation higher than peer ratings. Nevertheless, given the large variances, the lower-bound confidence intervals included zero in both cases, However, examination of the effect sizes showed that in only fone case were the self-ratings lower than supervisor ratings. ‘The second question concemed whether or not differences between studies within each rater pair could be attributed to statistical artifacts, ‘The results indicated that in all cases substantial amounts of variance still remained, even when the appropriate corrections were made. Accordingly, ‘moderator analyses were conducted, Of the three moderators, only job type showed any meaningful ef- fects. Specifically, self-supervisor and self-peer correlations were lower for managerial/ professional employees than for blue-collar/ service employ: ees, and no true variance existed for the former category. However, such effects were not obtained for peer-supervisor correlations. This suggests that while incumbents of managerial/professional jobs have very differ: cent views of their performance than do others, observers exhibit far more agreement. One possible explanation is that egocentric bias is more likely to occur in ambiguous contexts (i.e., managerial/ professional jobs) than in well-defined jobs (.e., blue-collar/ service). Whatever the explanation, Job type does not appear to affect observer agreement; further research is needed to test why this is the case. The one instance where rating scale met the criteria for a moderator should be viewed cautiously for sev- eral reasons. First, only @ small number of studies (5) using behaviorally based rating scales were available, and the total sample size was small (n = 506). Second, the decrease in amount of variance across subsets was very small, Thus, rating scale was at best a relatively minor moder- ator of peer-supervisor agreement. In answer to the third question then, ‘moderators did help account for some of the variance, Recall earlier that three basic explanations were reviewed as to why disagreement between raters may occur. The present findings are consistent with two versions of the egocentric-bias theory. Specifically, in accord with attribution theory (DeVader et al, 1986; Jones & Nisbett, 1972), observers (c., peer-supervisor combinations) displayed higher agreement than did “actors” with observers ([e., self supervisor and self—peer combinations). ‘The findings also mesh with the moderated defensiveness explanation: even after the range restriction corrections, observer-observer ratings (i... Peer supervisor combinations) demonstrated much higher agreement than self- observer ratings. Conversely, there was no support for the main-effect-of-defensiveness explanation as correcting for range restriction did not reduce the difference soumae we36 PERSONNEL PSYCHOLOGY between self-peer/self-supervisor correlations and peer-supervisor corre- lations, Nor was there evidence that observational differences accounted for rater disagreement as there was a relatively high correlation between peer supervisor ratings. Finally, there was no support for the organizational-level hypotheses: an almost identical correlation was found between ratings by individuals at the same level (sel-raters and peers) and individuals at dif: ferent levels (self and supervisors). Moreover, the highest correlation was found for raters at different levels (peer-supervisor). Nevertheless, further research is needed to conduct more direct tests forall possible explanations of rater disagreement I is also noteworthy that, with one minor exception, neither rating format nor rating scale were significant moderators. Other meta-analyses Of performance ratings have come to similar conclusions. For example, Kraiger and Ford (1985) found that rating scale did not moderate ratee race effects; Heneman (1986) also reported a lack of support for rating format as a moderator. Hence, despite what would seem 10 be critical moderators. research has found rating instrumentation to be less useful in identifying sources of variation than was previously thought to be the ‘case. Conversely, the findings regarding job type suggest that this may be ‘a moderator of importance that other researchers in performance appra should examine. : Although there are a number of potential reasons why a substantial amount of variance between studies remained in most instances, two seem ‘most plausible. First, it seems likely that there are differences between studies in regard to criterion contamination or rater independence. Specit cally, studies may have differed as to how much raters knew about others" ratings. In some studies, subtle forms of criterion contamination may have existed, For example, itis possible that a few studies involving self-ratings were conducted shortly after performance appraisal reviews, wherein ratees had substantial information regarding supervisor ratings. In other instances, cemployees may have received informal feedback, while in yet other studies, cemployees may have had litle or no feedback of any sort. Unfortunately, very few studies report enough detail about the amount or type of feedback to lest whether this variable moderates rater agreement. ‘A second explanation for variance between studies may be differences in raters’ opportunity to observe job performance. For example, workers ‘may have more interaction with peers in some cases than in others, thereby leading to differential agreement with supervisors. Jobs with much task interdependence may lead to greater agreement between self- and peer ratings than jobs where workers perform independently. ‘Other potential moderators include such things as purpose of rating (exg., Zedeck & Cascio, 1982), amount of rater training (¢.g., Smith, 1986), and rater motivation (Bernardin & Beatty, 1984). However, a review of alHARRIS AND SCHAUBROECK 37 ur studies showed that these variables were also rarely, if ever, reported. Hence, no moderator analyses could be performed. Moreover, Melntyre, Smith, and Hassett (1984) found that purpose of rating had little or no effect ‘on ratings, and many studies have failed to show that training increases the accuracy of performance ratings, particularly when only a lecture approach is used (Smith, 1986). Further theory development and empirical research will be necessary to achieve greater knowledge about which conditions affect rater agreement Although one can always argue that a “file-drawer" problem may exist such than inclusion of missing studies would change the results, this is unlikely here. First, an extensive literature review was conducted. Second, Rosenthal’s (1979) index suggests that it would take many studies to dra: ‘matically alter the results, given the size of the correlations and the total sample size found here Future research needs to address 1wo issues, First, a more direct test of these competing hypotheses is needed. That is, a study should be done wherein different raters (e.g., peers, supervisors, and self-raters) will assess performance and provide information regarding moderators. For example, attributions regarding cause of performance, self-esteem, amount of obser. vational opportunity, as well as other possible factors would be measured. Second, depending on the outcome of this research, further work on the ‘ways to ameliorate rater bias would be useful. For example, if differential attributions are a source of conflict, the Performance Distribution Assess- ‘ment (PDA), described by Bernardin and Beatty (1984), might provide a ve. hicle for increasing rater agreement. Specifically, the PDA attempts to take into account situational constraints. Research should also focus on identify ing possible moderators of self-other agreement, For instance, impression management may be a variable that can account for self-supervisor and self-peer disagreement (Zerbe & Paulhus, 1987), Clearly, more research examining a variety of individual differences is in order. In terms of practical implications, the results suggest that self-ratings will generally show only moderate correlations with ratings by others This is particularly true for managerial professional employees. Practi- tioners considering the use of self-ratings should be aware that there is liable to be much disagreement. Conversely, peer-supervisor ratings often (but not always) reflect relatively high rater agreement, even for manage- rial/ professional jobs. For legal purposes, then, peer supervisor ratings may be useful, Using raters from different levels may also help to develop consensus, eliminate biases, and perhaps in tur lead to greater acceptance by ratees. Nevertheless, there is a plethora of other problems associated with using peer ratings (Latham & Wexley, 1982).38 PERSONNEL PSYCHOLOGY In conclusion, the present results show that peer-supervisor ratings demonstrate a relatively high corelation, whereas slf-supervisor and self peer ratings exhibit moderate correlations. Compared with the correla tion between supervisor ratings and results-oriented outcomes. (produc tion, sales, etc.) (Heneman, 1986), the average correlation obtained here between supervisor and peer-ratings is substantially higher. Thus, from both an absolute and relative point of view, there is high convergence be- tween observer ratings. The moderator analysis showed that corelations between self-peer and self-supervisor ratings are particularly low for man- ageria/ professional workers, but not for peet-supervisor ratings. While these findings are consistent with an egocentrie-bias explanation for rater disagreement, future esearch should test the alternative explanations de- scribed earlier in & more direct fashion than was possible here. REFERENCES Baird LS. (1977) Self and supervor ratings of performance: As related wo self-eeem and satisfaction with supervision. Academy af Management Jounal, 20, 291-30, Berardn HJ, Beaty RW: (1984). Performance appraisal” Assessing human behavior ot work. Boston, MA: Keat Publishing Company. Borman WC. (1974), The rating of individuals in oxgaiztions: An aerate approach, ‘Organisational Behavior a Hunn Performance, 12. 105-128 ‘Campbell IP; Dunnette MD, Lawler EE, Weick KE. (1970). Managerial behavior, peor ‘mance, and efecrivenes. New York: McGraw Hil ‘Carol 8, Schneier CE. (1982). Performance appratel and review sates: The identi tion, measurement, and development of performance n organations. Glenview, IL: Scot, Fores Cascio WE. (1987). Applied psychology m personne! management. Englewood Clif, NI Prentice-Hall | ‘Carnmings LL, Schwab DP. (1973). Performance n organizations: determinants and ap | rasa. Glenview, TL: Seo Foresnan. DeVader CL, Bateson AG, Lord RG, (1986). Atibuton theory: A metw-analysis of at ‘utonal hypotheses In Locke E (Ed), Generalizing from laboratory 1 feld sentings (pp 63-81). Lexington, MA: Lexingion Books Ferris GR, Yates VL, Gilmore DC, Rowand KM, (1985). The infuence of subocinate age ‘on performance rang’ and causal aibations. FeRSONNEL PSYCHOLOGY, 3, 545-857 | Henderson RI (1984). Performance apratal, Rest, VA: Reson Publishing Compan. Heneman HG IL (1974). Comparisons of self and superior ratings of managerial perfor ‘ance. Journal of Applied Poychologs, $9, 638-682, Heneman RL. (1986). The reationship between supervisory ratings and results-oriented mes sures of performance: A meta analysis. ERSONNEL PSYCHOLOGY, 39, 811-826 olabach RL. (1978), Rater bis in perfomance ratings: Superior, sf, and pcr rings, ‘Journal of Applied Psychology 8%, 379-588, Hunier J, Schmit FL, Jackson GB. (1982). Mewe-anasis: Cuming research ndings across sues. Bevery Hills, CA: Sage Publications. gen DR, Favero IL. (1988). Limits in generalization from paychoogial research operfor- mance appraisal processes. Academy of Management Review, 10, 311-32 Soves EE, Nisbet RE. (1972). The ator andthe observer: Divergent percepions ofthe causes ‘of behavior, In ones EE, Kanouse DA. Kelly HH, Nisbett RE. Vans S, Weiner 8HARRIS AND SCHAUBROECK 9 (is), Anibution: Percesing the causes of behavior (pp.39-98). Moristowa, Ni (General Leaming Press ‘Kay E, Meyer HH, French IRP. (1968). Ect of thet i a performance appraisal interview: Lournalof Applied Prychlogy. 49, 311-317 Klimoski RJ, London M. (1974). Role of the rater in performance appraisal Journal of ‘Applied Psychology, 50, 445-451 Kraiger K. Ford JK. (1985), A met-analyis of exes race effects in perfomance ratings oural of Applied Psychology, 70. 56-68, Landy 1, Far JL, (1980). Performance rating. Payhologcal Bulletin, 87, 72-107 Landy 4, Far J, Stl PE, Freytag WR. (1976). Behavioral anchored scales fr rating the Performance of police officers. Journal of Applied Psychology. 6, 458-857 {Latham GP, Wexley KN. (1982). Increasing productivity though performance appraisal Reading. MA: Addison: Wesky, {Lawler EE, (1967), The multitait-multater approach to measuring managerial job perfor. mance. Journal of Applied Psychology, 51, 368-381 Mabe PA, West SG. (1982). Vly of el eafuation of ability: A review and meta-analysis. ournal of Applied Psychology, 67, 280-296, Melnyre RM, Smith DE, Hassett CE (1984), Accursey of performance ratings a alfected ‘by tater training and perecived purpose of rating. Jounal of Applied Psychology, 6 147-156, Pearlman K, Schmidt FL, Hunter JE. (1980), Validity generalization results fr tess wsed 10 feet ob proficiency and ining success i lena acupatons. Journal af Applied Psychology 8, 373-406, Pym DLA, Auld HD. (1965). The self-rating as a measure of employee sstisfatorinss, Journal of Occupational Paycotogy, 39, 03-113 Rosenthal 8 (1979). The “le dawer problem” and tolerance for nl esuks. Psychological Burin, 86, 638-641 Suchet PR. Tenopyr ML. Schmit N, Kehoe J. (1985). Commentary on forty questions about ‘ality genetalzationa and meta-analysis, reasonnisrsvcHoLicn 3, 697.798, Schmit. Schneider B. (1983), Curent ses in penonnel selection, Ta Rowland KW, Farris GD (Eds), Research in personnel and human resourcer, Vol! (pp, 85-123) Greenwich, CT JAI Press, Smith DE. (1986). Training programs for performance appraisal: A review, Academy of Managemen Review. 11, 22-40 Springer D. (1953). Ratings of candidates for promotion by co-workers an supervisors Journal of Applied Psschlogy. 37. 47-381 Stele RP, Oalle NK. (1985). Self-appaial based! upon supervisory feedback. eKGONNEL FSYCNOLOGY, 47, 657-68, Thornton GC, (1980), Psychometric properties of selE appraisals of job performance. rex SONNE. SYCHOLOGY, 3. 263-271 Tucker MF, Cline VB, Schmit JR. (1967). Prediction of creativity and other performance ‘measures from biographical information among pharmaceutical scents. Jounal of Applied Psychology $1, 131-138 Wenley KN. Klimoski R. (1984), Perfomance appraisal: An updste. fn Rowland KM, ems GD (ls). Resarch i personne a luman resoures, Vol. 2 tgp. 35-79), Greenvich, CT: JAI Press, Willams WE, Seiler DA. (1973. Relationship between measures of effort and perfomance, Journal of Applied Paschlogy, 57.49.54 ‘ammo RF. London M, Rowland KM. (1982), Organicaton end tate differences in per formance appraisals. veRsonves rsvenotocy, 4, 643-688PERSONNEL PSYCHOLOGY edeck 8, Baker HT. (1972), Nursing performance measured by behaviors expectations scales: A multiaitmulirater analysis. Orgunisnional Behavior and Human Perfor mance, 7, 457-46. edeck S, Casco W. (1982), Peformance decision a a function of purpose of rating and teining. Journal of Applied Paychlogy, 67, 752-158. Zerbe Wi, Paulus DL (1987) Socllydesiable sponding in organizational behave: A reconcepion. Academy of Management Review, 12, 250-264, STUDIES INCLUDED IN THE META-ANALYSIS. Baied LS. (1977). Self and superior ratings of performance: AS relae'd to seesteem and satisfaction with supervision. Academy of Management Journal, 20, 281-300. Blackburn RT, Clark MJ, (1975). An asessmen of faculty performance: Some comelates between administrator, colleague. student, and sel rings, Sociology of Education, 48, 202-256, Booker GS, Miller RW. (1966. A closer look at peer rating. Personnel 43, 42-47. Borman WC. (1978), The rating of individuals im organizations: An akematve approach, ‘Organisational Behavior and Human Performance, 12, 105-12, Boruch RF, Larking JD, Wolins L, MacKinney AC. (1970) Allemative methods of analysis: Mule mukimetbod dat Educational and Psychological Measurement, 30, 833 Brit AP, Aldag RJ, Van Sel M. (1977), Moderators of the relationships between self and superior evaluations of jab performance. Journal of Occupational Psychology, 50, 129-134 Burke RJ, Weivel W, Desaca G. (1982). Supervisors’ and subordinates’ communication experiences in managing jo perfomance: Two blind men desrbing an elephant? Paychologiea! Reports, 50, 527-834 Campbell DJ. (1985), SelFappraals from two perspectives: Estem versus consistency influences. 28h Annuc! Proceedings ofthe Midwest Academy of Management Cena JA. (19). Colleagues 35 raters of classroom instruction. Jounal of Higher Educ tho, 46, 37-337. Doyle KD, Crichton CL (1978). Student, peer and self evaluations of college instructors. Journal of Eductional Psychology, 70, 815-825, kpo-Ufot A. (1979). Selfperceivedtask-elevant abilities, rated job performance, and com Plaining behavior of junior employees in a government mnitey. Journal of Applied Psychology, 64, 429-434 Fes GR, Yates VL, Gilmore DC, Rowland KM, (1985). The influence of subordinate age ‘om performance raings and causal tributions. FERSONNEL PSVEHOLOOY, #4, S4S-SS7 Fuqua DR, Johnson AW, Newnan JL, Anderson MW, Gide EM. (1988). Varabliy eros sources of performance ratings. Journal of Counseling Psychology, 31, 249-282 GGormly 1. (1983). Comespondence between personality trait rings and behavior evens “ournal of Personality. 52, 220-282 CGrifihs RD. (1975). The accuracy and comelates of psychiatric patents self assessment of theie work behavior. British Jownal of Socal and Clinica! Poychoogy. 14, 181-189 ‘Gunderson EK, Nelson PD. (1966). Citron measures for exvemely isolated groups. Pak SONNE PS¥EHOLOEY, 19, 67-80. Heneman HG Il. (1974). Comparisons of seif- and superior ratings of managerat perfor. mance. Journal of Applied Psychology, 59, 638-642, Hortman EL, Rover JH. (984). An objecuve peer evaluation scale: Construction and “ality, Educational and Pychological Measurement, 4, 382-34HARRIS AND SCHAUBROECK 6 Holzbach RL. (1978). Rater bias in peefomance ratings: Superior, self, and per ratings. Journal of Applied Psychology. 63. $9588 Iigen DR, Peterson RB, Martin BA, Bocschen DA. (1981), Supervisor and subordinate reactions t0 performance appraisal sessions, Oreaniational Behavior and Human Performance, 28, 311-330 Kaufman GG, Johason JC. (1984), Sealing peer ratings: An examination ofthe diferent valdites of postive and negative nominations. Journal of Applied Prchology, 39, 302-306, Kay E, Meyer HH, Pench JRP. (196). Effects of teat ina perfomance appraisal interview. Journal of Applied Psychology, $0, 311-317 ‘King LM, Hur JE, Schmidt FL (1980). Halo in muli-dimensions forced-choice perfor. mance evaluation scale. Journal of Applied Pechology, 68. 07-516 Kirchner WK. (1968) Relationships Berseen supervisory ad subordinate ratings fo technical personnel, Journal of Industrial Peyehologs, 3, 37-60 Klimoski RJ, London M. (1974). Roe of the rater in performance appraisal. Journal of ‘Applied Pychology, 89, 485-451 Kraut Al. (1975), Prediction of managerial succes by peer and taining staff ratings. Journal of Applied Psychology. 60, 389-394 Kulbany AJ. (1957), Use of sociomewc peer nominstions in medical education research, Journal of Applied Psychology 41, 389-394, {Lawler EE, (1967), The mutiait-multiater approach to measuring manager job perfor ‘mance. Jwnal of Applied Psychology, 1, 368-38, Levine EL, Flory A. Ash RA, (1977). Seléasersment in personne selection. Journal of Applied Peychology, 62, 428-435, Love KG. (1981). Comparison of peer astesiment methods: Relabiliy, valid, friendship bias, ad user reaction. Journal of Applied Psychology, 66, 451-5), Maslow AH, Zimmerman W. (1956). College teaching ably, scholaly setvty and person- ality. Journal of Educational Psychology, 47, 185-189, Mial WL, Graumenz IL (198). An assessment ofthe accuracy of self asessment for career decision making. Journal of Vocational Behavion. 25. 248-253 Mitchell TR, Albright DW. (1972). Expectancy theory predictions of the satisfaction, effort. ‘performance, and reiention of naval aviation officers. Organizational Behavior and Human Performance, 8, 2. Motowildo $i. (1982), Relationship tetween self-rated perfomance and pay saisaction among sles representatives. Journal of Applied Paychology, 67, 208-213, ‘Mount MK. (1984, Supervisor, sl, and subordinate ratings of performance and satisfaction with supervision. Journal of Management, 10, 308-320 Nealy SM, Owen TW. (1970). multiaitmulimethod analysis of predictors and criteria of faursing performance. Organizational Behavior and Haman Performance, 3, 348.365 (Orpen: (1983. Etfects of race of tater and see on pecr-ratings of managerial potat) Psychological Reports, 2, 07-510 Parker W, Tylor EK, Bamet RS, MarensL. (1950) Rating scale content I Relationship between supervisor and seatings,PeRsowNe PS¥CHOURGY, 2, 89-63, Porter LW, Lawler EE. (1968). Managerial anitudes and performance. Hetewood, tL Irwin Prien EP, Liske RE. (1962). Assessmens of higher-level personel: Il Rating criteria: A ‘comparative analysis of supervisor ratings and incumbent ratings of jab performance PERSONNEL Psveun.00y, 15, 187-194 Pym DL, Auld HD. (1965). The selfsating asa meas of employe saisfetoriness. Journ of Occupational Pychology, 39, 103-113. Schmit N, Saar BB. (1978). Behsvor, situation, and rater variance in descriptions of leader behaviors, Mulibariote Behavioral Research, 13, 483-498a PERSONNEL PSYCHOLOGY Schneier CE, Beatty RW, (1978), The influence of role prescriptions om the performance ‘appraisal process. Academy of Management Journal, 21, 128-135, Shapiro GL, Desser G. (1988). Are self-aprasals more realise among professionals oF nonprofessionals in health care? Public Personnel Management 1, 28-290 Shore LF Thorton GC Ill (1986), Elects of gender on self and supervisory ratings. Academy of Managemen Jour! 29, 15-12, Shrauger JS, Lund AK. (1973). Self-evaluation and reactions to evaluations fom eter. ournalof Personality, 43, 94-108. Siegel L_ (1982). Pied comparison evaluations of manager effectiveness by peers and supervisors. Feasonn rsvenotocy, #5, 43-852 Smircich L Caeser RU (1981), Supervisor” and subordinates’ perceptions of performance: eyoredisareeman, Academy of Management Journal 24, 198-208, Springer D. (1953). Ratings of candidates for promotion by co-workers and supervisors Journal of Appled Psychology, 47. 47-351 Stel RP, Onalle NK. (984). SelFappraval based upon supervisory Feedback. exRsONNE, ravenna, 47, 667-085, “Thorton GC. 1968), The slationship between supervisory and scitappaisal of exceutve Performance, PRSONNEL YSYCHOLOGY, 27, 81-188. ‘Tucker MF. Cline VB, Schmit JR, (1963), Prediction of creativity and ater perfomance measures trom biographical information among pharmaceutical scents. Journal of Applied Psychology. 51. 131-138 Webs WE, Nolan CY. 1955), Student, supervisor nd selfatings of intctional proficiency. “oural of Educational Peycholoes, 48, 42-46 Wiliams WE, Seiler DA. (1973). Relationship beween measures of effon an job perfor: mance. Journal of Applied Paychology, 57, 49-88