BMJ 2002;324:S195 ( 22nd June ) Career focus 360 appraisal

Jennifer King outlines what it is and how it should be used so you won't be aimlessly turning in circles Do you know what your colleagues and patients think of you at work? Do you see yourself as others see you? These are questions that many of us may prefer not to ask in case we dislike the answers. But if you are a doctor you now have to provide evidence about your working relationships with patients and colleagues to comply with the GMC's standards on good medical practice. "360 appraisal" is suggested as one of the sources of such evidence, which will eventually form part of the portfolio required for GMC revalidation.1 However, many doctors are unfamiliar with 360 appraisal, and, although some doctors and departments have responded with keen interest, many are confused and sceptical about what it is and the value of the feedback it provides.

What is 360 appraisal?

The term originated in the commercial sector, and refers to "full circle" feedback from bosses, peers, and those junior to you. It is sometimes referred to as "multi-source feedback." An individual seeks feedback about his or her performance at work from a variety of colleagues with the aim of using it to highlight areas of strength and those in need of improvement. The method evolved as the limitations of the more traditional "top down" appraisal became apparent namely, that it was perceived as unfair, biased, limited to one person's perspective (usually the boss), and often demotivating. 360 appraisal had the potential to overcome these problems, and over the past decade it has been used extensively in industry and introduced in some general practices and hospital departments.

What is it for?
360 appraisal can be used for two broad purposes to make decisions about pay and promotion (performance management) and to determine how people are developing their skills and competencies. Because of the potential for bias, most organisations no longer use it for performance management and recognise it as a developmental tool. It is designed to look at the behaviours (or competencies) that are key to a job including teamwork, communication, managing others, and interpersonal skills. As the business culture has moved towards looking not only at what people do but how they do it, 360 appraisal provides a way of measuring relevant behaviours. It generally does not include many items on technical or clinical skills as there are other ways of measuring these.

What method should be used?

Methods in the health service have varied. They include open ended, unstructured interview questions (such as "Tell me three things that Dr Smith does well and three things he needs to improve"), statements with a simple rating scale (such as "Is enthusiastic about teaching" rated from 1 to 5), and structured questionnaires based on items from focus groups with general practitioners or consultants about what they (and other healthcare staff) consider to be indicators of good performance. The most widely publicised tool is an American questionnaire developed by Ramsey and colleagues,2 who concluded that reliable ratings can be obtained with 11 responses. Another view is that there should be at least five people providing ratings, to ensure some anonymity.3 Ramsey et al also reported that the method of selecting colleagues, and the relationship between the person doing the rating and the person being rated, do not substantially bias results. However, anonymity is crucial to the validity of these methods.1 Typically, a 360 feedback questionnaire would include key categories (such as communication, teamwork, etc) with five or six specific behaviours within each one. Instruments will probably be developed that specifically reflect the performance criteria in "Good Medical Practice," but, as yet, there is no standard or universal method, and a variety of tools are being tried in the NHS. Most questionnaires tend to ask how well a particular behaviour is performed and how important it is. In this way it is possible to prioritise the areas which need developing. The behaviours rated low on performance but high on importance are clearly of higher priority than areas considered less important by the person doing the rating. If the instrument includes self rating, then people being appraised can see how closely their views match with the perceptions of others. This is especially valuable for team development as different healthcare professionals have different views on what is important. When this method was first introduced into primary health general practitioners asked staff and patients to rate the practice rather than individual doctors. This method was adopted by several practices and primary care trusts, which used the feedback to guide their practice development plans. The more courageous began to gather personal feedback about individual doctors, which contributed to team building within practices. Some hospital departments have started using 360 appraisal in this way. They tend to be departments with an open and supportive culture, whose members are willing to learn from the feedback and use it to enhance team performance. Other trusts have introduced it as part of individual consultant appraisal,4 but the onus is on the person being appraised to gather the data and bring it to the appraisal to help inform the personal development plan produced at the end.

What are the benefits and pitfalls?

Benefits of 360 appraisal

"Peer ratings provide a practical method to assess performance in humanistic qualities and communication that are difficult to evaluate reliably with other measures"2 It is potentially more objective and less open to bias than feedback from just one person, or a doctor's subjective view about himself or herself. It is harder to discount the views of several colleagues or patients than the views of just one or two It provides some standardisation (rather than every doctor using a different method) Colleagues, staff, and patients feel they have some say in how they are being treated It helps to identify strengths as well as areas in need of further development It can help people to target specific behaviours that need to be addressed (such as "Dr Smith does not always make time to discuss my problems") instead of more general and less helpful feedback (such as "Communication skills need improving"). It can aid team development.

Pitfalls of 360 appraisal

Instruments may not be validated (so items may not measure what they are supposed to) Free text (included in some questionnaires) can produce destructive comments that can negate any motivating effect Colleagues may not give honest feedback if they fear that their responses can be identified or if they are unclear about how they will be used It can be used to support a hidden agenda (for example, to discipline a doctor or make a decision about promotion) It is time consuming to collect ratings from 5-11 people It may be difficult to get a representative sample of patients If the tool is not robust then any development plan based on the feedback could be misguided Appropriate support systems to help administer the questionnaires and interpret the feedback may not be in place The links between appraisal and improved performance are not well established and require more research evidence (studies show that ratings may become more positive over time but this does not necessarily show that performance has improved).5 This will be difficult because of the many other influences on performance.

How can 360 appraisal be successfully introduced?

Based on our own and others' experiences of designing and implementing a variety of tools, the following are essential: Robert Burns: "The gift to see ourselves as others see us"

The tool must be well validated and easy to administer, analyse, and interpret Feedback must be anonymous (there must be no way of identifying who said what) The team or department must be fully briefed about how the data will be used, who will analyse the results, etc It should be used only for developmental purposes, not performance management It should be introduced slowly on a voluntary basis, starting with the departments or practices keen to pilot it. It should not be used to resolve conflict or in a department or practice with a history of difficult relationships The tool should be constantly refined Any decision affecting a doctor's career should not be based on 360 feedback alone. It is part of a broader array of evidence about a doctor's performance, from which appraiser and appraisee can identify overall patterns, themes, and messages. Treat it in context Only those who work directly with appraisee should be asked to provide feedback, but it makes little difference to the feedback if they are chosen by the appraisee or the appraiser. Those who provide feedback should represent a range of people (for example, trainees, peers, administrative staff, other healthcare professionals, clinical director, etc) Training must be given to appraisers and appraisees about how to make the most of the feedback.

Conclusion
Many questions still have to be answered about 360 appraisal, but it has the potential to be a useful tool for individual and team development, when used responsibly and in combination with other sources of evidence. Some hospital and primary care trusts have successfully piloted different methods and incorporated the feedback into appraisal discussions. It is essential that best practice is shared so that tools become

standardised, especially if they are used for revalidation purposes. 360 appraisal has a great deal to offer and could provide what Robert Burns called "the gift to see ourselves as others see us." Jennifer King, director, Edgecumbe Consulting Group jenny.king@edgecumbe.co.uk A national conference, "A practical guide to 360 appraisal," organised jointly by Healthcare Events and Edgecumbe Consulting, will be held at the Royal College of Physicians, London, on 27 June 2002 (for further details contact Healthcare Events, info@healthcare-events.co.uk). Competing interests: Edgecumbe Consulting Group is a consulting company providing services to the NHS and the corporate sector and has developed its own 360 appraisal tool for doctors.

References
1. Mason R, Zouita L, Ayers B. Results from pilot study using portfolio and 360 questionnaire [letters]. BMJ 2001; 322: 1600[Full Text] (http://bmj.com/cgi/content/full/322/7302/1600). 2. Ramsey PG, Wenrich MD, Carline JD, Inui TS, Larson EB, LoGerfo JP. Use of peer ratings to evaluate physician performance. JAMA 1993; 269: 16551660[Medline]. 3. Fletcher C. Appraisal: routes to improved performance. London: Institute of Personnel and Development, 1999. 4. Johnson C, Leigh J, Lloyd S, Hasler J. Consultant peer appraisal. A structured system to support clinical governance and revalidation. Clinical Governance Bulletin 2000;1(2). 5. Fletcher C. Performance appraisal and management: the developing research agenda. J Occup Organ Psychol 2001; 74: 473-487.