Options for user engagement: what works and what doesnt work?

The following table provides a considerable range of practical mechanisms for user engagement. These have been drawn together from a wide range of sources and synthesized in order to minimise, where possible, overlap between the different methods. For each, the typical pros and cons have been highlighted. Where it is particularly relevant or important, the size of groups typically involved in the mechanism, plus an indicative sense of the cost of the approach, are highlighted. The purpose of the table is to provide an overview of what practical methods are available. Note: means by which to give information are not generally included within this table. They include: leaflets, posters, websites, organisation newsletters, exhibitions, mailouts, fact sheets, advertising, media, videos, photography, text messaging etc. Table: Practical mechanisms for user engagement Method Which part of involvement continuum Participation Description Typical pros Typical cons

21st Century Town Meetings

21st Century Town Meetings involve a large number of people (between 500 and 5,000) in deliberating on local, regional or national issues and make use of modern technology, including wireless voting pads and networked laptops. They combine the benefits of small-scale, face-to-face discussions with those of large group decision-making. Area Forums are meetings held in a particular locality. Often the meetings are attended by local councillors, together with senior representatives from the local authorities, the Police, Primary Care Trusts and other key local organisations, to debate key topics and answer residents' questions face-to-face.

Combines large number of participants with considered dialogue Gathers clear and instant information on what participants think about an issue, including demographic data on what different groups feel The immediacy and scale of the event can energise the participants Encourage openness and transparency around council decisions Discussions can be tailored to the concerns of local residents Provide a direct interface between elected representatives and the communities they represent

High cost Can raise expectations to unrealistic levels if not managed well Reliant on technology

Area Forums

Forums for debate

Tend to be attended by the 'usual suspects' although many try hard to attract particularly young people and residents from black and minority ethnic communities Individual workshops may be dominated by one

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

person or particular viewpoint Citizens Juries Forums for debate Citizens Juries consist of a small panel of non-specialists, modelled on the structure of a criminal jury. The group set out to examine an issue of public significance in detail and deliver a verdict. Citizens Juries are particularly appropriate for involving the wider public in decision-making about setting priorities and strategic planning choices. NHS organisations can pose difficult questions, for example on the prioritisation of services, which involve value judgements in reaching a decision. The jury consists of 12 to 16 members of the public, and members are drawn from a cross-section of the local population. The jury meets for several days, during which time it hears evidence from key witnesses e.g. health service commissioners, service managers, interest groups. Jurors are also able to ask to hear from additional witnesses identified through the process. After hearing all the evidence, the jury considers the issue in depth before making recommendations to the NHS organisation. High profile demonstration of public engagement Jurors broadly reflect the characteristics of the wider population, so bring a degree of representativeness. Jurors participate as citizens, so, in theory they are not direct stakeholders, but seeking the best outcome for local people as a whole. Jurors are provided with information from a wide range of perspectives, with time to discuss it in detail, and so are able to come to an informed view. Can provide an informed public view and generate wider public debate As with all activities involving a limited number of people, jurors are open to criticisms of being unrepresentative. Although jurors are drawn from the local population, their views and values may not reflect those of the wider population Jurors may not be able to articulate their views and concerns. Citizens Juries involve a huge amount of resources (estimate about 25,000) in terms of planning (months) and execution. The jury may not be able to reach consensus on its recommendations Rejecting the jurys verdict can be problematic

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

Proceedings are commonly open to the general public to attend and listen to, but they are not able to participate. The process is normally facilitated by an independent moderator. Citizens Panels Forums for debate A Citizens Panel is a large, demographically representative group of citizens regularly surveyed to assess public preferences and opinions. Citizens panels are generally used to build a picture of a communitys priorities, or to get a measure of public opinion on a specific issue a health organisation is working on. Panels are generally made up of 1,000 to 3,000 people (depending on the size of the population they represent) and members reflect the demographics of the local community. Membership of the panel is normally for a fixed period, e.g. two years, after which response rates and engagement tend to decline. Three or four times a year, panel members are surveyed, commonly by postal survey, on their views of a particular topic or topics the organisation is working on. Panel members may be given additional information in advance or alongside the survey to support them to develop a more Panel members are recruited to reflect the demographics of the local community and can therefore be seen as representative of the views of the wider community. Panels generally involve a larger number of people than other forms of engagement, and their findings therefore, tend to hold more credibility. Due to the complexity of recruiting and managing the panel and analysing its findings, panels are commonly run by external organisations (e.g. MORI), which gives further independence and credibility. Panels are complex and resource intensive to recruit, maintain and analyse their views. This means they are usually run by external organisations, which can be expensive. Panels have all the disadvantages associated with surveys in general e.g. more likely to generate quantitative rather than qualitative information, potentially exclude people with literacy problems etc. It can be difficult to maintain the enthusiasm of panel members, leading to a decline in the return rate of surveys.

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

informed view. Many local authorities have a citizens panel, which may be used by NHS organisations. Complaints analysis Getting information Scrutiny of complaints received to identify common themes Provides high-quality data about existing service Substantial issues can be identified Some complaints can be tackled before they become a problem Good service can be noted and rewarded New ideas from the public can be passed on and used You can determine who the most appropriate participants are, for example, this could be people that represent a cross-section of the local population or stakeholders in a particular issue, and could include health professionals. Conversation Cafs enable participants to both share knowledge and generate ideas. The use of mixed and changing groups during the course of the event means that participants hear the same issue from different perspectives, developing their own thinking. Conversation Cafs generate Generally only covers negative feedback Groups who tend not to complain will be underrepresented Might be seen by frontline staff as additional work

Conversation cafs

Forums for debate

Conversation Cafs or Open Surgeries are informal dialogue methods which invite people to take part in discussions about topical issues in an informal setting. Conversation Cafs are a way of enabling a large group of people to discuss questions in an informal and creative way. It does this by creating a relaxed, caftype environment, in which people can talk to others in small groups, and move between groups to meet new people and get a fresh perspective. Conversation Cafs normally take approximately 2 hours. To be successful, caf consultations really need to involve

It is helpful if the event is run by someone who is experienced in running Conversation Cafs Conversation Cafs can be expensive, as you will need to pay for room hire and refreshments, as well as the staff time organising them.

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

20 people or more. Deliberative events Participation Deliberative event is a generic term for dialogue events where the focus is on deliberation. Some deliberative events record/measure what the participants think about an issue having had an adequate chance to reflect and deliberate on the questions at hand. Deliberative mapping involves both specialists and members of the public. It combines varied approaches to assess how participants rate different policy options against a set of defined criteria.

qualitative information from a potentially large group of people. People can shape public policies or plans and feel they have ownership Requires careful planning and good moderation skills The organisation has to be prepared to devolve some decision-making Difficult to involve large numbers and high in cost and time-commitment. The results of the process can be contradictory views that leave decisionmakers without clear guidance. Very few people have practical experience of running this kind of process. Blogs: There can be offensive, personal, pointless and inappropriate comments written in response to blogs. Content may need to be moderated. People may need to be encouraged to contribute to

Deliberative mapping

Participation

The results are considered opinions rather than articles of faith or rash judgement Specialists contribute to the process without dominating Combination of different approaches creates a deep and comprehensible understanding of public priorities.

Electronic discussion groups

Forums for debate

A range of techniques that aim to impart, exchange and receive information electronically. They can be used to target existing patient and carer groups, interest groups and support groups or set up for a specific purpose. Blogs are online journals or noticeboards where people or organisations can

Blogs: Open and transparent, although often anonymous. Allows anyone to contribute and in their own time. Allows different views to be aired and discussed. Some bloggers have become famous in their own right.

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

provide commentary and critique on news or specific subjects such as politics, local events or health matters. Some blogs function like personal online diaries. Online forums are web tools that allow discussions to be held online. They allow participants to post their own comments online, which distinguishes them from one-way communication tools such as email bulletins. Webchats are based on the concept of instant messaging. This is a new and informal way to gather information from different stakeholders and to answer specific questions they may have. Participants are invited to contribute to the discussions, but normally anyone can observe the proceedings online even if they cannot contribute.

Engages people that may not normally be involved in face-to-face consultations.

the website and post their comments. Blogs rarely allow people to participate actively beyond responding to existing posts Online forums: Limited deliberation on online forums, many people just post their comments and do not engage with what others have said Limited to those with internet access Can be difficult to get people to post Requires moderation un-moderated Online Forums are often chaotic but anonymous and unaccountable moderators can also frustrate participants Webchats: Users expect a fast response from those answering the questions that may not always be possible. The webchats may not have any direct policy input even though participants are discussing directly with a

Online forums: Participants can access it at any time Anonymity of internet can help people feel comfortable stating their views Feedback can be gathered quickly Online forums combine the spontaneity of verbal communication with the clear records of written communication

Webchats: The discussion between the person answering questions and the stakeholders or general public is in 'real time' - there is little or no delay in responses.

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

The discussion is structured but the person answering the questions could participate in more than one webchat simultaneously; The discussion can be aimed at appropriate participants The webchat can be publicised on the normal website and the discussions can be left posted on it. There is a sense of the webchat being an event and more personal than other online forums. ePanels Getting information ePanels are a way for councils or other organisations to carry out regular online consultations with a known group of citizens. Can be run alongside traditional offline activities and their strength is seen to be as a way of increasing participation in local democracy, particularly amongst young people or those who are time poor. Enables local authorities to reduce their administrative costs - no paper questionnaires or postage is required, there are limited additional costs to run a focus group or live chat (just the cost of online facilitators), data input is not necessary and analysis is generally quicker and can be immediate depending on the type of econsultation being used. Open and transparent, although often anonymous. Allows anyone to contribute and in

relevant person

As with all online methods, there is the potential that ePanels don't encourage the participation of those people without ready access to the internet. If too much is asked of participants such as too many follow-up emails from ePanels then this can lead to alienation from the process, and calls for responses may be ignored.

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

their own time. Allows different views to be aired and discussed. Engages people that may not normally be involved in face-to-face consultations. Focus groups Forums for debate Focus groups are guided discussions of a small group of people. They are normally one-off sessions, although several may be run simultaneously in different locations. Allows you to identify a framework for discussion, whilst providing freedom for participants to contribute the content. Useful for providing an overview of issues and identifying issues to be explored in more detail or with a greater number of people later. Good way to explore issues in depth and get qualitative data. Interaction between participants may generate further thinking and ideas. Facilitator can ask for responses to be clarified or seek further information in particular areas as issues emerge. Can be an empowering and learning experience for participants. Can bring similar people together and help in forming an on-going group for consultation and involvement. Good way to involve those who cannot read or write, wish to communicate in languages other than Generally involve only a small number of people relative to the wider target population, so are not good for gathering quantitative information and are open to the criticism that participants views are unrepresentative. Resource intensive to set up, run, analyse and report on. Confidentiality harder to ensure. May generate a large amount of information that, due to the more flexible approach, is not directly comparable across groups. Requires a skilled facilitator to ensure everyone is enabled to participate fully.

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

English, or have other communication needs. Forum theatre Forums for debate Forum theatre is an interactive form of theatre that encourages audience interaction and explores different options for dealing with a problem or issue. This technique is used to work with socially excluded and disempowered groups. Combines high quality, innovative and interactive theatre with social objectives. Acts as an ambassador for the arts in the social sector. Provides an entertaining and meaningful way for working with socially excluded groups. Challenges established perceptions. Powerful tool for exploring solutions to difficult problems Develops skills of the actors, whom are often people for socially excluded groups. Panels are made up of people recruited for their characteristics in terms of age, gender, ethnicity and location of residence, rather than their particular interests or knowledge about health issues, and represent a crosssection of the population. Complex issues can be discussed and deliberated by people in an informed way. Panels can provide useful views on resource allocation and priorities Forum theatre requires the skills and ability amongst the organisers to create a powerful and meaningful play. Forum theatre requires actors with the skills to improvise around the audience participation. Forum theatre is rarely linked directly to decision making.

Health panels

Participation

Health panels have primarily been used to explore peoples views on live policy issues and the allocation of health service resources. Participants are usually recruited using a quota sampling technique to reflect the socio-economic make-up of the area. Membership is refreshed on a regular basis.

The information given to panel members is drawn up by professionals, who can only anticipate the type of information needed from their own perspective. It may be unknowingly selective or inadequate. If too many issues are discussed there may be insufficient time to allow much debate and panel

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

between treatments. Panels receive relevant information beforehand to enable them to give reasoned and informed opinions. Taking the views of panels into account when making decisions can help to demonstrate accountability. Panels can meet at regular intervals, which helps develop a dialogue with local people.

members may end up just giving their views, rather than the reasoning behind them and developing their own thinking. The success of the panel discussion is heavily dependent on the skill of the facilitator. Panels do not provide quantitative information because the number of people taking part is too small. Expertise is needed in preparing the questions. Setting up interviews can be time consuming. Without significant investment of resources, the number of people interviewed is likely to be comparatively low, so findings are open to criticism that they are not representative. Analysis of findings from semi-structured interviews is time consuming and requires skill. Some client groups may

Interviews

Getting information

One-to-one interviews can be conducted face-to-face or on the phone. Semi-structured interviews allow for more qualitative information, and aim to get feedback or explore and issue and enable interviewees to express their own feelings and concerns. Structured interviews can be used to get quantitative data, but are less flexible for getting qualitative information, as they do not provide an opportunity for users views and perceptions to be explored in any detail. Their findings are easier to analyse than those from semi-structured interviews.

Possible to get a good cross section of the population and reach specific groups e.g. by using interviewers who speak community languages. Semi-structured interviews provide a framework for discussion and freedom to explore peoples views in more detail. Can specifically target the characteristics of people you want to interview. May involve people who would find it difficult to attend an event or feel inhibited speaking in a group Independent view can be achieved by using external organisations to do

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

the interviewing. Response rate can potentially be improved by using community organisations although beware of potential bias in the reporting. Mystery shopper Participation Mystery shopping is a way of auditing services through the involvement of trained user volunteers. Mystery shoppers have been described as under cover service users. Mystery shopping can help to assess the customer care aspect of services. As real patients are not involved, there are no confidentiality issues.

necessitate the interviewer to be chaperoned or have CRB clearance.

This method cant be used when mystery shoppers would have to display symptoms or be prescribed medication for their conditions. Mystery shoppers are not generally able to explore an issue in-depth. Mystery shoppers do not necessarily bring expertise or experience of a service as a user. If not carefully planned, online consultations can generate unmanageable amounts of material Excludes people who do not or cannot access/navigate the internet The technology can shape the process rather than vice-versa; Written communication can be a barrier for some

Online consultations

Getting information

Online consultations utilise the internet to ask a group of people their opinion on an issue (typically a policy in the development stages). An unlimited number of participants can be sent information about the subject or download it online and respond via email or comment on a website.

Allows a large number of people to contribute Gives all participants an 'equal voice' Can reach people who are unlikely to respond to traditional engagement methods A quick and accessible mode of engagement from the participants' perspective Allows participants to discuss an

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

issue at their convenience (regardless of location or time); Anonymity of online processes can encourage open discussion; Open space Forums for debate Open space events are a large group event (15 people upwards), where participants themselves create their own programme around a pre-determined theme. Apart from the theme, there are no speakers and no set agenda, so participants decide exactly what is discussed and when. Open space events are generally run over one to three days, although it is possible to run shorter versions. Open space events can generate ideas and thinking completely determined by participants. Open space events can accommodate a large number of people. Open space events work well with a diverse range of participants, in terms of background e.g. health professional or other stakeholder demographics and other characteristics Participants only need engage in discussion areas that interest them, hopefully ensuring that they are fully involved, rather than disengage at different pointsduring the process. Over time, involving people in keeping diaries can help to develop a relationship of trust. Diaries record events and feelings as they happen. The use of guide questions enables you to explore the issues you are interested in.

already marginalised groups; Any perceived complexity, such as registration, can be a barrier to participation Although the theme is pre-determined, it is impossible to predict the areas of discussion that will be generated participants, therefore some areas may be left unexplored. Open space events can be complex to manage on the day.

Patient diaries

Getting information

These techniques invite participants to capture and record their experience of health services in a way that can be fed back to staff. Staff discuss the insights obtained and work with participants to make improvements. The techniques can be used in a variety of ways from being a means of getting information through to being a vehicle for working in partnership

The person completing the diary will usually feel they have invested a lot of time and effort and will want to see some clear actions resulting from it. As completion of the diary is unsupervised, responses to the guide

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

The diary approach provides qualitative information, allowing patients and carers to reflect, explain and suggest ideas and solutions.

questions may stray away from the point. Diaries can be very time consuming to analyse. Those who have literacy problems may not be able to take part. Public meetings are resource intensive, in terms of staff time planning and running them, and costs associated with publicity, venue hire, and refreshments. Unless well promoted, or debating a controversial issue, there may be a low turn-out. Participants are not generally required to book in advance, so numbers are unknown until the day, making the design of the session harder. As participants are not generally required to book, you have no control over who attends, and those attending may be unrepresentative. The event may only attract interested parties,

Public meetings

Forums for debate

A meeting for which there has been an open invitation. There may be a set agenda or discussion may focus on issues raised at the meeting. In the past, public meetings have tended to be used as a method of giving information, but may not be the best way to do this. They can be used creatively to get information from participants or as a forum for debate.

Opportunity to reach a wide range of people, and potentially everyone with an interest in the issue to be discussed. Enhances accountability, as the public can directly challenge those responsible for decision-making. Can combine information giving with discussion.

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

lobby or pressure groups. There is the potential for interest groups to hijack the meeting with their particular issues or views. A single meeting is rarely sufficient, as no one day, time or location will suit everyone. Seminars Forums for debate Seminars are discussion groups that aim to impart, exchange and receive information and views. There is normally much greater input from the facilitator than in a focus group, and participants may well include a mixture of professionals, public and patients. Depending on the design for example, you might include group working, as well as plenary discussions seminars can accommodate a range of numbers of people. There is an opportunity to put across information to inform the debate, so can generate informed discussion. Participants may come from mixed backgrounds e.g. health professional, service manager, service user so will have a (rare) opportunity to hear views from others perspectives. This can potentially lead to more informed discussion and thinking. Provides qualitative information about peoples views. Existing groups with an interest in the issues to be discussed are likely to find this process engaging and rewarding. You can potentially gather data from large numbers of people Can be used to target particular Generally involve only a small number of people relative to the wider target population, so are not good for gathering quantitative information and are open to the criticism that participants views are unrepresentative. Resource intensive to set up, run, analyse and report on. Confidentiality harder to ensure. Requires a carefully planned programme and skilled chair / facilitator to ensure everyone is enabled to participate fully. The questions are determined by those designing the survey, and

Surveys and questionnaires

Getting information

Surveys include a group of techniques that measure the opinion of a sample of people. Depending on the particular

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

technique employed, quantitative or qualitative data or a combination of both will be obtained. Techniques can include: Self-completed questionnaires Online surveys Telephone surveys

groups Can be used to give information at the same time You can design a sampling framework to survey a representative sample Findings can be easier to analyse as the questions (and potentially the responses) are clearly defined Findings provide quantitative data on how many people hold a particular view or prefer a particular option Questionnaires are quick and easy for people to complete, and do not need to be supervised or completed on-site You could use a professional organisation to do the work, which would give more independence Relatively cheap

can omit issues of major concern to people Findings are not in-depth as there is no opportunity to explore issues, ideas or experiences further People are less likely to complete questionnaires and questions that seek more qualitative data Questionnaires are unsuitable for people who have problems with literacy, for example, people with learning disabilities, and people who do not speak English as a first language Depending on how questionnaires are distributed and targeted and the response rate, findings may be unrepresentative of your wider target population The NHS uses this format often and you need to ensure you are not duplicating work already underway as there is the chance of survey fatigue with recipients Service users on the

User groups

Participation

User groups are groups of service users

By definition, members of the

Table: Practical mechanisms for user engagement Method Which part of involvement continuum Description Typical pros Typical cons

that meet regularly to discuss the quality of a service and other related topics. The forum has a recognised mechanism for feeding into the decision-making of a project or service. User forums involve longer-term engagement with people who are key stakeholders and generally wellinformed about the issues and service available. They help to identify the concerns and priorities of other service users and can lead to the early identification of problems or ideas for improvements. World Caf Forums for debate World caf is a method that makes use of an informal caf for participants to explore an issue by discussing in small table groups. Discussion is held in multiple rounds of 2030 minutes with a plenary at the end of the event.

group use the service and therefore bring some experience and / or expertise in the issues. Because the meetings are ongoing basis, you can build a positive relationship with the forum members. Because forum members are involved on an on-going basis, they can develop additional expertise in the issues.

group may be unrepresentative of the views of wider service users. A service user forum cannot capture the views of potential users who are for some reason unable or unwilling to access the service.

Creative process for developing new ideas Informal and inclusive Has the potential to be cheap and easy to organise

Requires a clear and relevant question Cannot be used to make direct decisions