Designing a Questionnaire

This document is intended to help business owners design their own research instrument, which can be used to gather market research information from their customers or others in their marketplace. Although survey questionnaires can be a relatively simple way to obtain market research data, a poorly designed questionnaire can lead to wasted time and effort, useless data and frustrated respondents. The tips that follow can help to ensure that your market research questionnaire is well designed.

Tips for Designing a Questionnaire

Keep it short and simple
A lengthy questionnaire is less likely to be completed and returned. It's important to establish clear goals for your market research project and to avoid including questions that do not contribute to the achievement of this goal. It is also important to avoid using an overly complex structure with the design of your questionnaire, especially if it is being administered on paper. An example of a questionnaire that uses a complex design is one that requires respondents to skip or complete questions or sections based upon their previous responses. These designs can be useful, and if designed properly can often be achieved without burdening the respondent. Complex designs are generally best suited for online or telephone questionnaires. The software can ask the appropriate questions based on previous responses. In paper questionnaires and in-person interviews, however, the use of 'skip patterns' and other potentially burdensome design elements should be kept to a minimum.

Keep it clean and visually appealing

Respondents will be more receptive to a questionnaire that they feel can be completed quickly and easily. Consider the following design elements before finalizing the look of your research instrument:

Text: Choose a font style that is clearly legible, and make sure the font size is large enough for most of your respondents to read. If you choose to use coloured text, make sure the contrast on the paper or screen is adequate. Paragraphs: Long paragraphs can be daunting to readers. Whenever possible, keep your blocks of text to a handful of lines. White space: The content of your questionnaire needs room to breathe. Don't make margins too small and ensure that there is adequate space between questions and sections.

Start with general questions and move to specific ones General questions often require less thought and are easier for respondents to answer. They can serve as a 'warm-up' that will help

your respondents to get into the questionnaire and help them to answer more specific questions faster and more accurately later on, by easing them into the correct frame of mind. Jumping back and forth between general and specific questions will force your respondents to zoom in and out mentally as they attempt to answer the questions. This can negatively affect the quality of the information they provide and lead to frustration. Potentially objectionable questions, such as demographic questions, should be placed near the end of the questionnaire. This will help to ensure that you will still have usable data from the questionnaire, even if the respondent stops answering the questions.

Keep questions brief and easy to understand

Brief questions that use simple language minimize the chances that your questions will be misunderstood, making your survey results more useful. Choose the simplest words with the right meaning when writing the questions for your questionnaire. Simple language is easy to read and comprehend, making the completion of your questionnaire less taxing for all respondents. The goal is to extract valuable information from your respondents, not impress them with poetic language. Be brief and direct with your questions, leaving out any unnecessary words and phrases. Short questions are easier to answer because they don't require as much retention of information, and respondents are less likely to be forced to re-read the questions.

Use open-ended questions appropriately

Open-ended questions allow respondents to answer freely using their own words. Closedended questions can be answered using a simple piece of information, such as a 'yes' or 'no' answer or a selection from multiple choices. The advantage of open-ended questions is that they do not limit respondents and can generate more detailed information. On the down side, open-ended questions can be more taxing to respondents, as they often require more thought and can increase the amount of time required to answer the question. The data generated from open-ended questions can also be more difficult to classify and tabulate once you have collected your responses.

Avoid using leading questions

A leading question is one in which the answer is suggested within the question itself. These questions can make the respondent feel compelled to answer in a particular way, which can be off-putting to the respondent and skew the survey results. Some examples of leading questions are below:

'You like eating at restaurant X, don't you?' 'Why do you like eating at restaurant X more than restaurant Y?'

Avoid using compound questions

Compound questions are two or more questions in one. These questions are problematic because the answer may be different for each part of the question. Examples of compound questions include:

'Have you ever shopped at store X and do you shop there frequently?' 'Do you purchase product X and product Y?'

Avoid using ambiguous questions

Ambiguous questions use words that do not have fixed definitions and are therefore open to a range of interpretations by respondents. Questions that use ambiguous words can produce inconsistent results, as respondents may interpret their meanings in a variety of ways. Avoid being ambiguous about the time period the respondent should consider. Some examples of questions using ambiguous words are below:

'Do you buy product X regularly?' 'Is product X a good product?'

Avoid using unanswerable questions

Unanswerable questions require the respondent to recall information that is impossible to answer with any accuracy, if at all. If your respondents find the questions too difficult to answer, your response rate is likely to suffer. Some examples of unanswerable questions include:

'What is the first restaurant you remember visiting?' 'How many fruits and vegetables have you eaten in the past three months?'

Avoid hypothetical or future intention questions (if possible)

Hypothetical questions force the respondent to provide an answer to something he or she may never have thought about and, therefore, the respondent may not be able to provide an accurate response. An example of a hypothetical question might be:

'If a fruit and vegetable store opened down the street, would you shop there?' 'If your income increased, would you buy more of product X?'

Be careful with response scales

Response scales are often used to assess a respondent's level of agreement or disagreement with a statement, or to assess the respondent's satisfaction with an experience. Response scales can be an excellent way to remove ambiguity from questions and gather data that is easy to tabulate and interpret. There are some pitfalls to be aware of, however, when designing questions that incorporate response scales.

First, when selecting the number of response categories, try not to have so few that respondents' answers fall in between the points, or so many that the values are too ambiguous to produce useful data. 5 and 7 point scales are generally viewed to provide the most valid and reliable responses. Offering a neutral response option as a midpoint appears to enhance the quality of the data that is produced. Try to ensure that the differences between the response categories are roughly equivalent and present the response items in a logical and consistent manner (for example, low to high ratings) to avoid confusing your respondents. You must also make sure that your response categories are mutually exclusive and exhaustive. Mutually exclusive response categories have no overlap between adjacent categories. Exhaustive response categories ensure that there are no gaps between categories and that the high and low categories account for all possible extreme answers.

Always pre-test your questionnaire

Even after taking all possible precautions during the design of your research instrument, problems can occur in the field if it has not been properly tested in advance. There are several testing options available to you. One option is having friends or family members complete your questionnaire. With this option, participants should be easy to recruit. Try to select people who are unfamiliar with the goals of your market research campaign - those who already know what you are up to will likely have some knowledge of the intent of the questions, and may not best represent those who will ultimately be participating in your survey. Also, try to select people who will honestly evaluate your questionnaire and won't hesitate to provide some constructive criticism if necessary. Ideally, you should test your questionnaire with individuals that represent the population you will be targeting with your market research campaign (for example: your customers or residents in the geographic location you serve). Conducting a small pilot test of the questionnaire is one way to get feedback from impartial people. Given that you may not be able to ask these participants questions about the questionnaire itself, this option may not clearly identify problems with survey questions. Problems may have to be inferred based on the results you receive through completed questionnaires, and some problems may slip through the cracks. Perhaps the best way to evaluate your questionnaire is to conduct personal interviews or focus groups with individuals that have taken the survey. Getting these individuals to take time out of their busy lives to test your questionnaire and discuss the questions can be very difficult. You may have to compensate participants in order to recruit them. Interviews and focus groups allow you to determine whether your questions were well understood, easy enough to answer, and whether or not the information requested was too sensitive. These approaches allow you the flexibility to shift gears on the fly or focus on particular problem areas depending upon the feedback you receive. While focus groups can save time by allowing you to get the opinions of several individuals in a single session, certain individuals can be influenced by others in the group, allowing the potential for important information to be missed.

The testing method you choose will likely be influenced by the length and complexity of your questionnaire, the amount of time you have available and your market research budget. In any case, the importance of having your questionnaire put to the test should not be overlooked.