Вы находитесь на странице: 1из 20

Conducting Statewide Customer Satisfaction Surveys

Brad Sickles USDOL / ETA ETA Performance Accountability Team

Measuring Customer Satisfaction


Participant Customer Satisfaction

Employer Customer Satisfaction


Customer ratings on each of the three core questions regarding satisfaction are weighted to account for regional differences and reported on a 1 to 100 scale. Computation of the ACSI is discussed in TEGL No. 6-00. The American Customer Satisfaction Index (ACSI) is used to report statewide results to the Department.

T he T he M easur es M easur es

The Core ACSI Questions


1. Utilizing a scale of 1 to 10 where 1 means Very Dissatisfied and 10 means Very Satisfied what is your overall satisfaction with the services provided from ________? 2. Considering all of the expectations you may have had about the services, to what extent have the services met your expectations? 1 now means Falls Short of Your Expectations and 10 means Exceeds Your Expectations. 3. Now think of the ideal program for people in your circumstances. How well do you think the services you received compare with the ideal set of services? 1 now means Not very close to the Ideal and 10" means Very Close to the Ideal.

Participant Customer Satisfaction


The weighted average of participant ratings on each of the three questions regarding overall satisfaction reported on a 1 to 100 scale. Who: Exiters When: 1to 60 days after date of hard exit or date a soft exit was determined
T he T he M easur es M easur es

How: Telephone survey Required completions: 500 statewide Required response rate: 50% first year; 70% following years

Employer Customer Satisfaction


The weighted average of employer ratings on each of the three questions regarding overall satisfaction reported on a 1 to 100 scale. Who: Employers who receive a substantial service. When: 1to 60 days after date of the completion of a service. For employers who listed a job order where no referrals were made, contact should occur 30 to 60 days after a job order was listed. How: Telephone survey Required completions: 500 statewide Required response rate: 50% first year; 70% following years

T he T he M easur es M easur es

Capturing Customer Satisfaction Results to Assess Local WIBs


States may use the ACSI questions and approach to measure satisfaction at the local level States may also develop their own customer satisfaction instruments and approach for assessing local WIBs Approaches must be applied uniformly and consistently to all WIBs TEGL 6-00 outlines parameters for collecting customer satisfaction results

Revisions to TEGL No. 6-00


Surveys to Obtain State Level Results
Response rates: 50% for the first year of data collection; 70% for following years. Response levels below the specified minimums will invalidate the results. States are required to determine the appropriate sample sizes. States are no longer required to draw minimum sample sizes of 1,000. Respondents must be told responding to the survey is voluntary and the information will be kept confidential. States must create a process for creating an up-to-date customer list, capturing information on each customers address and telephone number.

Revisions to TEGL No. 6-00


Surveys to Obtain State Level Results
Disregard earlier guidance that stated the sampling frame only included participants and employers eligible for the surveys who had valid contact information. A survey is complete when valid answers are provided by the respondents for each of the core ACSI questions. Valid answers are responses 1 through 10. The calculation of the ACSI score includes only the results from complete surveys. The introductory statements for both the participant survey and the employer survey were revised to better set the context for the interviews.

Revisions to TEGL No. 6-00


Surveys to Obtain State Level Results: Additions to Ensure Adequate Response Rates
Local program staff must inform the customer during the registration process about the importance of satisfying customers and the possibility of being contacted for information on his or her experience with the services. The interview should be limited to 15 minutes or less. A minimum of five follow-up attempts is required, involving various times of the day before closing the record.

Revisions to TEGL No. 6-00


Surveys to Obtain State Level Results: Additions to Ensure Adequate Response Rates Local programs should collect alternate contact information. A letter in advance of the survey should be sent out informing the customer that he or she can expect to be contacted about his or her satisfaction with the services.

Creating Questions to Supplement the ACSI


Customer Satisfaction Not the Same as Customer Service
Customer service is measured by standards set by you or your agency. Customer satisfaction is measured by the customers standards for the services, regardless of whether they make sense.

Drivers of Satisfaction
Service Quality Measures

Customer Service Characteristic


Courteous Professional Attentive Friendly Helpful Knowledgeable Prompt Informative Honest Candid

Convenience Accessibility Ease of use Timeliness Safety Reliability Accuracy Thoroughness Fairness Appropriateness Attractiveness and cleanliness

Characteristics of Firms With High Customer Satisfaction


Customers define quality Variety of services Customization of services Convenience of services Timeliness of services Continually identify factors that influence satisfaction Continually identify expectations and set customer service standards well above these expectations (to delight customers)

Writing Effective Questions


Remember your survey's purpose: to provide you feedback on the level of satisfaction. If you have doubts about a question, discard it. State your questions simply. Stay focused when writing questions; avoid vague areas. If a question can be misinterpreted by the respondent, chances are that it will be. Include only one subject per question. Avoid questions that lead respondents. Consider optional ways to ask questions that deal with sensitive areas.

Writing Effective Questions


Create questionnaire items to determine the key drivers of satisfaction

Constructing Effective Response Items


Answer options need to be mutually exclusive and exhaustive. Keep open-ended questions to a minimum. Respondents interpret time-oriented responses differently, such as never, sometimes, and always. Consider a "don't know" responses when you create answer sets. Create meaningful scales for rating services.

Ordering Questions
The first series of questions should be easy for the respondent to understand and should capture his or her attention and interest. Start with general questions and gradually shift to more specific questions. Group questions in logical sections and sequence sections or questions in a well thought out order. Introduce each new section with a summary statement so that participants have an opportunity to adjust to the new set of questions.

Ordering Questions
Position questions about personal or sensitive issues toward the end of the questionnaire. The order of similar items on a list can bias results. Randomly or alphabetically order items and indicate in the instructions how they are ordered, reducing the likelihood that respondents will see the first items as most important of the group. Put demographic questions at the end of the survey form. Seek to minimize the number of times the a skip rule is followed as confusion can result with too many skips or when skips become overly complicated.

Resources
Fowler, F. J., Jr. Survey Research Methods (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc., 1993. Hayes, Bob E. Measuring Customer Satisfaction: Development and Use of Questionnaires. Milwaukee: ASQC Quality Press,1993. Payne, S. L. The Art of Asking Questions. Princeton, NJ: Princeton University Press, 1951. Schuman, H and Presser, S. Questions and Answers in Attitude Surveys: Experiments in Question Form, Wording, and Context. New York: Academic Press,1981. Simply Better! The Voice of the Customer, Second Edition. Washington: U.S. Department of Labor, Employment and Training Administration, 1998. Sudman, S. and Bradburn, N. Asking Questions: A Practical Guide to Questionnaire Design. San Francisco, CA: Jossey-Bass Inc., 1982 Vavra, T. G. Improving Your Measurement of Customer Satisfaction. Milwaukee: ASQC Quality Press, 1997.

Вам также может понравиться