SSI Web v7.

Software for Web Interviewing and Conjoint Analysis (Updated 30 June 2011)

Sawtooth Software, Inc. Sequim, WA http://www.sawtoothsoftware.com

In this manual, we refer to product names that are trademarked. Windows, Windows 95, Windows 98, Windows 2000, Windows XP, Windows NT, Windows Vista, Windows 7, Excel, PowerPoint, and Word are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. SPSS is a trademark of SPSS, Inc. WS_FTP is a trademark of Ipswitch. We express our gratitude to the Apache Group for the development of Apache and mod_perl. We express our gratitude to the Perl community. Portions of lite.pm are copyrighted and used with permission by Shishir Gundavaram. PuTTY's "psftp.exe" is installed with SSI Web and is copyright 1997-2009 Simon Tatham. See license details in the Readme.rtf file. This product includes software developed by vbAccelerator (http://vbaccelerator.com/). Other acknowledgements regarding copyrighted work are in the Readme.rtf file.

Bryan Orme, Editor Copyright 1999-2010 Sawtooth Software

About Technical Support

Weve designed this manual to teach you how to use our software and to serve as a reference to answer your questions. If you still have questions after consulting the manual, we offer telephone support. When you call us, please be at your computer and have at hand any instructions or files associated with your problem, or a description of the sequence of keystrokes or events that led to your problem. This way, we can attempt to duplicate your problem and quickly arrive at a solution. For customer support, contact our Sequim, Washington office at 360/681-2300, email: support@sawtoothsoftware.com, (fax: 360/681-2400). Outside of the U.S., contact your Sawtooth Software representative for support.

Table of Contents
General Web Interviewing (CiW) Help
...................................................................................................................................................... 1 Getting Started ...................................................................................................................................................... 1 Getting Started with Web Interviewing ...................................................................................................................................................... 4 Getting Started: 45-Minute Tour of SSI Web ...................................................................................................................................................... 28 Software Requirements: Users, Servers and Respondents ...................................................................................................................................................... 29 Creating/Opening a Study ...................................................................................................................................................... 30 What's New in V7.0? ...................................................................................................................................................... 32 Composing Web Surveys ...................................................................................................................................................... 35 Composing Questions ................................................................................................................................................................... Adding Questions to Your Survey 35 ................................................................................................................................................................... Setting Page Breaks 38 ................................................................................................................................................................... The Start Question 39 ................................................................................................................................................................... Select Questions 41 ................................................................................................................................................................... Numeric Questions 48 ................................................................................................................................................................... Open-End Questions 51 ................................................................................................................................................................... Text/HTML Filler Questions 54 ................................................................................................................................................................... Terminate/Link Questions 55 ................................................................................................................................................................... Grid Questions 57 ................................................................................................................................................................... Constant Sum Questions 60 ................................................................................................................................................................... Ranking Questions 63 ................................................................................................................................................................... Free Format Questions 66 Introduction ............................................................................................................................................ 66 Radio ............................................................................................................................................ 69 Checkbox ............................................................................................................................................ 70 Select (Combo Box) ............................................................................................................................................ 71 TextArea ............................................................................................................................................ 72 Text (Text Box) ............................................................................................................................................ 73 Text (Numeric) ............................................................................................................................................ 74 Hidden Variables ............................................................................................................................................ 75 ................................................................................................................................................................... JavaScript in SSI Web 76 ...................................................................................................................................................... 84 How CiW License Sizes are Counted ...................................................................................................................................................... 86 What Happens at the End of the Survey? ...................................................................................................................................................... 87 Formatting Options ................................................................................................................................................................... Survey Settings (Style) 87 General Format ............................................................................................................................................ 94 Page Format ............................................................................................................................................ 96 Headers and Footers ............................................................................................................................................ 97 Progress Bar ............................................................................................................................................ 98 Browser Settings ............................................................................................................................................ 99 Error Messages ............................................................................................................................................ 100 Server Settings ............................................................................................................................................ 103 Advanced ............................................................................................................................................ 105 ................................................................................................................................................................... HTML Elements to Enhance Your Survey 107 ................................................................................................................................................................... HTML Cheat-Sheet 109

................................................................................................................................................................... Including Graphics 113 ................................................................................................................................................................... Graphics Management 114 ................................................................................................................................................................... Randomizing Questions/Pages/Blocks 115 ...................................................................................................................................................... 117 Skip Logic ...................................................................................................................................................... 122 List Building ...................................................................................................................................................... 143 SSI Script ................................................................................................................................................................... Displaying Responses Given to Previous Questions 145 ................................................................................................................................................................... Functions 147 List Functions ............................................................................................................................................ 148 System Functions ............................................................................................................................................ 151 Quota Control Functions ............................................................................................................................................ 155 Math Functions ............................................................................................................................................ 156 String Functions ............................................................................................................................................ 157 Error Functions ............................................................................................................................................ 158 CBC Functions ............................................................................................................................................ 159 ACA Functions ............................................................................................................................................ 160 CVA Functions ............................................................................................................................................ 162 MaxDiff Functions ............................................................................................................................................ 163 ACBC Functions ............................................................................................................................................ 164 ................................................................................................................................................................... More Detail on Boolean (True/False) Functions 166 ................................................................................................................................................................... Additional Details on Certain Functions 168 ................................................................................................................................................................... Generating Random Numbers 173 ................................................................................................................................................................... Unverified Perl 174 ...................................................................................................................................................... 196 Surveys in Non-English Languages ...................................................................................................................................................... 199 Print Survey ...................................................................................................................................................... 200 Respondent Passwords ...................................................................................................................................................... 200 Introduction to Passwords ...................................................................................................................................................... 201 Start Question Layout ...................................................................................................................................................... 202 Password Fields ...................................................................................................................................................... 204 Merged Fields ...................................................................................................................................................... 205 Adding/Importing Password Data ...................................................................................................................................................... 208 Pass-In Fields ...................................................................................................................................................... 209 Cookies and User-Defined Passwords ...................................................................................................................................................... 211 Modifying Passwords When in Field ...................................................................................................................................................... 212 Quota Control ...................................................................................................................................................... 212 Introduction to Quota Control ...................................................................................................................................................... 216 Quota Dialog ...................................................................................................................................................... 219 Quota Control Settings ...................................................................................................................................................... 220 Predictive Quota Control ...................................................................................................................................................... 222 Prequalification in Quota Control ...................................................................................................................................................... 223 Quota Changes Mid-Study ...................................................................................................................................................... 224 Testing/Running the Survey Locally ...................................................................................................................................................... 224 Testing/Running the Survey Locally ...................................................................................................................................................... 226 Survey Installation on Web Server ...................................................................................................................................................... 226 Uploading Survey to Web Server ...................................................................................................................................................... 229 Manual Upload to Server ...................................................................................................................................................... 234 Uploading Survey to Microsoft IIS Servers ...................................................................................................................................................... 235 Deleting Practice Data Prior to Taking the Study "Live"

...................................................................................................................................................... 236 Troubleshooting Server-Side Problems ...................................................................................................................................................... 238 How Respondents Access the Web Survey ...................................................................................................................................................... 238 How Respondents Access the Web Survey ...................................................................................................................................................... 239 "One-Click" Respondent Access to Survey ...................................................................................................................................................... 241 Linking to Sawtooth Software's Web Survey from Another Survey Program ...................................................................................................................................................... 243 Making Changes after Fielding Begins ...................................................................................................................................................... 243 Making Changes after Fielding Begins ...................................................................................................................................................... 245 CAPI Interviewing with "SSI Web CAPI" ...................................................................................................................................................... 245 CAPI Interviewing with "SSI Web CAPI" ...................................................................................................................................................... 248 Accumulating CAPI Files ...................................................................................................................................................... 249 Deleting Sensitive Data from CAPI Interviewing PCs ...................................................................................................................................................... 250 Managing Survey Data ...................................................................................................................................................... 250 How Data Are Stored ...................................................................................................................................................... 251 Admin Module ................................................................................................................................................................... Setting Passwords for Administrative Access 252 ................................................................................................................................................................... Logging On 253 ................................................................................................................................................................... Downloading Survey Data 255 ................................................................................................................................................................... Viewing Survey Data 256 ................................................................................................................................................................... Marginals: Tabulating Data On-Line 258 ................................................................................................................................................................... Downloading Password Report 261 ................................................................................................................................................................... Incompletes Report 262 ................................................................................................................................................................... Test Survey Mode 263 ................................................................................................................................................................... Reset Web Survey 265 ................................................................................................................................................................... Customizing the Administrative Module 266 ...................................................................................................................................................... 268 Exporting/Analyzing Data ...................................................................................................................................................... 268 Exporting Data ...................................................................................................................................................... 269 Selecting Data to Export ...................................................................................................................................................... 271 Additional Data Formatting Settings ...................................................................................................................................................... 273 Exporting Open-End Data ...................................................................................................................................................... 274 Analyzing Data with Online Simulator ...................................................................................................................................................... 275 Moving Data from SSI Web to SMRT for Analysis ...................................................................................................................................................... 278 Files created by SSI Web ...................................................................................................................................................... 278 Files created by SSI Web

CBC Help
...................................................................................................................................................... 281 Getting Started with CBC ...................................................................................................................................................... 281 Overview of CBC Software ...................................................................................................................................................... 283 What is Choice-Based Conjoint? ...................................................................................................................................................... 286 CBC Tutorial and Example ...................................................................................................................................................... 325 Designing CBC Studies ...................................................................................................................................................... 325 CBC Questionnaires and Designs ...................................................................................................................................................... 331 Defining Attributes and Levels ...................................................................................................................................................... 334 Specifying Prohibitions ...................................................................................................................................................... 337 Attribute Interactions ...................................................................................................................................................... 338 Number of Attributes/Levels/Tasks in CBC ...................................................................................................................................................... 339 None Option/Dual-Response None ...................................................................................................................................................... 341 Specifying Fixed or Holdout Tasks

...................................................................................................................................................... 343 Specifying CBC Design Parameters ...................................................................................................................................................... 347 Testing the CBC Design ...................................................................................................................................................... 354 Importing/Exporting CBC Designs ...................................................................................................................................................... 357 Advanced CBC Designs ...................................................................................................................................................... 357 Conditional Pricing for CBC ...................................................................................................................................................... 360 Conditional Display ...................................................................................................................................................... 361 Alternative-Specific Designs ...................................................................................................................................................... 365 Partial-Profile Designs ...................................................................................................................................................... 368 Shelf-Facing Display ...................................................................................................................................................... 371 Customized CBC Questions Using "Free Format" ...................................................................................................................................................... 375 Volumetric CBC ...................................................................................................................................................... 376 CBC Paper-and-Pencil Studies ...................................................................................................................................................... 378 Analyzing the Data ...................................................................................................................................................... 378 Preparing Conjoint Data for SMRT ...................................................................................................................................................... 380 Moving Data from SSI Web to SMRT for Analysis ...................................................................................................................................................... 383 .CHO File Format ...................................................................................................................................................... 385 .CHS File Format

Adaptive CBC (ACBC) Help

...................................................................................................................................................... 387 Getting Started with ACBC ...................................................................................................................................................... 387 Welcome to Adaptive CBC! ...................................................................................................................................................... 388 ACBC Specifications ...................................................................................................................................................... 389 Overview ...................................................................................................................................................... 391 Motivation for Adaptive CBC ...................................................................................................................................................... 393 Sections and Flow for Adaptive CBC ...................................................................................................................................................... 402 Design Tab (ACBC) ...................................................................................................................................................... 405 Attributes and Levels for Adaptive CBC ...................................................................................................................................................... 407 Price in Adaptive CBC ...................................................................................................................................................... 409 Hands-On Tutorials ...................................................................................................................................................... 410 Tutorial #1 (Beaches) ...................................................................................................................................................... 421 Tutorial #2 (Pianos) ...................................................................................................................................................... 429 ACBC Design Algorithms ...................................................................................................................................................... 429 How Concepts Are Chosen ...................................................................................................................................................... 434 Deviation from Target Frequencies ...................................................................................................................................................... 435 Advanced ACBC Surveys ...................................................................................................................................................... 435 Customized (Constructed) Attribute and Level Lists ...................................................................................................................................................... 439 Price Adjustments (Within Summed Pricing) ...................................................................................................................................................... 441 Conditional Graphics ...................................................................................................................................................... 444 Unverified Perl to Customize ACBC Questions ...................................................................................................................................................... 445 Checklist Prior to Fielding ...................................................................................................................................................... 445 Checklist Prior to Fielding ...................................................................................................................................................... 446 Analysis of ACBC Data ...................................................................................................................................................... 446 Counting Analysis for ACBC ...................................................................................................................................................... 449 Coding the Information for Parameter Estimation ...................................................................................................................................................... 450 Modeling the Price Function ...................................................................................................................................................... 454 HB Estimation ...................................................................................................................................................... 457 Interaction Effects

...................................................................................................................................................... 458 Utility Constraints, Including Customized Constraints ...................................................................................................................................................... 461 Monotone Regression (Pure Individual Analysis) ...................................................................................................................................................... 465 Analysis with Tiny Sample Sizes ...................................................................................................................................................... 467 None Threshold & Calibration to Purchase Likelihood ...................................................................................................................................................... 470 Exporting ACBC Data to .CHO File ...................................................................................................................................................... 472 ACBC: How Well Does It Work? ...................................................................................................................................................... 476 References

MaxDiff Help
...................................................................................................................................................... 477 Getting Started with MaxDiff ...................................................................................................................................................... 477 Introduction ...................................................................................................................................................... 478 Motivation for MaxDiff ...................................................................................................................................................... 481 What is MaxDiff? ...................................................................................................................................................... 482 MaxDiff Tutorial and Example ...................................................................................................................................................... 508 Creating a MaxDiff Experiment ...................................................................................................................................................... 509 Question and Label Text ...................................................................................................................................................... 512 Question Format ...................................................................................................................................................... 513 Developing a List of Items ...................................................................................................................................................... 516 Designing the Study ...................................................................................................................................................... 521 Analyzing MaxDiff Data ...................................................................................................................................................... 522 Counting Analysis ...................................................................................................................................................... 524 Individual-Level Score Estimation ...................................................................................................................................................... 530 Identifying "Bad" Respondents ...................................................................................................................................................... 531 Customized MaxDiff Questions with "Free Format" ...................................................................................................................................................... 534 Computing Scores On-The-Fly ...................................................................................................................................................... 535 Paper-and-Pencil Studies ...................................................................................................................................................... 537 MaxDiff Analysis Using CBC/HB v5.x ...................................................................................................................................................... 539 MaxDiff Analysis Using Latent Class v4 ...................................................................................................................................................... 542 Robustness of MaxDiff Designs to Prohibitions

ACA Help
...................................................................................................................................................... 549 Getting Started with ACA ...................................................................................................................................................... 549 Getting Started with ACA ...................................................................................................................................................... 550 What is ACA? ...................................................................................................................................................... 552 Basic Overview of ACA ...................................................................................................................................................... 553 ACA Tutorial and Example ...................................................................................................................................................... 590 Sections and Flow of ACA Questionnaires ...................................................................................................................................................... 596 Defining Attributes and Levels ...................................................................................................................................................... 599 Specifying Prohibitions in ACA ...................................................................................................................................................... 600 Setting ACA Control Parameters ...................................................................................................................................................... 603 How ACA Data Are Stored ...................................................................................................................................................... 604 Computing Utilities and Analyzing ACA Data ...................................................................................................................................................... 606 Is ACA the Appropriate Technique? ...................................................................................................................................................... 608 Staying out of Trouble with ACA ...................................................................................................................................................... 610 Advanced ACA Questionnaires ...................................................................................................................................................... 610 Omitting the "Importance" Question ...................................................................................................................................................... 612 Customizing the "Importance" Question

...................................................................................................................................................... 614 Algorithms and Estimation Techniques ...................................................................................................................................................... 614 Details of ACA/Web Utility Estimation ...................................................................................................................................................... 617 Choosing the Next Paired-Comparison Question ...................................................................................................................................................... 619 Updating Utilities During the Interview ...................................................................................................................................................... 621 ACA/HB Estimation ...................................................................................................................................................... 622 ACA File Formats ...................................................................................................................................................... 622 Format of .UTL and .ACD Files ...................................................................................................................................................... 627 Real-Time Feedback to Respondents ...................................................................................................................................................... 629 Moving Data from SSI Web to SMRT for Analysis ...................................................................................................................................................... 632 Analyzing ACA Data with the Market Simulator

CVA Help
...................................................................................................................................................... 633 Getting Started with CVA ...................................................................................................................................................... 633 Introduction to CVA ...................................................................................................................................................... 637 CVA Tutorial and Example ...................................................................................................................................................... 676 Is CVA the Appropriate Technique? ...................................................................................................................................................... 677 Designing/Creating a CVA Study ...................................................................................................................................................... 677 Attributes and Levels ................................................................................................................................................................... Defining Attributes and Levels 677 ................................................................................................................................................................... Level Prohibitions and CVA Studies 680 ...................................................................................................................................................... 681 Designing the CVA Tasks (Questions) ................................................................................................................................................................... Introduction to CVA Questionnaires 681 ................................................................................................................................................................... Advantages and Challenges of CVA Questionnaires 683 ................................................................................................................................................................... Selecting the Number of Tasks 684 ................................................................................................................................................................... Throw out "Obvious" Tasks 685 ................................................................................................................................................................... CVA Design Strategy 686 ................................................................................................................................................................... Testing the CVA Design 690 ................................................................................................................................................................... Importing/Exporting CVA Designs 692 ................................................................................................................................................................... Including Holdout Choice Tasks in CVA Questionnaires 696 ...................................................................................................................................................... 697 CVA/Web Paper and Pencil Studies ...................................................................................................................................................... 697 CVA/Web Paper and Pencil Studies ...................................................................................................................................................... 699 CVA Advanced Designs ...................................................................................................................................................... 699 Conditional Display for CVA ...................................................................................................................................................... 700 Conditional Pricing for CVA ...................................................................................................................................................... 703 Modeling Interaction Effects with CVA ...................................................................................................................................................... 705 Sparse Designs and Large Samples ...................................................................................................................................................... 707 Data Analysis ...................................................................................................................................................... 707 Estimating Part-Worth Utilities ...................................................................................................................................................... 710 CVA Ordinary Least Squares Settings ...................................................................................................................................................... 713 CVA Monotone Regression Settings ...................................................................................................................................................... 714 Technical Details ...................................................................................................................................................... 714 The CVA Designer ...................................................................................................................................................... 715 How CVA Calculates Utilities ...................................................................................................................................................... 719 Avoiding Linear Dependency ...................................................................................................................................................... 721 The CVA/HB Module for HB Estimation ...................................................................................................................................................... 721 What is CVA/HB (hierarchical Bayes for CVA)? ...................................................................................................................................................... 723 Running CVA/HB

...................................................................................................................................................... 726 Using Constraints in CVA/HB

Appendices for SSI Web

...................................................................................................................................................... 729 A: Interpreting Conjoint Analysis Data ...................................................................................................................................................... 732 B: Holdout Choice Tasks in Conjoint Studies

SMRT (Market Simulator) Help

...................................................................................................................................................... 735 What Is SMRT? ...................................................................................................................................................... 736 Data and Utility Run Management ...................................................................................................................................................... 736 Importing Utility Runs ...................................................................................................................................................... 739 Merging Segmentation Data ................................................................................................................................................................... Custom Segmentation Variables 741 ................................................................................................................................................................... Weighting Respondents 742 ...................................................................................................................................................... 743 Data Management Tools ...................................................................................................................................................... 746 Tables Program ...................................................................................................................................................... 750 Using Simulators to Answer Strategic Questions ...................................................................................................................................................... 750 Overview ...................................................................................................................................................... 751 What Is a Market Simulation? ...................................................................................................................................................... 752 Why Conduct Market Simulations? ...................................................................................................................................................... 754 Typical Questions for Conjoint Simulators ...................................................................................................................................................... 755 Market Simulation Theory and Models ...................................................................................................................................................... 755 Base Case Scenario ...................................................................................................................................................... 756 The Exponent (Scale Factor) ...................................................................................................................................................... 757 The Red-Bus/Blue-Bus Problem ...................................................................................................................................................... 758 Market Simulator Models ...................................................................................................................................................... 761 External Effects ...................................................................................................................................................... 763 Interpolating between Levels ...................................................................................................................................................... 764 The "None" Weight ...................................................................................................................................................... 765 Using the Market Simulator ...................................................................................................................................................... 765 The Market Simulator Dialog ...................................................................................................................................................... 767 Adding a Simulation Scenario ...................................................................................................................................................... 769 Other Controls in Simulation Scenarios ...................................................................................................................................................... 771 Running the Simulation ...................................................................................................................................................... 772 Practical Simulation Examples ...................................................................................................................................................... 772 Introduction ...................................................................................................................................................... 773 New Product Introductions ...................................................................................................................................................... 775 Estimating Demand Curves and Elasticities ...................................................................................................................................................... 778 Designing Products to Appeal to Unique Market Segments ...................................................................................................................................................... 781 Technical Details for Simulations ...................................................................................................................................................... 781 First Choice Method ...................................................................................................................................................... 782 Share of Preference Options ...................................................................................................................................................... 785 Purchase Likelihood Option ...................................................................................................................................................... 786 Randomized First Choice ...................................................................................................................................................... 790 CBC Analysis: Counts and Logit ...................................................................................................................................................... 790 Counting Analysis for CBC ...................................................................................................................................................... 794 Logit Analysis for CBC ...................................................................................................................................................... 802 Appendices for Simulation Topics

...................................................................................................................................................... 802 Importing Conjoint Data from Text-Only Files ...................................................................................................................................................... 806 How the Simulator Interpolates between Levels ...................................................................................................................................................... 808 How the Simulator Uses Latent Class Data

Index

809

General Web Interviewing (CiW) Help

1
1.1
1.1.1

General Web Interviewing (CiW) Help

Getting Started
Getting Started with Web Interviewing
Welcome to SSI Web! SSI Web is a software system for writing questionnaires and conducting market research studies over the Internet or via CAPI-based (stand-alone PC) interviewing. You can press F1 at most any dialog/screen within SSI Web to receive help about the options within that dialog. A complete copy of the help documentation with a table of contents is available within the PDF file entitled SSI Web Documentation.pdf that was installed with your SSI Web system. We suggest you start by viewing Getting Started with SSI Web: A 45-Minute Hands-On Tour. SSI Web is made up of various components that you can license from Sawtooth Software: CiW is the core interviewing component, a general-purpose survey research tool suited for a variety of simple to complex survey projects. CBC is for Choice-Based Conjoint analysis, currently the most popular trade-off method. ACBC is for Adaptive Choice-Based Conjoint, a trade-off method that is particularly well suited for problems involving five or more attributes. MaxDiff is for Maximum Difference (Best/Worst) scaling. This technique is for measuring the importance or preference of a list of items. It shares much in common with conjoint analysis, but it is applicable to a wider variety of situations and is easier than conjoint analysis. CVA is for traditional, full-profile conjoint analysis. This is the original conjoint method, and it particularly useful for small sample sizes and problems with few attributes. ACA is for Adaptive Conjoint Analysis, a trade-off technique for product design and segmentation studies involving many attributes. Sample CiW, ACBC, CBC, MaxDiff, CVA, and ACA studies and supporting files are included with your installation in the /Samples directory. You can easily access these example studies by clicking Help | Sample Studies. These questionnaires let you see examples and investigate how they are constructed. If you know a little HTML (which is not necessary to use this software), you can employ some "tricks" that can help you customize the look of your survey beyond the standard options. This is especially the case if using the "Free Format" question type. By opening the sample studies and looking at the settings and text we've used, you can become familiar with some of the possibilities. There is also a brief section available in the online help regarding HTML elements. Using this web-based survey tool requires the following skills and knowledge base: 1. The analyst managing the survey should know the basics of survey writing. If using conjoint analysis, the analyst should understand how to define attributes and levels properly, correctly phrase the questionnaire text, interpret the conjoint utilities, and run simulations. It is not necessary to be a statistician to use our software effectively, but knowledge of basic statistics for market research is highly recommended. 2. The questionnaire is first composed locally on a PC running under Windows XP Service Pack 2 or later, using your web browser to preview/test the survey. The individual inputting/formatting the survey should have a working knowledge of standard Windows applications such as word processing and web browsers. Though it is not necessary, some knowledge of HTML is

SSI Web v7 valuable. To post your survey on the Web, you'll eventually upload your survey to a Web server (or email the files to Sawtooth Software, if using our hosting services). 3. To upload the survey to the Web, you must have access to an Internet/intranet server running Perl 5.003 or later. This can be your company's own Web server, or your ISP's (Internet Service Provider's) Web server. SSI Web manages the communication, file transfer, and project setup on your web server. Note: Sawtooth Software also provides web hosting services for licensed SSI Web customers. Please call us for more information.

Steps to Creating a Web Survey 1. It is likely that you will first compose the script to your questionnaire using a word processing package. This is fine as a starting point, as you can cut-and-paste text from your word processing package into the questions you develop within SSI Web's survey building interface. 2. Create a new study in SSI Web. 3. Next, write your questionnaire, adding questions one-by-one within the Write Questionnaire interface. You can preview these questions using the Preview function, that shows how your Internet browser presents your questions. 4. Optionally, create respondent passwords, to provide a way to let qualified respondents in and keep unqualified people out of the survey. 5. Preview the questions, questionnaire pages, or the entire questionnaire on your PC during questionnaire development. You can also test or run your questionnaire locally on your PC using the Local Test Server installed with SSI Web. 6. When you are satisfied with your questionnaire, upload your project to the Web server (or use our hosting services). You then test your survey on the Web server, to make sure it functions properly and saves the data correctly. 7. When you are satisfied that your survey looks and functions properly, invite respondents to take the survey. The data are automatically stored on the server. You can view or perform tabulations of the data in real time at any point during data collection. 8. Download the data to your PC and use the data export capabilities within SSI Web to convert the survey data to a generic text-only format, .csv format, Excel format, or SPSS format. 9. If running a conjoint analysis project (e.g. CBC, ACBC, ACA or CVA), use the menu system to prepare the appropriate conjoint files for analysis within the SMRT system or within Sawtooth Software Online Market Simulator.

Technical Support Your license entitles you to free technical support via phone or email. We encourage you to contact us at support@sawtoothsoftware.com or 360/681-2300 if you cannot find the answer to your problem or question in this documentation. We are available Mon-Fri, 6:30 am to 4:30 pm Pacific time. Outside of regular business hours, you might try posting a question to our Online Forum at our website, www.sawtoothsoftware.com/forum.php. We provide full technical support for composing and testing your Internet survey on your local PC, and

General Web Interviewing (CiW) Help

provide instructions on how to upload your project to your server. We cannot take responsibility for or be expected to provide technical support to debug any customized HTML, CSS, Perl, or JavaScript you may add to your survey. You are also responsible to make sure that Perl is running and that you can run Perl (CGI) scripts on your web server. We can provide some assistance related to SSI Web's use with your server once the fundamental Perl functionality is in place.

Privacy and Research There is a great deal of concern regarding privacy rights and the Internet. Researchers must be respectful of respondents and the valuable information they provide. One should never use information collected during the course of research for purposes other than research. We suggest providing a privacy policy near the front of your survey, to let respondents know how the data they provide will or won't be used. We condemn direct marketing, selling, list building, or PR campaigns that masquerade as research.

SSI Web v7

1.1.2

Getting Started: 45-Minute Tour of SSI Web

What is SSI Web?
SSI Web is a software system for creating Web-based questionnaires. These questionnaires can include standard survey questions, such as "What is your primary occupation?" or "How many people live in your household?". Other SSI Web studies may include specialized research techniques such as Maximum Difference Scaling (MaxDiff), and conjoint (trade-off) analysis, including Choice-Based Conjoint (CBC), adaptive Choice-Based Conjoint (ACBC), CVA (traditional full-profile conjoint), and Adaptive Conjoint Analysis (ACA). The standard survey questions and conjoint questions are developed using the same SSI Web system, and are integrated within the same continuous survey. CBC, ACBC, MaxDiff, CVA, and ACA are just specialized components within the SSI Web system.

SSI Web stands for "Sawtooth Software Incorporated" and "Web" refers to the ability to create Webbased surveys.
This 45-minute tour focuses on general surveys rather than conjoint analysis. Even if your primary intent is to use SSI Web for its conjoint capabilities, you should find this tour useful, as we cover many topics that are essential to all SSI Web surveys, whether including conjoint questions or not.

The Big Picture

The SSI Web system should be installed onto your own desktop computer. (You can install a demo version of SSI Web from http://www.sawtoothsoftware.com/downloads.shtml ). SSI Web provides a simple point-and-click interface for writing questionnaires and testing themall on your own computer. For example, you can make sure that any skip patterns, randomizations, and question verification (such as whether respondents filled out all the required questions on a page) are working properly on your desktop PC.

You do not need to install the full SSI Web system on the Web Server. After you finish writing your questionnaire on your PC, you just need to upload the project to the Web Server (SSI Web handles the

General Web Interviewing (CiW) Help

details of this once you give it the proper URL and login protocol for your web server). This Web Server can be your own company's server, our servers (if you use our hosting service), or a server that you "rent" from a third party provider, such as an ISP (Internet Service Provider). By the way, we don't charge you any per-project or per-respondent interviewing fees if you use your own or a "rented" server. You are free to interview as many people (who browse/connect to the Web Server) and conduct as many projects as you like.

If you don't want to deal with hosting issues or Web Server set-up, you can let us host your survey. Fees start as low as $250 per survey.
If you want to read more detail about how SSI Web projects are configured on a web server, click Help | SSI Web Help | General Web Interviewing (CiW) Help | Survey Installation on Web Server. Alternatively, visit http://www.sawtoothsoftware.com/products/ssiweb/howssiwebworks.shtml.

Starting a New SSI Web Study

From this point on, we'll assume you have installed the SSI Web system on your computer. (You can install a demo version of SSI Web at http://www.sawtoothsoftware.com/downloads.shtml ). Start the SSI Web system by clicking Start | Programs | Sawtooth Software | Sawtooth Software SSI Web. If you are starting the system for the first time, SSI Web will ask you if you want to start a new study (otherwise, click File | New Study.) Go ahead and start a new study, with a studyname no longer than twenty-four characters long (e.g. "mydemo"). Note that this name appears within in the address field and in files you use on the internet, so you probably shouldn't use words that reveal something about your project that you wouldn't want revealed, such as your client's name. Next, choose a folder on your hard drive to store your project. Browse to the folder (by clicking the Browse button) and (if needed) create a new folder using the "Make New Folder" button on the dialog. Finally, select what type of study to create. For the purposes of this tour, select CiW Study.

SSI Web v7 When you start a new study (or open an existing study), the main menu is shown:

The first thing you should know is that while at any screen in SSI Web, you can press F1 to receive help related to the items on that screen. You can also receive help by clicking Help from the main menu and then by choosing SSI Web Help or alternatively by clicking the ? button on the toolbar.

Writing a Simple Survey

From this point onward, we'll assume you've installed the SSI Web system, and created a new study on your hard drive. Let's consider a simple questionnaire:

General Web Interviewing (CiW) Help Welcome to our survey! Today we are interested in ice cream preferences. Please type your user name and password and then click the "Next" button to continue -------------------------------------<Page Break>--------------------------------------Q1. Please tell me how often you typically eat ice cream Never Rarely Sometimes Often -------------------------------------<Page Break>--------------------------------------Q2. In the past week, how many times have you eaten ice cream? ______ times Q3. Which of these flavors do you often eat? (Select all that apply) <Randomize list> Chocolate Strawberry Vanilla Rocky-Road Peach Neapolitan

-------------------------------------<Page Break>--------------------------------------Q4. Please describe the last occasion in which you ate ice cream -------------------------------------<Page Break>--------------------------------------Thank you very much. That's all we need to know today.

SSI Web v7 To write this survey, click Compose | Write Questionnaire.... The Write Questionnaire dialog is shown:

The panel displays the list of questions that are currently in your survey. SSI Web automatically adds a Start "question" on the first page of your survey. To modify the Start question, highlight it within the List of Questions using the mouse, and click the Edit button (or double-click the question).

General Web Interviewing (CiW) Help The Questionnaire Access and Passwords dialog is shown:

This dialog lets you specify what the respondent sees on the first page of the survey, and also lets you specify respondent passwords if you want to include passwords in your survey. Each question in SSI Web has different text areas where you can directly type text to show on the screen (and optional HTML). The three main areas for the introductory Start question are: Header 1 Header 2 Footer

10

SSI Web v7 These text areas are arranged on the browser screen (if you were to preview the question) as:

"Header 1" "Header 2" and "Footer" provide an easy way for you to type information into SSI Web and have these sections separated by a blank line. Let's add the introductory text for our ice cream questionnaire. Within the "Header 1" field, type the text: Welcome to our survey! Today we are interested in ice cream preferences. Please type your user name and password and then click the "Next" button to continue

Note that you can directly type the text into the "Header 1" entry field, or you can click the "Pencil" icon to open a larger window for typing/editing text.

General Web Interviewing (CiW) Help After you have done this, the dialog should look like:

11

Create User Names and Passwords

We want to establish usernames and passwords for respondents in this study (it is optional to set up passwords for respondents). Click the Password Fields tab. The table on this tab is where you specify any passwords respondents must specify prior to accessing the survey. You can add many password fields, but it is typical to include just one or two password fields. In our survey, we'll be using a username and a password. We'll name the variables (fields) myusername and mypassword. Edit the first row of the table, inserting the name of the password variable (myusername) in the first column. If you want the text to appear as dots when respondents type (to mask their password so that others viewing the screen cannot read what is typed), click the Masked box. If you want the passwords to be case sensitive (so that respondents have to use capital and lower case letters exactly as specified), then click the Case Sensitive box. Specify the text that you want shown next to the myusername entry box in the Row 1 Password Label area. We'll use the text "User Name:". When you have finished supplying the settings for the myusername password variable, the table looks like:

12

SSI Web v7

To add a new password variable, type mypassword in the second row of the table. Let's mask mypassword when respondents type in their password, by clicking the Masked box in the second row of the table. Let's also add the label: "Password:" in the Row 2 Password Label field. After you've specified the mypassword row in the table, it should look like:

Next, click the Preview button at the bottom of the dialog.

General Web Interviewing (CiW) Help

13

A small browser window opens to give a representation of what this question would look like to your respondents:

Perhaps we would prefer to bold the text in the opening paragraph. Click OK in the Preview window to retire that window. Click the Start Question Text tab. To the right of the "Header 1" field, click the "paintbrush" icon , use the drop-down box to change bold to "Yes" (to override the existing style settings for this question) and click Preview. The question now appears as:

(Alternatively, for those that know some HTML, you could have enclosed the text you typed into the Header 1 field within <b></b> tags to accomplish the same effect.) Most surveys use a unique user name/password for each respondent. For simplicity sake, we'll use a single User Name and Password combination for all respondents in this practice survey. From the Questionnaire Access and Passwords, Password Fields tab, click the Password Data... icon:

First, you are asked if you'd like to save your work so far. Click Save and Continue. A table opens that looks a bit like a spreadsheet. This table displays the contents of your passwords file. Edit this table to add a User Name of "sawtooth" and a Password of "software" with 1000 total respondents allowed to use that password. When finished, the screen should look like:

14

SSI Web v7

Earlier, when we first created the two password field variables, we had specified that the username and password were not case sensitive, so respondents can access the survey by typing "sawtooth" and "software" with either upper- or lower-case letters. Each time you add a new row (a new myusername and mypassword combination), an empty row appears at the bottom of the table. A million or more password combinations could be specified, if needed. You can Auto-Generate... many random password combinations, or you could Import... password information from an existing text file. For simplicity sake, we have allowed 1000 respondents to use the same password combination. In most studies, each respondent receives a unique password combination (Max Respondents=1). With unique passwords for each respondent, respondents that terminate early can return and restart the survey where they left off.

Adding Questions
Now let's add question 1. Click Close to close the passwords file, and click OK to close the Questionnaire Access and Passwords dialog to return to the Write Questionnaire screen. The question to be added is: Q1. Please tell me how often you typically eat ice cream Never Rarely Sometimes Often

Whenever you add a question to the List of Questions, the new question is placed directly below

General Web Interviewing (CiW) Help

15

whatever question (or page break) is highlighted. If no question is highlighted, then the new question is placed at the bottom of the list. If you want to rearrange the questions within the list, you can highlight them and use the "Up" and "Down" arrows next to the list. Or, you can cut, copy and paste questions by using the cut, copy and paste icons. We want the first question to appear below the first page break. Highlight the page break on the list of questions, then click the Add button. The Add to Questionnaire dialog is shown:

First, choose the question type from the radio buttons shown. This is a "Select-type" question, so click Select. Next, specify a name for the question in the Name field. Type "Q1" and click OK.

16

SSI Web v7 The Select Question dialog is displayed:

This question requires just a single response (a radio button), so you should click the Settings tab, and select Radio button (single response) in the upper-left hand corner of this dialog (it is already selected by default). Back on the Question Text tab, in the Header 1 Text field, type: Q1. Please tell me how often you typically eat ice cream

Next, let's specify the response options:

Never Rarely Sometimes Often

Click the Response Options tab. On the Response Options tab, click Add..., and type the first response "Never". Press the ENTER key twice, and type the next item "Rarely." Repeat for

General Web Interviewing (CiW) Help "Sometimes" and "Often."

17

(Time-Saving Hint: Many users already have their questionnaires prepared in Microsoft Word. You can directly cut and paste a list of response options into SSI Web by highlighting the array of items in Microsoft Word or Excel, pasting to the clipboard using Ctrl+C or the Copy button in Microsoft Word or Excel (or another text editor), and then (within SSI Web) by clicking the Paste from word processor or spreadsheet icon .)

Click OK to return to the Select Question dialog, and click Preview: Your question should look like:

From SSI Web, Click OK to return to the Write Questionnaire dialog. We hope by now you are starting to get a feel for adding questions to SSI Web. We won't give you click-by-click details for the next three questions. Instead, we'll give you some bullet-pointed suggestions. Q2. In the past week, how many times have you eaten ice cream? ______ times

This is a "Numeric" question Type the "times" label in the Question text area From the Settings tab, specify a Minimum value of 0 and Maximum of 24 Specify to position the question text ("times") Right of input box.

When you are done and click Preview, the question should look like:

18

SSI Web v7

Next comes Q3: Q3. Which of these flavors do you often eat? (Select all that apply) <Randomize list> Chocolate Strawberry Vanilla Rocky-Road Peach Neapolitan

This is a "Select" type question From the Settings tab, choose Check Box (multiple response) Also from the Settings tab, for Number of Columns, specify 2 When at the Response Options tab, make sure to click Randomize Response Options

When you are done, click Preview and the question should look like (notice the randomized order):

General Web Interviewing (CiW) Help Next comes Q4: Q4. Please describe the last occasion in which you ate ice cream

19

This is an "Open-end" type question

When you are done, click Preview and the question should look like:

The final part of the survey is a sign-off screen thanking respondents and letting them know they are done. This really isn't a "question" in the sense that it doesn't require a response, but for the purposes of adding it to the list, SSI Web considers it a Terminate/Link "question": Thank you very much. That's all we need to know today.

This is a "Terminate/Link" type question You can name the question "Finish" From the Settings tab, make sure to check the Terminate Respondent box

When you are done and click Preview, the question should look like:

20

SSI Web v7

The "Powered by Sawtooth Software, Inc." is shown on the last page of the survey, but you can turn this message off if you would like, under the Survey Settings | General Format area accessed by clicking from the Write Questionnaire dialog (we'll discuss this dialog more at a later point).

Page Layout and Skip Patterns

While you were inserting questions into your question list, you may have also been adding page breaks by clicking the Add Page Break button from the Write Questionnaire dialog. If you have not already added page breaks between questions, we are ready to divide them into separate pages and add any skip patterns. Go to the Write Questionnaire dialog. Note that the Start question is on Page #1. If you haven't already inserted page breaks, all other questions are on Page #2. The original questionnaire instructions called for: Q1 to be followed by a page break Q2 and Q3 on the same page Q4 on its own page The termination screen at the very end, on its own page

This is actually quite easy to accomplish. With the mouse, highlight Q1 on the question list, then click Add Page Break. Add a page break after Q3 and Q4. When you are done, your question list should look like:

General Web Interviewing (CiW) Help

21

Now for the skip pattern. We know that if respondents answer item 1 (never eat ice cream) in Q1, they should be terminated. We therefore want to set a skip pattern with the following logic: From Q1, Skip To Finish If Q1=1 To do this, from the Write Questionnaire dialog, click Skip Logic. The Page Skip Logic dialog is shown. Click Add to add a skip pattern. Fill this next screen out as follows:

For simplicity sake, we only have created one terminating page both for people who disqualify (eat no ice cream) and those who complete the survey fully. You could include multiple terminating points and mark records that terminate as Qualified or Disqualified, or even assign "disposition numbers" to track respondents who terminate at various points in the questionnaire. Now that we have added our questions to the study, added page breaks, and defined skip logic, let's preview our questionnaire.

Previewing the Questionnaire

We have already shown you how to preview each question, one at a time. We'll now show you how to preview the entire survey. At the Write Questionnaire dialog (accessed from the main menu by clicking Compose | Write Questionnaire or by clicking the "Pencil" icon), click the "Preview"

22

SSI Web v7

icon

When you click Preview, you can choose what to preview. Choose Preview Questionnaire from Page 1. A browser window automatically opens page 1 for your review. In the background, SSI Web has created a Temp folder inside your study folder on your hard drive, and written separate static HTML pages to that folder named TempWeb1 through TempWeb5 (you have 5 pages in this questionnaire). These files are linked one to another by the Next button, for your convenience, so you can browse from page to page by just clicking the Next button. Another convenient way to preview the questionnaire (or share the questionnaire with colleagues) is to save the survey to a single HTML file or PDF-format file. Click File | Print Study to print the study in either of these formats (select Graphical View and click Print to File). Note: The Preview Survey mode is just for preview purposes onlyno data are stored, and no skip patterns are executed. These static pages are not uploaded to the Internet for data collection purposes. Instead, your questionnaire file is uploaded to the Internet, and the Perl scripts on your Web Server understand the instructions within the questionnaire file and create each questionnaire page, as requested by the respondent, "on the fly."

Test Running Your Survey Using "Local Test Server"

Although the Preview Survey function is nice, you generally will want to test your survey in runtime mode, with any skip patterns, response verification, and randomizations in place. You could upload your files and the Perl scripts to the Web Server, but SSI Web provides a more convenient way to test run your survey locally on your PC or laptop. When SSI Web was installed on your computer, web server software called "Apache" was also installed. Apache lets your computer run the questionnaire just like a remote Web Server would.

Local Test Server

From the Write Questionnaire dialog, Click the Test Survey button Locally.

and select to Test Survey

Your browser opens up the first survey page. Enter your User Name (SAWTOOTH) and password (SOFTWARE) and click the Next button. This is your survey, as it would appear to a respondent over the Internet (there are slight visual differences from browser to browser). To close the survey, simply close the browser window (using the X in the upper-right hand corner). To run the survey again, click the Test Survey button .

Go ahead and take a survey or two, as we'll need the data for a later section of this tour.

Adding Polish and Style

You have probably noted that the survey we've created is pretty bland looking. We can add some polish and style by clicking the Survey Settings icon from the Write Questionnaire dialog (or by

General Web Interviewing (CiW) Help clicking Compose | Survey Settings from the main menu).

23

First, let's select a style from the Styles tab. Select a style you wish to use, and click Use Selected at the bottom of the dialog to implement the style. We'll select Panama as the style, but you might want to experiment with other styles. On the General Settings tab, you can select to use graphical Next, Previous, and Select/Checkbox buttons. A library of buttons is provided, available in the C:\Program Files\Sawtooth Software\SSI Web\graphics folder. On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Ice Cream Questionnaire as the header text. The Progress Bar tab lets you add a progress bar to your survey. We'll click the check box to add a progress bar to the page footer. When you go back and test run the survey, your survey should have a bit more polish. Under Panama style, our survey now looks like:

The Admin Module

When you eventually upload your project to your Web Server (or our servers if using our web hosting service), an administrative area is available for you. You can access this administrative area from any computer connected to the Web. You protect this area using a password, so that others cannot access your survey settings or the survey data. You may change the password under the Server

24

SSI Web v7

Settings | Server Settings dialog (accessed by clicking the dialog). With the Admin Module, you can

icon from the Write Questionnaire

Check how many completed records you have View/Edit respondent data Tabulate responses for individual questions Download data to your hard drive Reset survey (delete all data and start fresh) Download passwords for respondents who have completed/not completed

In this tour, we'll just cover downloading data, but we encourage you to investigate the other options within the Admin Module.

Running the Admin Module

From SSI Web's main menu, click Test | Admin Module. The Admin Module appears just as it would over the Web, but we are only working with files and data locally on your hard drive.

Downloading Data
Downloading completed surveys places all the complete respondent records into a single data file, within your study folder on your hard drive. You can do this from the Admin Module by clicking Download Data. In that case, it will ask you to browse to your study folder on your hard drive. You may download data that way, but it is easier to do it from within SSI Web's interface (since you don't have to browse to your study folder). From within SSI Web, click Test | Download Data. You are informed that the test data were successfully downloaded. The master file of respondent data is called STUDYNAMEdat.dat, where STUDYNAME is the study

General Web Interviewing (CiW) Help

25

name you created (e.g. mydemodat.dat). Before proceeding, you may want to make sure that this file is indeed within your main project folder. A quick way to browse to your folder to check this is by clicking the Explore Study icon on the tool menu, which automatically opens the study folder. The STUDYNAMEdat.dat file must be saved to that same folder.

Exporting (Converting) Data to Excel(TM) Format

After you have collected data and downloaded the data file to your hard drive, you typically will want to convert the data to a common format for analysis. SSI Web can convert the data (and labels) to TM TM SPSS , Excel or text-only formats. For this tour, we'll just use the .CSV format which may be opened using Excel. If you haven't already done so, close the browser window for the Admin Module and return to the main SSI Web menu. Click File | Export Data | All Data (or click the Export Data icon Export All Data dialog is shown: ), and the

The Export File Type tab lets you specify the type of file and filename/folder to save to. The Respondent Filter tab lets you choose which respondents to include, the Data to Export tab lets you choose which variables to export, and the Data Formatting tab lets you specify additional format settings.

26

SSI Web v7 For this example, we'll just use the default settings. Click the Export button, and the .CSV file is created in the indicated folder. To see the result, using your Windows Operating System, browse to the folder (don't forget the handy Explore Study icon on the tool menu that automatically explores your study folder) and doubleclick the STUDYNAME.csv file (probably called demo.csv if your study was named "demo"). If you have Excel on your computer, it should be associated with a .csv file, so this action should immediately open your data file within Excel. If you don't own Excel, the .csv file may be opened using Notepad or another simple text editor. Here are our results (yours will be different):

Responses for Q1 and Q2 are quite straightforward. Q1 was a single-select question, and the first respondent answered with the first response option (coded as a "1"). This response option disqualified the respondent, so no more data are available for this case. Respondent number 2 clicked the second option for Q1 and typed a 3 for Q2 (how many times eaten ice cream in last week). Q3 is a multiple-response (check box) question. Respondent number 2 checked the first and fourth items (a "1" indicates checked and a "0" indicates not checked). SSI Web automatically unscrambles the random order and presents it here in the natural order you specified when you created the survey.

Adding Graphics
As a last point, we'd like to show you how easy it is to add graphics to your survey. The typical kinds of graphics files you use for web surveys are in .GIF or .JPG format. You can likely find a number of graphics files with this format on your hard drive (use the File Find capability within Windows if you do not immediately know of a .GIF or .JPG file on your computer). You may recall that in SSI Web, most any place that you provide survey text, there is also a "Pencil" icon, which opens up a larger Window for typing text. This larger window also includes a toolbar, as shown below:

For example, this toolbar helps you bold (B), italicize (I), or Underline (U) text within your survey (using the toolbar buttons automatically insert HTML instructions, which browsers know how to interpret to create the desired effects). You can also use the toolbar to add graphics. Place your cursor at the place in your text at which you wish a graphic to show. Then, click the "camera" icon. A dialog appears that asks you to type a file name or browse to find the graphic to add to the survey. Type the name, or browse to the file and click the Open button to insert the graphic. Special HTML instructions are added to your text to tell the

General Web Interviewing (CiW) Help browser what file to display and where to find it on the server.

27

When you preview or run the questionnaire using Local Test Server, the graphic is displayed. (Make sure to keep your graphics small, as large graphics files can take a long time to show for respondents with slow connections. Typical graphics sizes for surveys are about 20KB or smaller.)

Conclusion
This quick tour has only touched on the basics. We encourage you to search the online help, or to print out the complete documentation. The documentation is provided in a .PDF format called "SSI Web Documentation.PDF" within your SSI Web installation folder. And of course, if you have any questions, you can contact us at 360/681-2300 or write us at support@sawtoothsoftware.com. Our hours are 7:00 AM to 5:00 PM Pacific, Mon-Fri.

28

SSI Web v7

1.1.3

Software Requirements: Users, Servers and Respondents

For the User (Survey Developer): PC running under Windows XP Service Pack 2, Windows Vista, Windows 7 or later. Any browser with JavaScript enabled. The Internet/Intranet Server: Support Perl Version 5.003 or later (capable of running CGI scripts) Permit "write" and "delete" capabilities to allow our programs to write and delete files on the server. (If your ISP doesn't support these capabilities, we can recommend an ISP that does, or we can host your surveys for a fee.) For the Respondent: Respondents can use any PC, Mac or Internet-enabled terminal. Since the surveys use only HTML pages, any browser that interprets standard HTML will work. Browsers that don't support JavaScript 1.2 or later uses the Perl error checking method. In addition, if you want to turn off JavaScript verification altogether, you can turn it off from the Compose | Survey Settings | General Settings menu.

General Web Interviewing (CiW) Help

29

1.1.4

Creating/Opening a Study
A "Study" is a collection of files pertaining to a particular research project. SSI Web requires that you place your studies in separate directories (folders) on your local drive. If your study is named "abcd," a folder called abcd is created, and a study file named abcd.ssi is created within that folder: /abcd/abcd.ssi We suggest that you place your study directory on your local hard drive rather than a shared network drive. If using a network drive, the performance can be slower, and SSI Web does not support multiple people accessing it at the same time. To create a new study: Click File | New Study, browse to the desired directory (or click the Create a New Folder icon) and provide a studyname consisting of four or fewer characters. A folder called STUDYNAME is also created, the STUDYNAME.ssi file is placed within that target directory, and the study is opened. To open an existing study: Click File | Open Study, and if necessary browse to the folder containing the study. Then click the STUDYNAME.ssi file to open a study previously created. You may also select a study name from the list of most recently opened studies, available under the File menu, or when browsing folders double-click a STUDYNAME.ssi file. To copy the current study to a new path or name: Open an existing study. Click File | Save Study As.... Browse to the target folder, and type a study name that is no more than twenty-four characters. SSI Web automatically supplies a .ssi extension to the filename and saves all associated study files (including passwords, graphics directory, and data) using that naming convention to the target folder.

30

SSI Web v7

1.2

What's New in V7.0?

General Improvements: 1. No more limitation of 4 character studynames. Up to 24 characters now supported. 2. Multiple CBC studies, ACA studies, and CVA studies may be included within the same project. This means less linking of surveys! 3. Back button (not just the back button on the browser toolbar, but a back button within the survey). You can even back up to all parts of the survey after a restart. 4. Graphical select and radio buttons (which also means you can dramatically change the size of buttons). Also, an option to highlight the text of response items on mouseover. 5. Randomized Blocks of questions (e.g. the 3 blocks of questions: Q1-Q3; Q4-Q10; Q11-Q25 can be randomized, and Q1-Q3 always stay together as a block, Q4-Q10 always stay together, etc.) 6. The standard formatting controls in the edit question dialogs will operate nicely in conjunction with styles. In previous versions, styles typically would override most of the formatting controls. Starting in v7, SSI Web surveys by default use a more attractive style, so your surveys will look much better even before you apply your favorite style. 7. Many new styles available in the library. 8. Dramatically improved data export speed. 9. Ability to export all the text for your survey to an XML XLIFF file format (commonly used by translation houses). Once your survey is translated, you can import the XLIFF file back into your survey in one step. 10. Automatic survey upload and data download. The FTP process is built into SSI Web, so this feature will make working with the server faster and less complicated. SFTP (a more secure file transfer protocol) is also supported. 11. Ability to edit data (change responses) through the online administrative module, including changing the status of a respondent from an incomplete to a completed record. 12. Ability to set filters for the marginals report. This lets you, for example, tally the responses for a question among respondents belonging to large businesses. 13. Incompletes (drop out) report. Reports what percent of respondents dropped out at each question. 14. Internal respondent numbers are now all numeric, not alpha characters. Numbers are much easier to interpret and work with! 15. Ability to save response time (latency) per page. This is on by default, but you can easily turn it off (to reduce the size of the stored data file) via SSI Web's interface on the Survey Settings dialog, Advanced tab. 16. We've redone the way that "error messages" work (messages that tell respondents that they've missed a question on the page, or used an invalid response). In v7, we highlight the question with the problem using a (typically red) box, and place a (typically red) message within the box describing the nature of the error. (You can change the warning color from red.) 17. Auto advance for questions involving constructed lists that include a single response item with a

General Web Interviewing (CiW) Help

31

required response. Previously, the respondent would see the question and be forced to check the one (and only) possible response. With v7, we recognize that the response is a foregone conclusion, and we automatically record the answer and advance to the next question. MaxDiff-Specific Improvements 1. New function for computing MaxDiff scores on-the-fly using individual-level logit. You can return scores for each item, or a ranking of the items. This makes it possible, for example, to force just the top (most important) n items onto a new constructed list. 2. Paper-and-Pencil data collection and accumulation methodology incorporated within SSI Web. No more need to use the separate MaxDiff Designer software. Note: More detailed version information including bug fixes can be accessed on the Sawtooth Software website at http://www.sawtoothsoftware.com/version/ssiweb/ssiweb_history.html

32

SSI Web v7

1.3

Composing Web Surveys

When you click Compose | Write Questionnaire, the Write Questionnaire dialog is displayed:

This dialog is the main workhorse within SSI Web. It provides a list of the current questions in your study, shows question types for each, and has many buttons that lead to other useful functions. You add questions to your project using the Add... button, which leads to the Add to Questionnaire dialog that lets you specify the question name and type. When you highlight a question on the list, the page number this question appears on is shown in the footer of the dialog. When you right-click a question name, a menu appears that include many of the options shown on the dialog above, as well as the ability to rename the question.

General Web Interviewing (CiW) Help

33

Re-ordering Questions
You can re-order questions within the list by highlighting them and clicking the Up Arrow and Down Arrow icons directly at the right of the questions list. You can also re-order questions by cutting and pasting questions to a new location (see next section for further details).

Cutting/Copying Questions within and across Studies

You can cut or copy questions within the same study, or to another SSI Web study. To copy questions, highlight a single or multiple questions (by holding down the Ctrl or Shift keys while selecting with the mouse) and click the Copy button (or press Ctrl + C). The question(s) are stored in the buffer. Then, place your curser within the current question list or in the question list for another SSI Web study (to copy to another SSI Web study, you must close the current study--but do not close SSI Web). Click the Paste button (or press Ctrl + V) and a copy of those question(s) are pasted into the target questionnaire. The questions are automatically renamed to resolve duplicate labels, by incrementing a number at the end of the question name. Add... Adds a new question directly following the question that is currently highlighted in the List of Questions. Add Page Break Adds a page break directly following the question that is currently highlighted in the List of Questions. Edit... Edits (lets you modify) the question that is currently highlighted in the List of Questions. You can also double-click a question within the list to edit. Delete Deletes the question that is currently highlighted in the List of Questions. Skip Logic... Leads you to a dialog in which you can specify skip logic, to control the flow of the questionnaire. Randomize... Lets you add randomization of pages or questions on a page. Save Saves the current changes to your study. Close Closes this dialog.

Print... Prints a copy of the question list, along with the number of data fields used in each question.

Survey Settings...

34

SSI Web v7 Leads you to a dialog in which you can specify a number of global settings that control the look and functionality of your survey, including fonts, colors, and global headers and footers.

Lists... Accesses the List Manager, for managing lists of items used in questions.

Check for Errors... Checks your questionnaire for logical and other errors.

Preview... This option lets you preview how your questionnaire looks when it is posted on your web server. When you click this button, you can select whether to view the entire questionnaire, the highlighted question(s) on the list, or the entire page containing the highlighted question. The page(s) of your survey are saved within the /temp subdirectory off of your study directory as TempWeb1.htm through TempWebN.htm, where N is the number of pages in your study. If Internet Explorer is installed on your system, it opens TempWeb1.htm automatically in your default web browser. When you click the "Submit" button at the bottom of each page, it links to the next page in sequence. If you want to share a printed copy of survey with colleagues, you'll want to use the File + Print Study functionality rather than Preview.... The TempWeb?.htm files are only updated/modified when you click the Preview... button. Changes you make to your questionnaire are not reflected in the TempWeb?.htm files unless you click the Preview... button.

Test the Survey Locally... Uses the local test server to test your survey. Uses your default browser to display the questions. All skip patterns, question verification, randomizations are executed.

General Web Interviewing (CiW) Help

35

1.3.1
1.3.1.1

Composing Questions
Adding Questions to Your Survey
When you click Add... from the Write Questionnaire dialog, the Add to Questionnaire dialog is displayed:

The dialog lets you specify the question name and type to add to your survey.

Question Names and Types

Question names can include up to 24 letters or numbers. They cannot begin with a number, and cannot include spaces. They also cannot include a reserved name (the software will report whether you have used a reserved name and ask you to make a new selection). Available question types are: Constant Sum Adds a question in which respondents provide a series of numeric values that sum to a certain value, or sum to the value provided in a previous question. Free Format Lets you compose the HTML to define most every aspect of a question or series of questions. This is a powerful area for users who know HTML. One can compose special question layouts not supported by the standard templates for CiW question types. Grid Lets you compose Select, Numeric, Constant Sum, Ranking, and Open-end questions in a grid format (usually multiple rows and/or columns).

36

SSI Web v7 Numeric Adds a numeric question. Numeric questions are text boxes in which respondents type numeric values. Open-end Adds a text box question, in which respondents provide a written answer to an open-ended question. Responses can be short (a few words), or long (many sentences). Quota Questions Lets you evaluate whether respondents have qualified for a quota bucket based on previous question(s). This quota "question" is not really a question in the traditional sense (it doesn't require any response), but is a qualification point at which you write logic to decide whether a respondent has qualified to take the survey or not. Ranking Adds a question in which respondents can rank multiple items, from "1" meaning best to "n" meaning worst. Select Lets you add a select-type question. Examples include radio button (select one from a list), drop-down combo boxes, and select all that apply (check-box). Terminate/Link Adds a section of text or HTML instructions that requires no user input, that either ends the survey or links respondents to another web site or survey. Text/HTML Filler Adds a section of text or HTML instructions that does not require any response. Available exercise types are: ACA, ACBC, CBC, CVA, and MaxDiff If you are licensed to use these analytical tools, you can add these exercise(s) to your study. The control parameters and attribute list associated with each analytical exercise determines the number of questions and question types that are added for that exercise.

Copy from Question of Similar Type

If you want to add a new question to the questionnaire that is very similar or identical to another question within this same survey or within another SSI Web project, you can save a lot of time by copying that question and giving it a new name. You can create copies of questions by highlighting a question or questions from the Write Questionnaire dialog, clicking the Copy Question icon (or pressing Ctrl + C), placing your cursor in the list of questions of the target survey (either the current or a different SSI Web survey), and clicking the Paste icon (or pressing Ctrl + V). If copying files to a different SSI Web survey, make sure to keep SSI Web running (and do not open a second instance of SSI Web). After copying the question(s) to the clipboard, close the first survey. Open the second survey and paste the questions within the question list. If you copy a question named ABC within the same questionnaire, SSI Web automatically names the

General Web Interviewing (CiW) Help

37

copy ABC1, and a second copy (produced by clicking the Paste icon a second time) is called ABC2, etc. You can rename the copies of questions by editing the question and clicking the Rename button.

38 1.3.1.2

SSI Web v7

Setting Page Breaks

There are a number of options for organizing your questions in your survey. You can display each question on a separate page in your survey. Or, you can organize questions into groups displayed on separate pages. You could also put all questions on just one page, but there are a number of reasons you probably shouldn't: 1. Too many questions on a single page can make for a very long and clumsy form. 2. Response verification for numeric ranges or combo boxes would only occur at the very end of the survey. Respondents might have a difficult time reviewing such a long list of questions to discover their errors. 3. Respondents might look ahead to sections of the questionnaire that you'd rather they didn't see until the proper time. 4. Respondents who wanted to quit a survey early and resume at a later time would lose all their work. If you organize your survey into separate pages, respondents who quit early only lose the questions they've answered since clicking the last Submit button (assuming each respondent has a unique password with a quota of 1). We suggest that you organize your survey questions in logical groups displayed on separate pages. You might discover that some questions (such as conjoint questions, or questions requiring verification) are best shown one at a time, even though this means more back-and-forth between the server and the respondent's PC.

Setting Page Breaks and Previewing Pages

1. 2. Highlight the question that you want the page break to follow. Click Add Page Break, and a page break is inserted after the highlighted question.

Multiple consecutive page breaks (without any questions defined on the separate pages) are not permitted. You are asked to delete the redundant page breaks when you compile your questionnaire or prepare it for running on the web server or local server.

Clearing Page Breaks

1. 2. Highlight the page break you wish to delete. Click the Delete button.

General Web Interviewing (CiW) Help 1.3.1.3

39

The Start Question

Every SSI Web questionnaire includes a Start "question" that may not be deleted. The Start question can serve as an introductory screen where you provide text and/or graphics (additional questions can also be added to the same page). The Start question can also include password fields, where respondents type passwords to access the survey. Many SSI Web users assign respondents passwords but bypass the Start screen using one-click access to the survey. This technique uses a hyperlink (usually in an email message) that passes the necessary login information directly to SSI Web's main processing script, ciwweb.pl. If you are letting respondents access the survey without a user name or password, the Start screen serves only as a welcome/instructional page. The Start question is always on the first page in the List of Questions on the Write Questionnaire dialog. You use the Questionnaire Access and Passwords dialog to compose either introductory text and/or password login fields. You access this screen by highlighting the Start question on the list of questions, and clicking Edit.... Changes to the layout of the Start question are reflected in the STUDYNAMElogin.html file that is eventually placed on the server for data collection.

Start Question Layout

The Start question has four main sections: 1. 2. 3. 4. Header 1 Header 2 Password Fields (optional) Footer

If you use passwords, they are indented according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The Questionnaire Access and Passwords dialog has four buttons: Preview, OK, Cancel, and Advanced. Advanced... Preview OK Cancel This button accesses advanced settings for this question. Lets you preview how the Internet browser displays the current question. Saves changes to the current question and returns to the Write Questionnaire dialog.

Returns to the Write Questionnaire dialog without saving changes to the current question. You specify header and footer text by typing directly into the Text fields within the dialog or by Clicking the "pencil" icon to edit the text within a larger window. You can include HTML instructions in any of the Text fields to enhance your surveys with additional styles or graphics. The toolbar within the text editor offers a number of buttons to automatically add HTML instructions within your survey. You can modify the Size, Color, Justification, and also control Bolding, Italics, and Underlining by clicking the "paintbrush" icon .

40

SSI Web v7

Example: Composing a Start Page

In this example, we'll assume you are not using respondent passwords. The use of respondent passwords is described later in the Respondent Passwords section of this documentation. 1. While at the Write Questionnaire dialog, highlight the <Start> question in the List of Questions by clicking it with the mouse. 2. Click Edit.... (Skip Header 1) 3. Header 2: Text: Welcome to our survey. Please click next to continue.

4. Click Preview: Notes: You can include other questions on a Start page besides password fields. However, if respondent passwords are being required and a respondent restarts the survey, the respondent must completely fill out the Start screen (including the additional variables) prior to continuing. If the respondent enters different values on the password screen for these additional questions than he/she answered before, the respondent cannot jump forward in the survey to where he/she left off. The respondent must complete the entire survey again. Questions involving randomization cannot appear on the first page.

General Web Interviewing (CiW) Help 1.3.1.4

41

Select Questions
Select questions are very common in market research surveys. Three general types of Select questions are offered: 1. Radio Button (single response) 2. Check Box (multiple response) 3. Combo Box (single response) To add a Select question to your survey, click the Add... button from the Write Questionnaire dialog. The new question is inserted directly after any highlighted question in the List of Questions. If no question is highlighted, the new question is inserted at the end of the list. The Add to Questionnaire dialog is displayed. Choose the Select question type. Type a question name in the Question Name field. Question names can include up to 24 letters or numbers (no spaces or underscores permitted, and cannot start with numbers). Click OK. The Select Question dialog is displayed.

Select Question Layout

The Select question has four main sections: 1. 2. 3. 4. Header 1 Header 2 Question Footer

The Question text line appears directly above the response options. The Question text line is indented according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The response options are indented a bit further relative to the Question text line. The Select Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel. Rename Advanced... Preview OK Cancel Lets you rename the current question. This button accesses advanced settings for this question. Lets you preview how the Internet browser displays the current question. Saves changes to the current question and returns to the Write Questionnaire dialog. Returns to the Write Questionnaire dialog without saving changes to the current question.

Format controls (Color, Size, Bold, Italic, Underline, and Justification) are available for formatting text using the paintbrush icon.

Choosing the Type of Select Question

You choose the type of Select question by selecting (under the Settings tab) from a Radio Button (single response), Check Box (multiple response) or Combo Box (single response).

42

SSI Web v7 Radio Button (single response):

When using a Radio Button select question, you also specify on the Settings tab how many columns to use in presenting the response options (the example above formats the response options in a single column), and whether to require a response. Check Box (multiple response):

When using a Check Box select question, you also specify on the Settings tab how many columns to use in presenting the response options (the example above formats the response options in a single column), and the minimum and maximum number of boxes that respondents are allowed to check. Combo Box (single response):

When using a Combo Box select question, you also specify on the Settings tab whether a response is required, and what text to display as the default (the example above uses "(Click Here)" as the default text).

Specifying Response Options

Next, you specify response options by clicking the Response Options tab. When you specify response options for a select-type question, you are creating/using a "predefined" list of response options that can be used in this question and in other questions. This predefined list has a name (up to 24 characters, starting with a letter). By default, the list name is QuestionNameList, but you can change this name if you wish. 1. From the Response Options tab, select an existing list of items, or add a new list of items by clicking the Add... button. 2. If adding new items, type the list item in the space provided. 3. Click OK when finished, and the list item is added.

General Web Interviewing (CiW) Help

43

Tip: If you are adding many response options at the same time, you can save time by pressing ENTER twice after typing each list item. Pressing ENTER twice is equivalent to clicking OK and then clicking Add.... Time-Saving Hint: You can cut-and-paste lists of response options from Microsoft Word or Excel by highlighting the array of items in Word or Excel, copying to the clipboard using Ctrl+C or the Copy button in Word or Excel (or another text editor), and then (within SSI Web) by clicking the Paste from word processor or spreadsheet icon .

A numeric index is displayed at the left of your list items for reference, but is not displayed as part of the question. Click OK to save your changes and to return to the Select Question dialog.

Formatting Response Options

On the Settings tab, the number of response options you specified for this question is displayed in the Number of Response Options field (which you cannot edit). Response options can be formatted in a single column, or over multiple columns by specifying a value greater than 1 in the Number of Columns field. For example, if you have 11 response options (say, colors) and specify four columns, the response options are distributed as follows: Which of these colors is your favorite? 1) Red 2) Purple 3) Pink 4) Blue 5) Yellow 6) Green 7) Violet 8) Brown 9) Orange 10) Magenta 11) Gray

Other Specify Responses

Often, researchers need to provide an "other specify" option, that includes a open-end text field for typing a response.

SSI Web automatically inserts an open-end text field next to a response option when you indicate that a response item is a Respondent Specify ("Other Specify"). When adding list items under the Response Options tab, highlight the item that is to include an other specify entry field. Check Respondent Specify ("Other Specify"), and use the Settings... button to indicate if the respondent is required to specify text if they click this item, and the minimum and maximum characters permitted. The open-end data for the other specify response are saved as a new text variable in the data file, called QuestionName_Item#_other, where QuestionName is the name of the question and Item# is the item's number on the list (natural order). More than one list item may be an "other specify" for a

44

SSI Web v7 question. Please note that "Other Specify" open-end response boxes are not supported with combo box select questions. It would seem much more appropriate to use a radio button layout if including "other specifies." Typed other specify responses can be carried forward as response options to later questions if using constructed lists.

Exclusive ("None of the Above") Responses

When check-box (multiple response) questions are used, researchers often need to specify that an item is "exclusive," meaning that if it is chosen, no other items are valid. The classic example is the "None of the above" response item. When adding list items under the Response Options tab, highlight the item that is exclusive. Check Exclusive ("None of the Above"). More than one list item may be "exclusive" for a question.

Randomizing Response Options

To randomize response items for a question, from the Response Options tab, click Randomize Response Options. This randomizes all items except for any Other Specify or "None of the Above" options at the bottom of the list. When using this method to randomize options, information regarding the specific order shown to each respondent is not saved in the data. If you need additional flexibility for randomizing a subset of the range of items beyond that provided with this simple dialog, you have great power and flexibility when constructing lists and using the Randomize command. When you randomize list items using constructed lists, information is saved in the data file regarding the specific order shown to each respondent.

Re-ordering, Copying and Pasting Response Options

The "up" and "down" arrow icons can be used for moving the selected response option up or down in the list. You can also highlight a single or multiple items and cut or copy-and-paste the items to a new position on this list, or to a response list for another select-type, ranking, or constant-sum question. Highlight the item(s) to be moved or copied, then click the Copy or Cut icons. Highlight a new place on the list (or within a response list for a different question). Click the Paste icon.

General Web Interviewing (CiW) Help

45

Referencing Select Questions in SSI Script

Radio and Combo boxes Syntax: QuestionName Examples: Q1 [% Q1 %] Description: You refer to single-response (radio) button using the question name. For example, [% Q1 %] returns the numeric response associated with a radio button select question named Q1. Check Boxes Syntax: QuestionName_Response# Examples: Q1_4 [% Q1_4 %] Description: For multiple-response questions, each response option is either checked (value=1) or un-checked (value=0). You refer to each response option within a multiple-response question using underscore "_" notation. For example, [% Q1_4 %] returns a "1" or a "0" depending on whether the fourth response option in multiple-response question Q1 was checked.

Example: Composing a Radio Button (Single Response) Question

1. Click Add from the Write Questionnaire dialog. 2. Select the Select question type. 3. Type a valid name in the Question Name field, such as Education, then click OK. 4. From the Settings tab, click Radio Button (single response). 5. Click the Response Options tab. Add the following six response categories by clicking Add..., and pressing ENTER twice after typing each response option: Some high school Completed high school Some college Completed college Some graduate studies Completed advanced degree Click the Question Text tab. 6. Header 1: Text: Please indicate the highest degree of education you attained:

7. Click Preview:

46

SSI Web v7 Typing a 2 into the Number of Columns field of the Settings tab spreads the response options across two columns. Tip: Try adding optional HTML instructions in Header 1 to italicize "highest" and add a horizontal line beneath the question: Header 1 Text: Please check the <i>highest</i> degree of education you attained:

Footer Text: <hr> Notes: You can italicize the word "highest" by using the toolbar accessible when you click the "pencil" edit icon . Highlight the word "highest" with the mouse, and then click the italics button (looks like an italics I ). You can also add a horizontal line by editing the Footer text (again using the "pencil" edit icon) and clicking the "horizontal rule" button (looks like a dash "")

Example: Composing a Check Box (Multiple Response) Question

1. Click Add from the Write Questionnaire dialog. 2. Select the Select question type. 3. Type a valid name in the Question Name field, such as Music, then click OK. 4. From the Settings tab, click Check Box (multiple response). 5. Click the Response Options tab. Add the following seven response categories by clicking Add..., and pressing ENTER twice after typing each response option: Country Classical New Age Jazz Soft Rock Hard Rock Klingon Opera Click the Question Text tab. 6. Header 1: Text: 7. Header 2: Text: Which types of music do you enjoy listening to? (Select all that apply)

8. Click Preview: Note: When using the multiple-select check box, you can require respondents to check a minimum and maximum number of items (from the Settings tab). (These can be specified using SSI Script)

Example: Composing a Combo Box (Single Response) Question

1. Click Add from the Write Questionnaire dialog.

General Web Interviewing (CiW) Help 2. Select the Select question type. 3. Type a valid name in the Question Name field, such as Party, then click OK.

47

4. From the Settings tab, click Combo Box (single response). Specify the Default Text to be used in the combo box is "(No Response)" 5. Click the Response Options tab. Add the following response categories by clicking Add..., and pressing ENTER twice after typing each response option: Democrat Independent Republican None of the above When the combo box is displayed, the default text you specified on the Settings tab is always shown and selected by default. Respondents choose a category by clicking the arrow to reveal the drop-down list of response options. If you check Require Response from the Select Question dialog, respondents must select a category other than the default option before proceeding to the next page in the survey. 6. Header 1: Text: 7. Question: Text: Which political party do you most closely align with? (Click the down arrow and choose an option)

8. Click Preview: Tip: You can use combo boxes for collecting numeric responses. Simply type numbers for the response categories. Combo boxes can make sure that answers conform to certain values when respondents answer the question, rather than requiring the verification that occurs for the numeric question type when respondents click the Submit button. For example, a question might ask respondents how many people are in their family. The seven response options (not including the default option) might be: 1, 2, 3, 4, 5, 6, 7 or more.

48 1.3.1.5

SSI Web v7

Numeric Questions
Numeric questions let respondents type a numeric answer from -99999999 to 999999999 into an answer field. First we'll describe how to compose Numeric questions, and then we'll provide a simple example for you to try. When you add a numeric question to your survey, the new question is inserted directly after any highlighted question in the List of Questions on the Write Questionnaire dialog. If no question is highlighted, the new question is inserted at the end of the list. The Numeric Question dialog is used for formatting numeric questions.

Numeric Question Layout

The numeric question has four main sections: 1. 2. 3. 4. Header 1 Header 2 Question Footer

The Question text line with its answer field is indented according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The Numeric Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel. Format controls (Color, Size, Bold, Italic, Underline, and Justification) are available for formatting text using the paintbrush Advanced... icon.

This button accesses advanced settings for this question.

Numeric Question Settings

From the Settings tab, you can control the range of values that can be entered by specifying a Minimum Value and a Maximum Value (SSI Script may also be used). When you preview the question, the response field lets you type the number of characters used for the Minimum or Maximum Value. The value constraints are not enforced until the survey runs on the server. There are three main controls for the numeric question: Require Response Indicates whether respondents are required to enter a valid response before continuing the survey. Allow Decimal Answers Controls whether answers with a decimal point are accepted as valid answers. You can specify whether commas or periods are to be used as decimal delimiters under the Survey Settings | Advanced dialog. (Note: the Marginals program within the Admin Module truncates all answers with decimal points, treating them as integers, prior to computing frequencies, min, max, and means.)

General Web Interviewing (CiW) Help Position Question <Left/Right> of Input Box

49

Determines whether the text in the Question text area appears at the left or right of the numeric entry box.

Referencing Numeric Questions in SSI Script

Numeric Questions: Syntax: QuestionName Examples: Q1 [% Q1 %] Description: You refer to numeric questions using the question name. For example, [% Q1 %] returns the numeric response associated with Q1.

Example: Composing a Numeric Question

1. Click Add from the Write Questionnaire dialog. 2. Select the Numeric question type. 3. Type a valid name in the Question Name field, such as AGE, then click OK. 4. Click the Settings tab. Specify a Minimum Value of 18 and a Maximum Value of 99 by clicking and editing those fields. Click the Question Text tab. 5. Header 1: Text: (Skip Header 2) 6. Question: Text: 7. Footer: Text: Age: (If you really don't want to tell us your age, just type 99) What is your age?

8. Click Preview: Tip: Try adding optional HTML instructions in the Footer to bold the "99" and add a horizontal line beneath the question: Footer Text: (If you really don't want to tell us your age, just type <b>99</b>)<hr>

Notes: You can bold the "99" by using the toolbar accessible when you click the "pencil" edit icon . Highlight the "99" with the mouse, and then click the bold button (looks like a bold B). You can

50

SSI Web v7 also add a horizontal line by editing the Footer text (again using the "pencil" edit icon) and clicking the "horizontal rule" button (looks like a dash "")

General Web Interviewing (CiW) Help 1.3.1.6

51

Open-End Questions
Open-end questions let respondents type any characters (including letters and numbers) into an answer field. You can create either fixed entry fields (single line) or free entry fields (multiple lines) that give text scrolling capabilities when typing long text strings. First we'll describe how to compose Open-end questions, and then we'll provide a simple example for you to try. When you add an open-end question to your survey, the new question is inserted directly after any highlighted question in the List of Questions of the Write Questionnaire dialog. If no question is highlighted, the new question is inserted at the end of the list. The Open-end Question dialog is used for constructing open-end questions.

Open-end Question Layout

The open-end question has four main sections: 1. 2. 3. 4. Header 1 Header 2 Question Footer

The Question text line with its answer field is indented according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The Open-end Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel. Format controls (Color, Size, Bold, Italic, Underline, and Justification) are available for formatting text using the paintbrush Advanced... icon.

This button accesses advanced settings for this question.

Open-end Question Settings

From the Settings tab, you can specify options that control how the open-end box looks and operates. You can specify whether to Require Response, whether the entry box has a single or Multiple Lines. You can specify the Maximum and Minimum Number of Characters required, as well as the Width and Height of the box (in pixels).

52

SSI Web v7

Referencing Open-End Questions in SSI Script

Open-End Questions: Syntax: QuestionName Examples: Q1 [% Q1 %] Description: You refer to open-end question using the question name. For example, [% Q1 %] returns the typed response for Q1.

Example: Composing a Single Line Open-end Question

1. Click Add from the Write Questionnaire dialog. 2. Select the Open-end question type. 3. Type a valid name in the Question Name field, such as FirstName, then click OK.. Question names can include up to 24 letters or numbers (no spaces or underscores permitted, and cannot start with numbers). 4. From the Settings tab, uncheck Multiple Lines. Specify a Maximum Number of Characters of 15. Click the Question Text tab. 5. Header 1: Text: (Skip Header 2) 6. Question: Text: (Skip Footer) 7. Click Preview: Name: What is your first name?

Example: Composing a Multiple Line Open-end Question

1. Click Add from the Write Questionnaire dialog. 2. Select the Open-end question type. 3. Type a valid name in the Question Name field, such as VanStyle, then click OK.. 4. From the Settings tab, specify to use a width of 400 (pixels) and a height of 100 (pixels). Click the Question Text tab.

General Web Interviewing (CiW) Help 5. Header 1: Text: 6. Header 2: Text: (Skip Question) (Skip Footer) 7. Click Preview:

53

What do you like or dislike about the styling of the Summit minivan? (Type your response below)

54 1.3.1.7

SSI Web v7

Text/HTML Filler Questions

Sometimes you may want to add transitional text that does not require any response. You can specify text "questions" that stand alone on a survey page, or that separate other survey questions. Note that Text/HTML filler questions don't limit you to just text. You can include graphics or tables using HTML instructions. Text/HTML Filler questions do not count toward the data field limit specified by your software license. No data are saved for Text/HTML Filler questions. When you add a Text/HTML Filler question to your survey, the new question is inserted directly after any highlighted question in the List of Questions on the Write Questionnaire dialog. If no question is highlighted, the new question is inserted at the end of the list.

Formatting Options
You use the Text Question dialog to format text questions. Text Question Layout The text question has four main sections: 1. 2. 3. 4. Header 1 Header 2 Body Footer

The Body text line with its answer field is indented according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The Text Question dialog has four buttons: Rename, Preview, OK and Cancel . Format controls (Color, Size, Bold, Italic, Underline, and Justification) are available for formatting text using the paintbrush icon.

General Web Interviewing (CiW) Help 1.3.1.8

55

Terminate/Link Questions
This question type is used to indicate that a respondent should stop at this point in the survey (terminating question), or that the respondent should immediately *link to another survey or web site, either produced by SSI Web or another software system. No other questions can appear on the same page as a terminate/link question type. *Note: It is a violation of the SSI Web license agreement to link multiple SSI Web surveys together such that the total resulting survey exceeds the data fields supported by your license.

Terminating Question
When a respondent reaches a Terminate/Link question marked as "terminate," the survey is over. Reaching a terminating question prohibits respondents from backing up to change answers to previous questions. A terminating question must appear on a survey page by itself. Most survey authors include at least one, and sometimes multiple terminating questions in their surveys, where each is formatted on a separate page. A common use of the terminating question is to terminate a respondent who is not eligible to complete the survey, given his/her answers to previous questions. You can have more than one terminating question in your survey. To make the current Terminate/Link question a terminating question, click the Settings tab, and check the Terminate Respondent box. Specify whether respondents reaching this question are Qualified/Complete (5) or Disqualified (4). You can additionally assign a disposition code for each terminating question (a value from 0 to 999) that is saved in the respondent's data set. This is useful for tabulating how respondents terminated the questionnaire, if there are multiple termination points. While terminating respondents, you could choose to immediately link them to a web site or another survey (rather than show the text for the current question on the screen). In that case, you would also check the Link to a Questionnaire or Website box. This functionality is described further below.

Linking Question
SSI Web can link to other surveys or another web page using the linking question type*. The linking question must appear on its own page, and must have no other questions on that page (and it cannot display text, as this question is not actually "shown" to respondents). When the page including the linking question is encountered, SSI Web immediately links to the Web Address (URL) specified on the Settings tab. That URL can be a web site, another program, or a login script to start up another survey. (The page containing the link question is not shown, but signals to SSI Web to execute the link.) Typically, you will want to terminate respondents and mark them with Qualified/Complete (5) status when linking to another survey or website. To do so, check the Terminate Respondent box, and use the drop down control to specify the Termination Status. Once a respondent is marked as Qualified/Complete, they cannot restart the current survey. If you are linking to an outside survey or program and plan to have respondents re-enter the current survey to complete it, you should not check the Terminate Respondent box. When you link to other SSI Web surveys, the URL you use in the Settings tab specifies the ciwweb.pl file (a Perl script) for the next survey to start up. That survey will most likely be located in a separate folder on your server, or perhaps on a new server altogether (see below). The typical approach is to bypass the login (password) screen for the linked survey, and automatically pass through the user name, password, or other information from the first survey. We'll describe how to do this below.

56

SSI Web v7 URL link with Passwords If you want to link to a new SSI Web survey and automatically "pass through" the user name and/or password, you should link respondents directly to the Perl script located in the cgi-bin directory. This lets qualified respondents start the survey without having to type in passwords or see the login page. Let's imagine that we wanted to link to a new SSI Web survey called "tep1" installed on the sawtoothsoftware.com server, located in the "gonzo" study root directory, with valid passwords named myusername of "michael" and mypassword of "A6Z99". In the Web Address (URL) field on the Settings tab, we would specify: http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&myusername=michael&mypassword=A6Z99 When linking from an SSI Web survey to another SSI Web survey, to dynamically insert the respondent's passwords (named, as an example, myusername and mypassword), use [%myusername%] and [%mypassword%], for example: http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&myusername=[%myusername%]&mypassword=[%mypassword%] You can also pass additional variables into the data set, as long as those are defined as pass-in fields: http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&mypassword=michael&zip=98370 In this example tep1 is the study name, a password called mypassword is used, and the zip code for each respondent is already known and passed into the data file. You can pass in as many additional variables as you want, as long as they are initially defined as pass-in fields in the passwords area, and the length of the URL doesn't exceed that supported by browsers. When respondents are linked to the new survey, if the passwords are valid (and the quota is not complete), the respondent is sent to the page following the password screen. If the respondent is restarting an interview, the page where he/she left off is displayed. Dealing with Special Characters in URLs Data that include special characters should use the SSI Script function EncodeForUrl( ). This function will encode non-alphanumeric characters in the ASCII table between 32 and 126. For example, email addresses contain the @ symbol. If you need to pass an email address as the username, it should be passed in the following manner: http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&myusername=[%EncodeForUrl(emailaddress)%] To be safe, you could always use the EncodeForUrl( ) function when using SSI Script to include a variable within your URL. *Note: It is a violation of the SSI Web license agreement to link multiple SSI Web surveys together such that the total resulting survey exceeds the data fields supported by your license.

General Web Interviewing (CiW) Help 1.3.1.9

57

Grid Questions
Introduction to Grid General Headers and Footers Rows and Columns Grid questions can include any of the standard question types used in SSI Web surveys (numeric, radio, check-box, combo-box, constant sum, ranking, or open-end), but Grid questions give you much greater power to format them in new ways and within a unified table. The most common use of the grid question is to provide multiple rows and columns for labels and inputs, such as this array of radio buttons:

The main response area for this grid question has 4 rows and 5 columns (not counting the rows and columns for labels). There are four separate radio button variables, one variable for each row. This grid question uses four data fields toward your license capacity. The "Question Direction" is by rows. The alternating colors by rows help respondents orient themselves to the question direction. Grid questions can sometimes be challenging for new users to author. When you add a Grid question to your survey, an intuitive Grid Wizard is available to help you set up the essentials of the Grid. The key to using the main Grid interface efficiently is to always keep in mind whether the question direction is by rows (as in the previous example) or columns (as illustrated below).

This grid question requires eight data fields to store a respondent's answers ("checked" vs. "not checked" for each box).

58

SSI Web v7 Grid questions also can include numeric or text entry boxes, and rankings or constant-sums. They also can include combinations of different question types within the same grid. There are many options within the Grid area. We suggest you review the help text for each area, but especially that you experiment with the various settings and click Preview to see the results. As you learn to use Grid questions, you may find that the increased formatting options lead you to use Grid questions to compose standard questions that don't involve multiple rows or columns. The Grid question area has six main tabs (Question Text, Grid Text, Format, Rows, Columns and Skip Logic). The Rows and Columns tabs have near-identical functionality, depending on whether the Question Direction is by rows or columns. Therefore, we describe their functionality in one common documentation area. If the Question Direction is by Rows, then additional controls for adding new variables to the Grid are on the Rows tab. If the Question Direction is by Columns, then you add and edit variables from the Columns tab. Introduction to Grid General Headers and Footers Rows and Columns

Referencing Grid Questions in SSI Script

Numeric, Open- Syntax: GridName_r#_c# end, Combo, and Check Examples: boxes Q1_r2_c4 [% Q1_r2_c4 %] Description: You refer to single-response field in a grid using the grid name plus row and column reference. For example, [% Q1_r2_c4 %] returns the value associated with row 2 column 4 in grid Q1. For check boxes, the value is "1" (checked) or "0" (not checked).

General Web Interviewing (CiW) Help Radio button questions Syntax: GridName_r# GridName_c# Examples: Q1_r2 Q1_c4 [% Q1_r2 %] [% Q1_c4 %]

59

Description: Radio buttons in grids can use either row-based or column-based question directions. The most common layout is the row-based question direction. With row-based question direction, the responses are stored as Q1_r1, Q1_r2, etc. With column-based questions, the responses are stored as Q1_c1, Q1_c2, etc.

60 1.3.1.10

SSI Web v7

Constant Sum Questions

Constant sums are used to let the respondent allocate, say, 100 points across multiple items. Common uses include amount of time spent in various activities, or amount of money spent on various items. You can also ask respondents to allocate numbers that sum to a particular value that the respondent provided in an earlier question. Constant sum questions require a data field to be stored for each response option. Therefore, if respondents are to allocate points across 10 response items, this uses 10 data fields toward your license capacity.

Constant Sum Question Layout

The Constant Sum question has four main sections: 1. 2. 3. 4. Header 1 Header 2 Question Footer

The Question text line appears directly above the response options. The Question text line is indented according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The styles you apply (color, bold, size, etc.) in the Question row also apply to the response options. The Constant Sum Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel. Rename Advanced... Preview OK Cancel Lets you rename the current question. This button accesses advanced settings for this question. Lets you preview how the Internet browser displays the current question. Saves changes to the current question and returns to the Write Questionnaire dialog.

Returns to the Write Questionnaire dialog without saving changes to the current question. Format controls (Color, Size, Bold, Italic, Underline, and Justification) are available for formatting text using the paintbrush icon.

Specifying Response Options

Specify response options by clicking the Response Options tab. When you specify response options for a constant sum question, you are creating/using a "predefined" list of response items that can be used in this question and in other questions. This predefined list has a name (up to 24 characters, starting with a letter). By default, the list name is QuestionNameList, but you can change this name if you wish. 1. From the Response Options tab, select an existing list of items, or add a new list of items by clicking the Add... button.

General Web Interviewing (CiW) Help 2. If adding new items, type the list item in the space provided. 3. Click OK when finished, and the list item is added.

61

Tip: If you are adding many response options at the same time, you can save time by pressing ENTER twice after typing each list item. Pressing ENTER twice is equivalent to clicking OK and then clicking Add.... Time-Saving Hint: You can cut-and-paste lists of response options from Microsoft Word or Excel by highlighting the array of items in Word or Excel, pasting to the clipboard using Ctrl+C or the Copy button in Word or Excel, and then (within SSI Web) by clicking the Paste from word processor or spreadsheet icon .

A numeric index is displayed at the left of your list items for reference, but is not displayed as part of the question. Click OK to save your changes and to return to the Constant Sum Question dialog.

Randomizing Response Options

To randomize response items for a question, from the Response Options tab, click Randomize Response Options. This randomizes all items except for any Other Specify or "None of the Above" options at the bottom of the list. When using this method to randomize options, information regarding the specific order shown to each respondent is not saved in the data. If you need additional flexibility for randomizing a subset of the range of items beyond that provided with this simple dialog, you have great power and flexibility when constructing lists and using the Randomize command. When you randomize list items using constructed lists, information is saved in the data file regarding the specific order shown to each respondent.

Re-ordering, Copying and Pasting Response Options

The "up" and "down" arrow icons can be used for moving the selected response option up or down in the list. You can also highlight a single or multiple items and cut or copy-and-paste the items to a new position on this list, or to a response list for another select-type, ranking, or constant-sum question. Highlight the item(s) to be moved or copied, then click the Copy or Cut icons. Move the cursor to a new place on the list (or to a response list for a different question). Click the Paste icon.

Constant Sum Settings

Total Type a value such as "100" into this field. This is the amount that the numeric entry items should sum to. (You can provide a permissible range around the total, such as plus or minus 5, by clicking the Advanced... button on this dialog.) You can also specify [%Value(questionname)%], where questionname is a name of a previous numeric question in your survey. In that case, the value the respondent typed earlier in the questionnaire is used as the required sum.

62

SSI Web v7 Allow Decimals Specify if respondents are allowed to use decimals. Number of Columns Lets you arrange the response options across multiple columns. Require Response to Question Specifies whether respondents are required to answer the question. Require Response to each individual input box Use this field to specify whether respondents must supply a value for every response option of the multi-part constant sum question. Include a "Total" at the bottom If you want a running total to be displayed within the question, check this box. Also specify a label to be placed next to that field. The totals field appears as a grayed-out text box that updates as respondents enter values in the other fields in the constant-sum question. The totals value is not stored in the data file. JavaScript must be enabled to display totals. (In the Error Messages tab under Survey Settings..., if respondents do not fill out the question properly, you can display messages to the user regarding the current sum and the required total. ErrTotal( ) returns the required total, and ErrCurSum( ) returns the current sum.)

Referencing Constant Sum Questions in SSI Script

Constant Sum Questions: Syntax: QuestionName_Response# Examples: Q1_4 [% Q1_4 %] Description: For constant-sum questions, each response option can contain a numeric value. You refer to each response option within a constant-sum question using underscore "_" notation. For example, [% Q1_4 %] returns the numeric value typed into the fourth response option in Q1.

General Web Interviewing (CiW) Help 1.3.1.11

63

Ranking Questions
Ranking questions are useful if you need respondents to rank multiple items from best to worst. Two response formats are possible: combo box or numeric entry. In either case, respondents mark the best item as "1" and the worst item as "K", where K is the total number of items to be ranked. Each value must only be used once. An error message prompts respondents if the ranking question is not completed properly.

Ranking Question Layout

The Ranking question has four main sections: 1. 2. 3. 4. Header 1 Header 2 Question Footer

The Question text line appears directly above the response options. The Question text line is indented according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The styles you apply (color, bold, size, etc.) in the Question row also apply to the response options. The Ranking Question dialog has five buttons: Rename, Advanced..., Preview, OK and Cancel. Rename Lets you rename the current question. This button accesses advanced settings for this question.

Advanced... Preview OK Cancel

Lets you preview how the Internet browser displays the current question. Saves changes to the current question and returns to the Write Questionnaire dialog.

Returns to the Write Questionnaire dialog without saving changes to the current question. Format controls (Color, Size, Bold, Italic, Underline, and Justification) are available for formatting text using the paintbrush icon.

Specifying Response Options

Specify response options by clicking the Response Options tab. When you specify response options for a ranking question, you are creating/using a "predefined" list of response items that can be used in this question and in other questions. This predefined list has a name (up to 24 characters, starting with a letter). By default, the list name is QuestionNameList, but you can change this name if you wish. 1. From the Response Options tab, select an existing list of items, or add a new list of items by clicking the Add... button. 2. If adding new items, type the list item in the space provided. 3. Click OK when finished, and the list item is added. Tip: If you are adding many response options at the same time, you can save time by pressing ENTER twice after typing each list item. Pressing ENTER twice is equivalent to clicking OK and then clicking Add....

64

SSI Web v7 Time-Saving Hint: You can cut-and-paste lists of response options from Microsoft Word or Excel by highlighting the array of items in Word or Excel, pasting to the clipboard using Ctrl+C or the Copy button in Word or Excel, and then (within SSI Web) by clicking the Paste from word processor or spreadsheet icon .

A numeric index is displayed at the left of your list items for reference, but is not displayed as part of the question. Click OK to save your changes and to return to the Ranking Question dialog.

Randomizing Response Options

To randomize response items for a question, from the Response Options tab, click Randomize Response Options. This randomizes all items except for any Other Specify or "None of the Above" options at the bottom of the list. When using this method to randomize options, information regarding the specific order shown to each respondent is not saved in the data. If you need additional flexibility for randomizing a subset of the range of items beyond that provided with this simple dialog, you have great power and flexibility when constructing lists and using the Randomize command. When you randomize list items using constructed lists, information is saved in the data file regarding the specific order shown to each respondent.

Re-ordering, Copying and Pasting Response Options

The "up" and "down" arrow icons can be used for moving the selected response option up or down in the list. You can also highlight a single or multiple items and cut or copy-and-paste the items to a new position on this list, or to a response list for another select-type, ranking, or constant-sum question. Highlight the item(s) to be moved or copied, then click the Copy or Cut icons. Move the cursor to a new place on the list (or to a response list for a different question). Click the Paste icon.

Ranking Settings
Ranking Format Select whether to use Combo Boxes (drop-down control) or Numeric input Boxes. If you use combo boxes, integers 1 through n are automatically provided. You can only use these preset integers within combo boxes in a ranking question. If you want to use other characters or words, such as "best, middle, worst", you have that flexibility if specifying a ranking question within a grid. Respondent to Rank Indicate whether respondents are required to rank all items, or a subset of the items (such as ranking only the top three). A question is counted toward your license capacity for each response option that is ranked. Require Response to Question Specifies whether respondents are required to answer the question. Combo Box Settings (only applicable if Ranking Format is Combo Box)

General Web Interviewing (CiW) Help Specify the default text such as "click here" to display in the combo box for ranking. Formatting Response Options

65

On the Settings tab, the number of response options you specified for this question is displayed in the # Response Options field (which you cannot edit). Response options can be formatted in a single column, or over multiple columns by specifying a value greater than 1 in the Number of Columns field. For example, if you have 11 response options (say, colors) and specify four columns, the response options are distributed as follows: Which of these colors is your favorite? 1) Red 2) Purple 3) Pink 4) Blue 5) Yellow 6) Green 7) Violet 8) Brown 9) Orange 10) Magenta 11) Gray

Referencing Ranking Questions in SSI Script

Ranking Questions: Syntax: QuestionName_Response# Examples: Q1_4 [% Q1_4 %] Description: For ranking questions, each response option contains a numeric value. You refer to each response option within a ranking question using underscore "_" notation. For example, [% Q1_4 %] returns the numeric rank value for the fourth response option in Q1.

66 1.3.1.12

SSI Web v7

Free Format Questions

Warning: Because of the complexities of the Free Format Question type, you must completely test your survey to ensure that it functions properly and that the data for your entire survey are being saved correctly. We do not provide support to help you troubleshoot your own HTML and JavaScript. Response Types Currently the Free Format Question allows you to use any of the following types of variables: RADIO CHECKBOX SELECT (Combo Box) TEXT (Numeric) TEXT (Text Box) TEXTAREA HIDDEN

1.3.1.12.1 Introduction

Introduction
The SSI Web system provides pre-set templates for you to construct commonly used question types. Most users find these adequate to compose their surveys. However, some projects (or clients) may require that you create a certain look/layout with a question type that our templates don't allow. For such instances, the Free Format question type can provide those who know HTML near total control of the look of a question or entire survey pages containing many questions. For example, you can create complex "grid" questions, with combinations of numeric, open-end, check-box or radio button responses. To see an example of the possibilities, please see the "Free Format" study that was installed in your /Samples directory. You must be familiar with HTML to use the Free Format question type (the following URL has some of the basic HTML tags with descriptions: http://www.webmonkey.com/reference/HTML_Cheatsheet). As the name of the Free Format question suggests, you supply your own HTML code to display the question and input controls. Like all other question types, you can specify a Header 1, Header 2, and Footer text (if you are applying Styles to your survey, you may wish to utilize these question sections for better uniformity of look across both your standard and your free format questions.) SSI Web still provides the <form> tag and its parameters. But, most everything that you would put within (but not including) the <body></body> tags is up to you (with the exception of the "submit" button control). Free Format question HTML code cannot have any <html>, <body>, or <form> tags (like all CiW questions, SSI Web generates these tags). If you are applying Styles in your questionnaire (most surveys do), your HTML must include proper CSS "hooks" for the question to look like the others within your survey.

Adding a Free Format Question

A Free Format question can contain a single, or multiple questions (variables). If using multiple variables, they do not necessarily have to be all of the same response type. You can include

General Web Interviewing (CiW) Help

67

combinations of numerics, combo boxes, radio buttons, check boxes and open-end text fields within the same Free Format question. To add a Free Format question to your survey, click the Add... button from the Write Questionnaire dialog. The new Free Format question is inserted directly after any highlighted question in the List of Questions. If no question is highlighted, the new question is inserted at the end of the list. The Add to Questionnaire dialog is displayed. Choose the Free Format option. Type a question name in the Question Name field. Question names can include up to 24 letters or numbers (no spaces or underscores permitted, and cannot start with numbers). Please note that the name given to the Free Format question plus an underscore character that SSI Web supplies acts as a prefix to all of the variables you define within this particular Free Format question. For example, you may wish to add a multi-part numeric question that has three response blanks. If you name your Free Format question "FF1", you later assign three variable labels with FF1_ as a prefix (e.g. "FF1_A", "FF1_B"," FF1_C"). By requiring you to provide a unique prefix name for Free Format questions, SSI Web can help you make sure you do not accidentally give two different variables within your survey the same name, resulting in potential data loss. Click OK. The Free Format Question dialog is displayed, with three tabs: Question Text, Variables/Question HTML, and Skip Logic.

Question Text Tab

Like all other question types, you can specify a Header 1, Header 2, and Footer text. Even though free format questions let you specify all the HTML for your question (and you could specify all the HTML within the Variables/Question HTML tab), you may wish to type text/HTML into these three different sections so that if you apply Styles, your free format questions will carry many of the same style formatting as your other standard SSI Web questions.

Variables/Question HTML Tab

The Variables/Question HTML tab is divided into two parts. The lower section contains a text box where you compose your HTML code. (HTML can also be written in a separate editor and then copied and pasted into this section.) The upper section contains a list box of Free Format variables. Remember, a Free Format question actually can contain many sub-questions (Free Format Variables). The Free Format Variables you add to the Variable Names list must correspond exactly to the input tags and variable names you write within your HTML section (including distinction between upper and lower case). (If you use the Free Format Controls icon this helps ensure that your variable names match.) Each Free Format Variable counts as a CiW question against your license limitation.

Free Format Variables

Free Format Variables must have unique names that begin with the name of the Free Format question. When specifying the name of a variable, the Free Format question name is made the first part of the variable name automatically. It is very important that the full variable name be exactly the same as the name parameter for the corresponding input tags in your HTML code section (including case).

68

SSI Web v7

Error Messages to Respondents

You may also specify an Error Message for Free Format Variables. This message is used by the verification code to direct the respondent to the specific response item/question that needs to be corrected. For example, if you define a multi-part question with numeric blanks in which respondents type ratings for restaurant chains, you may define the Error Message as "McDonald's" for the text to be displayed to respondents if they forget to provide a score within the correct range for McDonald's. The error message might return: "I'm sorry, but you forgot to answer question McDonald's". The Error Message text you provide for Free Format variables is assigned to the ErrText internal variable when displaying error messages within the interview. To enable error messages for respondents that return the question name (or Error Message for Free Format variables), make the change under Compose | Write Questionnaire... | Survey Settings, Error Messages tab. Rather than use ErrField (that returns the question number on the current page that needs to be remedied), use ErrText. Note: If you turn off the JavaScript verification (or the respondent does not have JavaScript enabled) and use Perl verification instead (under Survey Settings | Advanced), if the respondent enters invalid data, when that Free Format question is re-displayed (along with the error message), the respondent will lose any previously entered data for that free format question. The respondent will need to re-enter before submitting again.

General Web Interviewing (CiW) Help

69

1.3.1.12.2 Radio
Warning: Because of the complexities of the Free Format Question type, you must completely test your survey to ensure that it functions properly and that the data for your entire survey are being saved correctly. General Information about Free Format Questions The RADIO variable is used for asking the respondent to select a single option from a set of response options. Within the Free Format Variable dialog, specify a Free Format Variable with RADIO as the Variable Type. SSI Web relies heavily on styles, and default behavior is to use graphical radio buttons. This increases the polish of the SSI Web questionnaire, but it raises the bar for the sophistication of HTML that should be supplied when writing free format questions.

Radio Buttons in Free Format

By default, SSI Web questionnaires use graphical radio buttons (specified under Survey Settings, General Format tab). The HTML used by SSI Web to display radio buttons is more complex than the standard HTML used to display radio buttons. You can use View Source within your browser to examine (and copy) the syntax SSI Web uses when displaying standard Select-type questions. But, a simpler solution is to use the Free Format Tools button that you access when editing the HTML section of Free Format questions using the the SSI Web function called RadioSelect. icon. It inserts template HTML, including the use of

When you click to add Radio buttons, a dialog appears that lets you specify vertical, horizontal, or no table. The table provides a structure for displaying the radio buttons. Depending on the layout of your radio button question, you'll select one of these options. We recommend you add the template structure, and click the Preview button to see which layout best fits what you want to accomplish. The name attribute for each <input> tag must be the name of the variable (all radio <input> tags should have the same name attribute). The VALUE attribute for each <input> tag should be set to a unique value, so you can determine which option the respondent selected (we recommend sequential integers starting with one).

70

SSI Web v7

1.3.1.12.3 Checkbox
Warning: Because of the complexities of the Free Format Question type, you must completely test your survey to ensure that it functions properly and that the data for your entire survey are being saved correctly. General Information about Free Format Questions The CHECKBOX variable is used for presenting a set of check boxes in your survey. Within the Free Format Variable dialog, specify a Free Format Variable with CHECKBOX as the Variable Type. Even though the response to each check box is stored independently as a unique variable name, you do not specify at this point each variable name associated with each check box. Specify the Number of Response Options, or the number of check boxes you have in the set. You may also specify the Min Checks and the Max Checks to force the respondent to check a certain number of check boxes (within the range you specified). Within the HTML editing area, specify an <input> HTML tag with the type set to "checkbox" for each check box in the set. The easiest way to do this is by using the Free Format Tools button . When using the Free Format Tools button, SSI Web automatically provides default HTML with appropriate variable naming conventions.

Checkbox Select in Free Format

By default, SSI Web questionnaires use graphical radiot buttons/checkboxes (specified under Survey Settings, General Format tab). The HTML used by SSI Web to display checkboxes is more complex than the standard HTML used to display checkboxes. You can use View Source within your browser to examine (and copy) the syntax SSI Web uses when displaying standard Select-type questions with checkboxes. But, a simpler solution is to use the Free Format Tools button that you access when editing the HTML section of Free Format questions using the including the use of the SSI Web function called CheckSelect. icon. It inserts template HTML,

When you click to add checkboxes, a dialog appears that lets you specify vertical, horizontal, or no table. The table provides a structure for displaying the checkboxes. Depending on the layout of your checkbox question, you'll select one of these options. We recommend you add the template structure, and click the Preview button to see which layout best fits what you want to accomplish. The name attribute for each <input> tag must be the name of the variable (all radio <input> tags should have the same name attribute). The VALUE attribute for each <input> tag should be set to a unique value, so you can determine which option the respondent selected (we recommend sequential integers starting with one).

General Web Interviewing (CiW) Help

71

1.3.1.12.4 Select (Combo Box)

Warning: Because of the complexities of the Free Format Question type, you must completely test your survey to ensure that it functions properly and that the data for your entire survey are being saved correctly. General Information about Free Format Questions To display a combo box type input within your Free Format Question, you define a Free Format Variable with SELECT (Combo Box) as the Variable Type. Within the HTML editing area, specify a <select> HTML tag in your HTML code. The easiest way to do this is by using the Free Format Tools button . When using the Free Format Tools button, SSI Web automatically provides default HTML with appropriate variable naming conventions. The name of the Free Format Variable must be the same as the name attribute in your HTML code for the <select> tag (including case). Checking Required Response requires a respondent to choose an option other than the first visible default option. Be sure to change the Default_option text between the <option> tags. Not checking the Required Response allows the respondent to skip this question. The MULTIPLE (multiple input list box) attribute is not yet supported in SSI Web.

SELECT (Combo Box) Example

(The following parameters represent a Free Format Question with one variable defined.) Free Format Question Name: Variable Name: Variable Type: Error Message String: Reject Default Response: Example HTML Code: Please select your favorite color: <BR> <SELECT NAME="FF1_COLOR"> <OPTION SELECTED VALUE="">Choose One</OPTION> <OPTION VALUE="1">Red</OPTION> <OPTION VALUE="2">Green</OPTION> <OPTION VALUE="3">Blue</OPTION> </SELECT> Note that we assigned numeric values to the response options. This permits you to tabulate responses within the Marginals program in the online administrative module. If "Require Response" is selected, SSI Web verification will not let the "Choose One" option to be selected. FF1 FF1_COLOR SELECT (Combo Box) Favorite Color (checked)

72

SSI Web v7

1.3.1.12.5 TextArea
Warning: Because of the complexities of the Free Format Question type, you must completely test your survey to ensure that it functions properly and that the data for your entire survey are being saved correctly. General Information about Free Format Questions The TEXTAREA variable is used for asking longer, open-ended text questions. Within the Free Format Variable dialog, specify a Free Format Variable with TEXTAREA as the Variable Type. Within the HTML editing area, use a <TEXTAREA> HTML tag in your HTML code. The easiest way to do this is by using the Free Format Tools button . When using the Free Format Tools button, SSI Web automatically provides default HTML with appropriate variable naming conventions. The name of the Free Format Variable must be the same as the name attribute in your HTML code for the <TEXTAREA> tag (including case). If desired you can check the Require Response option and then specify the minimum number of characters you require the respondent to enter. SSI Web forces the respondent to enter at least the minimum and no more than the maximum number of characters before continuing to the next question.

TEXTAREA Example (The following parameters represent a Free Format Question with one variable defined.) Free Format Question Name: Variable Name: Variable Type: Require Response: Error Message String: Minimum Characters: Example HTML Code: Please tell us about your past work experience: <BR> <TEXTAREA NAME="FF2_WORKEXP" ROWS="3" COLS="35"> </TEXTAREA> (Regardless of the length permitted in the textarea box, responses are trimmed to a maximum of 10,000 characters to maintain data file integrity.) FF2 FF2_WORKEXP TEXTAREA (checked) Work Experience 30

General Web Interviewing (CiW) Help

73

1.3.1.12.6 Text (Text Box)

Warning: Because of the complexities of the Free Format Question type, you must completely test your survey to ensure that it functions properly and that the data for your entire survey are being saved correctly. General Information about Free Format Questions The TEXT (Text Box) variable is used for asking shorter text questions. Within the Free Format Variable dialog, specify a Free Format Variable with TEXT (Text Box) as the Variable Type. Within the HTML editing area, use an <input> HTML tag with the type attribute set to "text." The easiest way to do this is by using the Free Format Tools button . When using the Free Format Tools button, SSI Web automatically provides default HTML with appropriate variable naming conventions. The name of the Free Format Variable must be the same as the name attribute in your HTML code for the <input> tag (including case). If desired, you can check the Require Response option and specify the minimum and maximum number of characters you wish the respondent to enter (similar to the TEXTAREA variable) before continuing.

TEXT (Text Box) Example

(The following parameters represent a Free Format Question with one variable defined.) Free Format Question Name: Variable Name: Variable Type: Require Response: Error Message String: Minimum Characters: Example HTML Code: Last Name: <BR> <INPUT TYPE="text" NAME="FF3_LASTNAME" SIZE="100"> *Note: The INPUT tag has a SIZE attribute that determines the size of the text box. It also has a MAXLENGTH attribute that limits the maximum number of characters that can be typed into the box. Regardless of the number of characters specified, responses are trimmed to a maximum of 10,000 characters to maintain data file integrity. FF3 FF3_LASTNAME TEXT (Text Box) (checked) Last Name 2

74

SSI Web v7

1.3.1.12.7 Text (Numeric)

Warning: Because of the complexities of the Free Format Question type, you must completely test your survey to ensure that it functions properly and that the data for your entire survey are being saved correctly. General Information about Free Format Questions The TEXT (Numeric) variable is used for asking numeric only type questions. Within the Free Format Variable dialog, specify a Free Format Variable with TEXT (Numeric) as the Variable Type. Within the HTML editing area, use an <input> HTML tag with the type attribute set to "text." The easiest way to do this is by using the Free Format Tools button . When using the Free Format Tools button, SSI Web automatically provides default HTML with appropriate variable naming conventions. The name of the Free Format Variable must be the same as the name attribute in your HTML code for the <input> tag. If desired, you can check the Require Response option, that forces respondents to enter a numeric value. You must specify a Minimum Value and a Maximum Value for the variable. SSI Web requires the respondent's answer to be within that specified range. If you would like the respondent to be able to enter decimal type values, then check the Allow Decimal Answers check box.

TEXT (Numeric) Example

(The following parameters represent a Free Format Question with one variable defined.) Free Format Question Name: Variable Name: Variable Type: Require Response: Error Message String: Minimum Value: Maximum Value: Allow Decimal Answers: Example HTML Code: How many cars do you currently own? <INPUT TYPE="text" NAME="FF4_CARS" SIZE="2" MAXLENGTH="2"> FF4 FF4_CARS TEXT (Numeric) (checked) Number of Cars 0 10 (not checked)

General Web Interviewing (CiW) Help

75

1.3.1.12.8 Hidden Variables

This is an advanced area for declaring a hidden variable using JavaScript and setting its value.

Example #1: Setting the hidden variable equal to a random number used in the survey. Suppose you declared a random value earlier in the survey (such as "RandNum(53)") and wanted to store the value of that random number in your data set. 1. Add a Free Format question to your survey, and a variable (e.g. myvar1_rand) of type hidden. 2. Edit the HTML (using the "pencil" icon , and using the Free Format Tools button myvar1_1 from the drop-down control list, to insert the following text in your HTML: <input name="myvar1_rand" type="hidden"> Then, edit that text to assign the value equal to the function RandNum(53): <input name="myvar1_rand" type="hidden" value="[%RandNum(53)%]"> select

Example #2: Setting the hidden variable equal to the sum of two numeric variables on a previous screen. Assume the two previous numeric variables were Value1 and Value2. 1. Add a Free Format question to your survey, and a variable (myvar1_sum) of type hidden. 2. Edit the HTML (using the "pencil" icon insert the following text in your HTML: , and select the Free Format Tools button to

<input name="myvar1_sum" type="hidden" value="[%Q1+Q2%]">

76 1.3.1.13

SSI Web v7

JavaScript in SSI Web

JavaScript in SSI Web JavaScript Introduction JavaScript Basics Custom JavaScript Verification SSI Web JavaScript Library JavaScript Debugging

JavaScript Introduction JavaScript is a scripting language that adds enhanced functionality and programming power to a web page. JavaScript code can verify that the input on a web page is correct before submitting it to the server. It can be used to do calculations on user input. Together with CSS, it can be used to change the appearance of the web page in real time. It is important to understand that JavaScript runs in the web browser, not on the server. SSI Web makes extensive use of JavaScript. It is used to help validate responses to questions, fill in Constant Sum total boxes in real time, and to un-check select options when an "Other Specify" is chosen, etc. Most of SSI Web's system JavaScript is contained in ssi_javascriptX_X_X.js located in the graphics/system directory. SSI Web also includes the MooTools JavaScript library. More information about this library can be found at http://www.mootools.net. For a complete tutorial on JavaScript see http://www.w3schools.com/js.

JavaScript Basics How to Include Custom JavaScript in SSI Web Variables Flow Control Functions Form Objects How to Include Custom JavaScript in SSI Web JavaScript can be included in an SSI Web survey almost anywhere text can be entered. For example, you could place some JavaScript code in the header of a question or as part of a response option. To include JavaScript on a page in SSI Web, you enclose JavaScript code in <script> </script> tags. For example, suppose you wanted to display a message in a pop-up box on the first page of your survey. You could place this text in Header 1 of the first question: <script type="text/javascript"> <!-alert("Welcome to the survey."); //--> </script> The JavaScript code between the <script> tags is executed when the page is loaded. In this case, as

General Web Interviewing (CiW) Help

77

soon as the web page is loaded in the browser, a box pops up with the message "Welcome to the survey". Any JavaScript that needs to be available to the whole survey can be included in the HTML <head> Tag (Survey Settings, Headers and Footers). JavaScript can also be included as an attribute of an HTML tag. For example: <a href="#" onclick="alert('hello');">Click Here</a> When the link above is clicked, a message box with the message "hello" appears. JavaScript can also be included in a separate file so that it can be accessed by multiple pages in your survey. A text file can be created containing your JavaScript code. This file can then be placed in your graphics folder and included with the following syntax: <script type="text/javascript" src="[%GraphicsPath()%]CustomJavaScriptFile.js"></script> The text above should be placed in the HTML <head> Tag (Survey Settings, Headers and Footers).

Variables Variables in JavaScript can be declared with the "var" statement. For example: var NumDaysInWeek = 7; var MonthName = "March"; After this code is executed NumDaysInWeek will contain 7 and MonthName will contain "March". Notice that quotes are used when assigning text to a variable. To retrieve the value of a question on the current page you can use SSI Web's built-in SSI_GetValue(QuestionName) JavaScript function. Given any question name (or variable name) this function will return the value entered for that question on the current page. For example: var Q1Value = SSI_GetValue("Q1"); var BlueChosen = SSI_GetValue("FavColors_1"); In the above example, the function SSI_GetValue retrieves the value for Q1 entered on the page. It is then assigned to the Q1Value variable. The next line retrieves the value for the first check box in a Select question. BlueChosen is assigned a 1 if it is checked and a 0 otherwise. To retrieve the response to a question from a prior page and use that value in your JavaScript, you should use SSI Script in conjunction with JavaScript. For example, if NumWeeks was asked on a prior page of the survey and you wanted to use its value to do a simple calculation and display the result on the current screen, the following JavaScript could be used: var NumDaysInWeek = 7; var Weeks = [%NumWeeks%]; document.write(NumDaysInWeek * Weeks); Before the web page comes down to the browser the SSI Script [%NumWeeks%] is changed to the value that was entered on a prior page by the respondent. Lets assume that value was 4. The JavaScript loaded by the browser is: var Weeks = 4;

78

SSI Web v7 To load text from a prior question, use: var Name = "[%LastName%]"; which when it arrives at the browser would be: var Name = "Smith"; Arrays, which are variables that can hold a collection of items, are also available: var ItemList = [1,2,3,4,5,6,7]; document.write(ItemList[0]); In the example above, an array of 7 items is declared. The first item in the array (with value 1) is written to the screen. Individual array items can be accessed with 0-based index numbers between [ and ]. So [0] returns the 1st item, [1] the 2nd, and so on.

Flow Control JavaScript "if" statements can be used to conditionally execute code: var Age = SSI_GetValue("Q1"); if(Age < 18) { alert("You do not qualify for this survey."); } In the code above, the value entered for Q1 is assigned to Age. If Age is less than 18 a message box appears. If there are multiple paths that can be taken the "else if" statement can be used. For example: var Age = SSI_GetValue("Q1"); if(Age < 18) { alert("You do not qualify for this survey."); } else if(Age < 65) { alert("You qualify for group 1."); } else if(Age >= 65 && Age <= 125) { alert("You qualify for group 2."); } else { alert("Are you sure you are older than 125?"); } If the first "if" statement is false, the next "else if" logic is tested, and so on. If a section evaluates to true, its code is executed. Execution flow then continues after the final "else" or end of the block.

General Web Interviewing (CiW) Help The following symbols are used in conditional logic: "&&" for AND "||" for OR "==" for equal <, >, <=, >=, etc. The "for loop" allows your code to loop for a given amount of iterations. For example: var myArray = [1,2,3,4,5]; var i = 0; var sum = 0; for(i = 0; i < myArray.length; i++) { sum += myArray[i]; } alert(sum);

79

The variable "i" starts at 0 and counts up to the length of the array (in this case 5). The code inside the "for" loop block is executed 5 times. The code accesses each item in the array, storing the sum of all the items in the "sum" variable.

Functions Functions are sections of code that can be defined and used when needed by "calling" them. For example: function AddTwo (Num1, Num2) { var Result = Num1 + Num2; return Result; } alert(AddTwo(4, 6)); //The value 10 will appear in a pop-up box. alert(AddTwo(56, 33); //The value 89 will appear in a pop-up box. In the code above, a function named "AddTwo" is defined. This function takes two numbers for input named "Num1" and "Num2". Any two numbers that are passed into this function are added and the result is returned. Functions are best defined in the HTML <head> Tag (Survey Settings, Headers and Footers) or in a separate file that is included in the HTML <head> Tag. This makes it so that the functions are available throughout the whole study.

Form Objects Each survey page created by SSI Web contains an HTML form tag named "mainform". All of the HTML input elements for the questions on this page are inside this form. Using JavaScript, you can get access to JavaScript objects that represent these form elements. Having access to these form elements gives you access to their properties and functions. The following JavaScript code assigns a form object to a variable:

80

SSI Web v7 var Q1Obj = document.mainform["Q1"]; This can also be done by calling an SSI Web JavaScript system function like this: var Q1Obj = SSI_GetFromObject("Q1"); Now Q1Obj can be used to get and set values and to set up events etc. For example: Q1Obj.value = 5; Q1Obj.addEvent("keyup", SetToFive); function SetToFive() { this.value = 5; } Here, we are assigning a 5 to the "Q1" numeric question. This causes a 5 to appear in the numeric box on the screen. The next line of JavaScript then assigns an event to the question. If the respondent types into the Q1 box every time a key is pressed and released ("keyup") then the function "SetToFive" is called. In this function "this" represents "Q1Obj". This code essentially makes it so that the respondent cannot alter the value for "Q1". The events "keyup", "click", "mouseover", "mouseout", etc. can be used. To get a value from a question on the page, use SSI_GetValue("QuestionName"). This function handles the complexities of different question types (radio, checkbox, numeric, etc.).

Custom JavaScript Verification Warning: This is an advanced area, intended for those who know JavaScript. You must completely test your survey and custom JavaScript code to ensure that it functions properly and that the data for your entire survey are being saved correctly. Sawtooth Software does not support your custom JavaScript. There might be times when our default question verification will not accomplish the functionality you want. Most questions (with the exception of the password, Terminate/Link, and conjoint questions) have a Custom JavaScript Verification section. This is available by editing the question, clicking on the "Advanced" button, and then selecting the "Custom JavaScript Verification" tab. Custom JavaScript Verification allows you to specify custom verification for your questions. Your custom verification, written in JavaScript, is invoked when the respondent clicks the "Submit" button. You can configure your custom JavaScript to be called "before" or "after" the built in question JavaScript verification. Your respondents must have JavaScript enabled (most do) in order for your custom JavaScript to work.

General Web Interviewing (CiW) Help

81

In the example above, an error message is displayed and respondents cannot continue the survey if they have specified that they are older than their father. The age of the respondent ("respondentAge") and of the father ("fatherAge") are asked on the current page. These values have not been submitted to the server yet or recorded in the data file. They are on the same page where this custom JavaScript is executed. The values of questions on the same page as the JavaScript that you are writing are available through the following JavaScript: SSI_GetValue("QuestionName") The JavaScript SSI_GetValue is defined in SSI Web's system JavaScript file ssi_javascriptX_X_X.js located in the graphics/system directory. In the example above if the value of "respondentAge" is greater than or equal to "fatherAge" then "strErrorMessage" is set to a custom error message. Setting strErrorMessage equal to something other than empty will cause the system JavaScript to display an error message on the screen and highlight the current question. The current page will also not submit to the server as long as strErrorMessage has a value assigned to it. If multiple questions on a single page all have custom JavaScript verification, then each section of custom JavaScript is run in order from top to bottom. If you would like to retrieve information from previous pages you need to use SSI Script. If the respondent's age was submitted on a previous page, the JavaScript would then employ SSI Script and look like this:

82

SSI Web v7 if([%respondentAge%] >= SSI_GetValue("fatherAge")) SSI Script executes on the server and the respondent's age (lets assume it is 37) is inserted. The resulting JavaScript is sent down to the browser and looks like this: if(37 >= SSI_GetValue("fatherAge"))

SSI Web JavaScript Library SSI Web comes with a system JavaScript file named ssi_javascriptX_X_X.js. It is located in the graphics/system directory and is included and available on every page of the survey. The system JavaScript file is used by SSI Web to help validate responses to questions, fill in Constant Sum total boxes in real time, and to un-check select options when an "Other Specify" is chosen etc. There are a few functions from this library that you might find useful. SSI_GetValue("QuestionName") Returns the current value for the specified QuestionName, for any question type (radio, checkbox, numeric, etc.). A zero ("0") is returned for questions that have been left empty. Example: var Num1 = SSI_GetValue("Q1"); // returns the value of question Q1 var FirstStateChecked = SSI_GetValue("State_1"); // If the first check box of question "State" is checked 1 will be returned, otherwise 0. SSI_GetFormObject("QuestionName") Returns the object associated with the form element on the page for QuestionName. Example: var Num1Obj = SSI_GetFormObject("Q1"); // Assigns the form object for question "Q1" to "Num1Obj". Num1Obj.value = 46; // Assign the value of 46 to the Q1 question. SSI_RoundNumber(Number, NumDecimalPlaces) Round "Number" to "NumDecimalPlaces". Example: var Value1 = SSI_RoundNumber(SSI_GetValue("Q1"), 2); // SSI_GetValue returns the value for Q1 it is then rounded to 2 decimal places. alert(Value1); // Assuming Q1 was 45.236 the Value1 would yield 45.24 SSI_PriceFormat(Number, "ThousandSymbol", "DecimalSymbol") Formats a number with the thousands symbol and decimal symbol. Example: var FormatValue = SSI_PriceFormat(5123693.25, ",", "."); // 5,123,693.25 is returned. SSI_CustomGraphicalCheckbox(GraphicalCheckboxObj, InputObj, blnCheck) SSI_CustomGraphicalCheckbox is a function that has been provided so that you can write custom code that gets called when a graphical checkbox is clicked. This function must be defined by you in your custom JavaScript. The function will be called by SSI Web's system JavaScript each time a check box is clicked. GraphicalCheckboxObj represents the graphical item on the page that is being clicked. InputObj is

General Web Interviewing (CiW) Help

83

the actual hidden checkbox on the page that the graphical checkbox represents. blnCheck is true if the checkbox has been checked, otherwise it is false. Example: function SSI_CustomGraphicalCheckbox(GraphicalCheckboxObj, InputObj, blnCheck) { if(InputObj.name == "mySelect_1" && blnCheck == true) { alert("The first box was checked"); } } SSI_CustomGraphicalRadiobox(GraphicalRadioObj, InputObj) SSI_CustomGraphicalRadiobox is a function that has been provided so that you can write custom code that gets called when a graphical radio button is clicked. This function must be defined by you in your custom JavaScript. The function will be called by SSI Web's system JavaScript each time a radio box is clicked. GraphicalRadioboxObj represents the graphical item on the page that is being clicked. InputObj is the actual hidden radio button on the page that the graphical radio button represents. When ever this function gets called the radio button represented by InputObj will have been checked. Example: function SSI_CustomGraphicalRadiobox(GraphicalRadioboxObj, InputObj) { if(InputObj.name == "mySelect") { alert("A radio button with the value " + InputObj.value + " has been selected."); } }

JavaScript Debugging You are responsible to ensure your custom JavaScript is working. The following tips are highly recommended when trying to debug your custom JavaScript: Turn on script error reporting. This is done in Internet Explorer by clicking Tools | Internet Options | Advanced. Under the Advanced tab go to the Browsing section and check Display a notification about every script error. This is extremely useful in alerting you to mistakes in your code. Use the "alert( )" function. If you are unsure what certain values are at certain points in your code, you can print them out to the screen by using the "alert( )" function. For example: alert("My value at this point is: " + SSI_GetValue("Q1")); Firefox has a plug-in named "Firebug" that provides a powerful JavaScript debugger. Test your custom JavaScript in multiple browsers (e.g. Microsoft Internet Explorer, Firefox, etc.).

84

SSI Web v7

1.3.2

How CiW License Sizes are Counted

CiW is licensed with a specific capacity, counted in terms of stored data fields (not questions). The sizes are: CiW/100 CiW/500 CiW/1000 CiW Unlimited (If you purchase CBC, ACA, CVA, ACBC, or MaxDiff without an accompanying CiW license, you receive a CiW/50 license capacity.) Many question types use just one data field (variable) per question: Numeric Select (Radio button or Combo box) Open-end Some question types don't use any stored data fields: Text/HTML Filler Terminate/Link Select-type questions with multiple responses use more than one data field (variable) per question, but we count these as just one stored data field toward your license capacity. The example below stores three data fields, but we count only one against your license capacity:

When grid questions are used, these can involve multiple data fields counted against your license. In the grid question below, there are four separate radio button variables, one variable for each row. This grid question below uses four data fields toward your license capacity.

This grid question below actually requires eight data fields to store a respondent's answers ("checked" vs. "not checked" for each box). But, we count only two data fields toward your license capacity in this

General Web Interviewing (CiW) Help case.

85

The use of constructed lists can greatly impact your data field usage. Consider a list of colors, such as: 1. 2. 3. 4. 5. 6. 7. 8. Red Green Blue Yellow Purple Peach Pink Orange

Let's also assume that we'll first ask respondents which colors they like (one data field counted toward your license capacity for this multiple-response question). Next, we construct a new list that includes only the colors each individual prefers. If we show only the colors preferred as part of a grid question, and ask respondents to rate each preferred color, SSI Web counts eight data fields toward your license, even though each individual would likely see just a subset of the eight colors in the follow-up question. "Other specify" responses (in select-type questions) add an additional field per other specify, and are counted as a question toward your license capacity. Ranking questions count one question against the license for each rank that is cast. Therefore, if ranking ten options, ten questions are counted. If ranking just the top three of ten options, then three questions are counted.

When you click the icon from the Write Questionnaire dialog, SSI Web prints the number of data fields used by each question in your survey.

86

SSI Web v7

1.3.3

What Happens at the End of the Survey?

You should plan what the respondent does at the end of your survey. Your survey can "dead end" (default), you can send respondents to a specific site, or you can link respondents to another survey. To terminate a survey, you should place a Terminate/Link Question question within your survey. Creating Your Own Hyperlink to Send Respondents to a Specific Site SSI Web's Terminate/Link question can automatically link a respondent to a specific site. However, you can also create your own hyperlink (clickable area) that sends respondents to a specific web page or site. If using this method, you should still include a Terminate/Link Question and provide proper settings regarding the respondent's status (complete or incomplete) when reaching this question. You can then embed a hyperlink (following HTML conventions) or button in the header, body or footer section of the Terminate/Link question. As an example, the following text creates a hyperlink to send a respondent back to Sawtooth Software's home page: Click <a href= "http://www.sawtoothsoftware.com"> HERE </a> to finish the survey and go to Sawtooth Software's home page. (You can add hyperlinks when using SSI Web's text editor by clicking the "anchor" icon, and customizing the address for the HTML that is automatically inserted.) You can also embed a clickable graphic that sends respondents to a specific web site. Assume that you created an attractive graphic named "finish.gif" that says "Finish Survey" with your logo in the background. If you wanted respondents to click the graphic to go to the Sawtooth Software web site, you would first add a hyperlink using the "anchor" icon. Then, within the anchor HTML, you would add the graphic by clicking the Add Image icon , and selecting the graphic to add. The resulting HTML might look like: <a href = "http://www.sawtoothsoftware.com.com"> <img src= "[%GraphicsPath( )%]finish.gif" BORDER="0"> </a> If you are using this software to conduct repeated central-site or mall-intercept surveys, you may want the survey to return to the Password (login) screen. In that case, specify "STUDYNAMElogin.html" as the destination in your hyperlink. You are not limited to one hyperlink--multiple "exit" options are possible. Do not embed a hyperlink in the middle of your survey as this would lead to incomplete interviews.

General Web Interviewing (CiW) Help

87

1.3.4
1.3.4.1

Formatting Options
Survey Settings (Style)
The Survey Settings dialog (accessed by clicking or by clicking Compose | Survey Settings...) lets you specify settings that apply to all questions within your questionnaire. It has eight tabs: Style General Format Page Format Headers and Footers Progress Bar Browser Settings Error Messages Server Settings Advanced The Style tab on the Survey Settings dialog lets you select different Styles (implemented using Cascading Style Sheets, or CSS) to apply to your entire survey. This dramatically changes the appearance of your survey, creating some unique and professional displays. Select from the styles currently shipping with the software (as shown in the Styles Library window), or develop and import your own styles. Please note that the style you select controls many aspects of the survey, including text formatting, fonts, and bolding. This has the benefit of controlling the entire look of your survey, without the need to modify the formatting for each question individually. If you want to modify an aspect of the style that applies to all questions in the survey, you can do so on the Page Format tab via the Paintbrush icons. If you want to modify an aspect of the style just for a specific question, you can do that by editing the question and using the Paintbrush icons.

To use a style, highlight the style you wish to use within the Style tab of the Survey Settings dialog. A sample graphic is shown to give you some idea regarding the look of the style. To apply the style to the study, select the desired style and then click the Use Selected button. To discontinue using styles, click No Style.

Using Styles
From the Compose | Survey Settings dialog, select the Style tab. A list of System Styles and any User Styles that you might have created are shown. User Styles are shown toward the bottom of the list. Duplicate style names are not allowed. When you choose a style from the library and click Use Selected the appropriate style folder is copied to your study's graphics folder. Your study then uses this style to alter its formatting.

88

SSI Web v7

How Styles Work

Overview CSS Basics SSI Web HTML "Hooks" for CSS Modify an Existing Style Create Your Own Style Overview By default, the style settings throughout SSI Web are empty. For instance if you create a new question and click the Paintbrush icon for Header 1 there is no font size specified. If no font size is specified at the question level, the font size specified at a higher level is used. SSI Web uses CSS (Cascading Style Sheets) to control the style in surveys. CSS features cascading levels of priority. A font size can be specified for all text in a survey or for a particular question. A font size defined for a particular question overrides the font size defined at the study level. The diagram below helps illustrate how CSS is used in SSI Web, demonstrating the different levels of Style priority. Styles defined at the question level in SSI Web override styles defined in the survey style. Styles defined in the survey style override styles defined in the base style sheet ssi_style.css.

CSS Basics CSS stands for Cascading Style Sheets. CSS allows you to modify the style (background color, font style, spacing, etc.) of the HTML for web pages. CSS allows you to separate the style from the structure of a web page. CSS can be defined within the HTML elements. For example:

General Web Interviewing (CiW) Help <p style="color: green;"> Please answer the questions below. </p>

89

The CSS "color: green;" changes the text color within the <p> tag to green. This style can be separated from the <p> tag. The following instructions: <style type="text/css"> p { color: green; } </style> <p> Please answer the questions below. </p> change all of the text within <p> tags on page to green. Suppose that you have multiple web pages and you want the text inside all <p> tags for all pages to be green. You can create a separate text file named "mystyle.css" that includes the following CSS: p { color: green; } At the top of each web page inside of the <head> tag you would then include: <link rel="stylesheet" type="text/css" href="mystyle.css"> This controls the style for all web pages from one file. To change the color from green to blue you would simply edit mystyle.css. Using CSS in this way makes changing the style across your web pages much easier than defining the style on each individual page. Styles defined in an external file have lower priority than styles defined within a <style> tag on the web page. Styles defined within the HTML inside of a "style" attribute have a higher priority than styles defined within the <style> tag. CSS consists of various style "rules" that define how to modify the appearance of the HTML. In the CSS rule below the "p" is the selector and the "color: green" is the declaration. p { color: green; } The selector identifies which part of the HTML will be affected by the declaration. The declaration consists of a property and a value separated by a colon. Multiple declarations are separated by semicolons. For example: p { color: green; font-size: 18px; padding: 10px;

90

SSI Web v7 } The selector can be the name of any HTML element. For example: p, div, span, table, tr, td, etc. Multiple elements can be added to the rule separated by commas. For example: div, p, td { color: green; } The CSS above changes text within <div>, <p>, and <td> tags to green. You can also specify that only <p> tags within <div> tags should be selected: div p { color: green; /*Turn the text green*/ } Notice that there is a space separating "div" from "p". This indicates that only <p> tags within <div> tags are affected. Also notice the comment "Turn the text green." CSS comments are contained between /* and */. Any text between /* and */ will not affect the CSS. A Class Selector is a way that you can identify specific HTML elements. Suppose you have the following HTML: <div class="header1"> How old are you? </div> The <div> above has an attribute named "class" with a value of "header1". You identify a class in CSS by placing a "." in front of its name. For example .header1 { color: green; } The CSS above makes the text within tags with the "header1" class be green. SSI Web uses class selectors heavily. Classes are designed to identify one or more elements on the web page. If there is a unique element identified with an "id" then a "#" sign is used instead of a ".". For example: <div id="Q1_div"> How old are you? </div> The following CSS could be used to select this <div>: #Q1_div { color: green; } SSI Web includes many classes and IDs within the HTML that it creates to help you identify specific areas of the page. You can then use these "class" and "id" names within your CSS. More information on CSS can be found at http://www.w3schools.com/css.

General Web Interviewing (CiW) Help

91

SSI Web HTML "Hooks" for CSS SSI Web generates the HTML for your survey based on the different question types and settings you configure in the software. The HTML contains "class" and "id" attributes in many of its tags. For example the HTML for Header 1 looks like this: <div class="header1"> Header 1 Text </div> The "class" attribute in the <div> tag above identifies Header 1. The following CSS can then be used to change the Header 1 text color throughout the whole survey: .header1 { color: green; } The diagram below shows some of the main classes present in SSI Web's HTML.

Additional classes and ids can be examined by viewing SSI Web's HTML. This can be done using "View Source" in your web browser. Some HTML elements in SSI Web contain multiple classes and an id. Various ways have been provided to allow the CSS to manipulate the presentation or style of the survey. The HTML for a select question named "Q1" looks like this: <div id="Q1_div" class="question select"> ...

92

SSI Web v7 </div> Notice that two classes "question" and "select" have been assigned to this <div> tag. The following CSS changes the Header 1 style for Q1. No other questions are affected. #Q1_div .header1 { color: blue; } The # sign above identifies the HTML tag with the "Q1_div" id. After "#Q1_div" there is a space and then ".header1". This CSS rule changes the text for Header 1 to blue inside question Q1. The following draws a black border around all questions: .question { border: 1px solid black; } The following changes the border only for Select questions to green: .select { border: 1px solid green; }

Modify the Look of an Existing Style

To modify an existing System Style, from the Style tab click the icon. Browse to the system style folder (C:\Program Files\Sawtooth Software\SSI Web\Styles) and select the style you wish to modify. When you select the style folder and click OK, SSI Web copies this style folder into your User Styles folder and asks you to rename it (so that there aren't duplicate names for styles). As an example, let's say that you want to change the background color of the charcoal style to green. First, from the Style tab, click the icon and browse to the System Styles folder (C:\Program Files\Sawtooth Software\SSI Web\Styles) and select charcoal from the list. Click OK, and you are prompted to change the name so that there aren't duplicate style names. Specify greencharcoal as the new style, and SSI Web renames the style and copies it into your User Styles folder: C:\Users\<user>\Documents\Sawtooth Software\SSI Web\Styles Next, you modify the contents of the greencharcoal.css file to modify the style. To change the background color to green, open the greencharcoal.css file with a text editor (e.g. Notepad). At the top of the CSS file you will see: /* Background setting for whole page, currently grey */ body { background-color: #DDDDCC; } You should change background-color: #DDDDCC; to background-color: green; (or you can directly

General Web Interviewing (CiW) Help

93

specify the hex code for green rather than use the key word "green") and save the file. You can also change the contents of the comment contained between /* and */. You might also delete the preview.jpg since it no longer reflects the modified style. You could capture a screen shot of the modified style and save it (size the graphic to a maximum of 500 x 375 pixels) in the greencharcoal folder as preview.jpg. Next, highlight the greencharcoal style in your style list and click Use Selected. The greencharcoal style folder with its .css file are copied into your study folder, within the graphics directory. Now, when you preview your study you will see the green background. If you need to continue to make modifications to your customized .css file, you should recognize that when you click Use Selected, the .css file being used is now within your study folder. You should continue to make changes there if you want to see the changes reflected in the current study. If you want to use your modified style in the future for other studies, you should copy the .css file from your study directory back into your User Styles folder.

Create Your Own Style (User Styles)

The best way to learn how to create your own style is by examining one of the existing system styles in the System Style Library that ships with SSI Web. These styles are located by default in: C:\Program Files\Sawtooth Software\SSI Web\Styles To create your own style, create a new folder within your User Styles area (C:\Documents and Settings\<user>\My Documents\Sawtooth Software\SSI Web\Styles Library) and give it the name of your new style. You will then need to create a CSS file with the same name as your style and place it inside the folder. For example, to create a style named red, create a folder also named red and place a file named red.css inside it. Place this folder in the User Styles path on your hard drive or import it into the Style Library by going to the Style tab and Style Library, clicking the icon then by browsing to the red folder. Highlight the new style and click Use Selected to apply it to your study. A graphics file named preview.jpg can optionally be placed in your newly created style folder. This file should contain a screenshot of the style and is used in SSI Web to give a small preview of what the style looks like. We recommend the graphic be sized to 500 x 375 pixels.

94

SSI Web v7

1.3.4.1.1 General Format

Display
Browser Title This field lets you specify the text that appears in the title bar of your browser (very top of browser window). This lets you further customize the browser appearance for your survey. For example, you may want the browser title bar to say: "ACME Customer Satisfaction Study 2007". Font You can specify the font face used for text in the survey. Type a font face, such as Times or Arial. Do not put quotes around the font face. The font face doesn't apply to text in response fields for the open-end, numeric or combo box question types. It is best to use a standard font, since respondents may not have the particular unique font you might want to use installed on their machine. If the font you specify is not available on the respondents machine, the browser will choose a default font. If you specify multiple fonts separated by commas, if the first font is not found on the respondent's machine, the second font will be attempted, and so on. Font Format You may override the Font Face specified in the Style along with other characteristics of font display (bolding, italics, etc.) using this control. Sawtooth Software Logo If you don't wish to display the Sawtooth Software notification on the last page of your survey, uncheck the Display "Powered by Sawtooth Software" on the last page box.

Navigation Buttons
Text Respondents click the Submit and Previous buttons at the bottom of the pages of the survey. You customize the text that appears on this button by typing text directly into the Submit Button or Previous Button fields. Graphics Styles by default have graphics buttons (for Submit and Previous buttons) that are applied if you choose to use graphical navigation buttons. You can select other graphics by selecting Settings... and browsing for the graphic using the Camera icon. When you select the graphic in this manner, it is copied into your Graphics folder when you prepare for web upload. If you browse to the graphic, the correct HTML and an appropriate path is configured for your graphic using the GraphicsPath() SSI Scripting syntax. We have provided some sample "Next" and "Previous" button graphics in the Graphics Library folder. Using graphics requires respondents to have JavaScript enabled on their browsers. If the respondent does not have JavaScript, the respondent will see a generic next button with the following label ">>". Button Justification Specify whether the Submit button is left justified, center justified, or right justified on the page, by selecting a Submit Button Justification option.

General Web Interviewing (CiW) Help

95

Respondent Input
You can use graphical radio and checkboxes, which lets you polish the look of your survey, as well as make the buttons any size you like. If you decide to create your own graphical buttons, you'll need to create a graphic for the checked and also the unchecked state of the button.

96

SSI Web v7

1.3.4.1.2 Page Format

Page Format
The Page is the broader area within the browser on which the questions are positioned. By default, the settings are blank, meaning that these settings are being controlled by the Style that is selected. You can override the Style settings for Page Width, Page Border, Page Justification, Space Between Questions, Page Background Color and Screen Background Color. Screen Background Color This sets the color for the area surrounding the main body of the questionnaire, extending to the edges of the browser window.

Question Format
Here, you can override the settings in the Styles for all questions in your survey. The Question area is subset of the total Page area. By default, the settings are blank, meaning that these settings are being controlled by the Style that is selected. Click the Paintbrush icons to override the Style settings for Header 1, Header 2, Question, and Footer sections of the question. You may also override the Style settings for all questions for the Question Width, Question Indent, Question Justification, Question Background Color, and Alternating Colors.

General Web Interviewing (CiW) Help

97

1.3.4.1.3 Headers and Footers

Page Header and Footer

This section allows you to specify a general header and footer that appears on every page of your survey. Headers and footers can contain text, HTML formatting commands, and graphics. A Preview button lets you see how your Web browser displays the header and footer you provide. A common use of headers and footers is to print some company information, logos, or a phone number to call for help if respondents have questions about the survey.

<Head> Tag
This area lets you specify additional information to include between the HTML <HEAD> and </HEAD> tags. Here are some examples of things you might include: CSS Tags JavaScript Meta tags

98

SSI Web v7

1.3.4.1.4 Progress Bar

Sometimes it is useful to display a progress bar at either the top or bottom of each page to let respondents know how far they have come in a survey and approximately how far they have to go. Show Progress Bar You can automatically include a progress bar in the Page Footer or Header by checking this box and selecting the position. You can exert additional control over the positioning of the progress bar by placing it manually within the global header or footer (see below). Manually Including a Progress Bar To manually place a progress bar in the questionnaire, simply type [%ProgressBar( )%] within the text boxes of the Survey Settings dialog box under Headers and Footers tab. The progress bar won't appear on the login page. Progress Bar Settings Alignment: Choose from left, center, or right. Width (%): Specifies the width of the progress bar, as a percentage of the total width of the browser window. Beginning Value (%): Specifies the beginning value for the progress bar. This is useful if you are linking multiple SSI Web surveys together and need to specify a certain beginning value. Ending Value (%): Specifies the ending value for the progress bar. This is useful if you are linking multiple SSI Web surveys together and need to specify a certain ending value. Background Color, Bar Color, Bar Background Color and Text Color These controls let you customize the look of your progress bar. Click the drop-down button to access the "color picker" to select a new color.

General Web Interviewing (CiW) Help

99

1.3.4.1.5 Browser Settings

General Settings Disable Back Button By checking this option, the ability to back up in a survey is removed. This feature requires JavaScript to be enabled in the browser. In addition not all browsers support this feature (Chrome, iPhone, etc.). Simple Browser Settings This area lets you specify that SSI Web should remove the regular toolbar and buttons used by the respondent's browser and show the survey within a "simple browser." This is useful for maximizing the window real estate for use in presenting your survey, and for removing buttons such as the "back" button from many common browsers. To see how the simple browser operates, you need to test your survey in Local Test Server or on your remote server. Preview mode doesn't support simple browser. Notes: The respondent's browser must support JavaScript to enable simple browser. If the respondent has disabled JavaScript processing, the survey will appear in normal mode. Warning: simple browser may not always work correctly on every configuration. We recommend you test simple browser using different browsers, as the presentation can vary. If you use simple browser and the respondent has enabled JavaScript processing, even if the browser still displays the toolbar including the "back" button, SSI Web prevents the users from backing up. Specify Window Size: If you choose not to use full screen window, this lets you specify the width and height of the simple browser window. Use Full Screen Window: This setting maximizes the browser window for most browsers, and prohibits the respondent from resizing the window. Because the simple browser can remove the toolbar and icons the respondent is accustomed to seeing, it can be helpful to provide an easy way for them to close the simple browser window once they've finished the survey. If you provide text in the Close Text field, it becomes a clickable hyperlink at the upper right-hand corner of the survey that closes the window. You may include a graphic instead of text. If the Close Text field is empty, no link will display for closing the window. Specify Transition Page Text: If you use the simple browser, SSI Web inserts a "transition" page to let respondents know that they are about to launch a new browser window. It also gives them instructions if the window does not appear (holding the CTRL key while clicking next, or disabling their pop-up blocking software). For internationalization, you will want to translate the Transition Page Text into the target language. Also note that there is no Preview button to view this text. You must test the survey on a web server or on Local Test Server to see the implementation.

100

SSI Web v7

1.3.4.1.6 Error Messages

The Error Messages tab on the Survey Settings dialog lets you customize the error messages shown to respondents during the course of an interview. For example, when respondents specify an invalid response, or skip a required question, error messages prompt them to remedy the error. This area is also useful for customizing the messages for foreign languages. When respondents do not complete a questionnaire page properly and click the Submit button, warning messages appear to tell them about the error and how to fix it. If you have checked Use JavaScript 1.2 within the Format Settings tab (default), the errors are displayed on the screen without submitting information to the server. If you have not checked the Use JavaScript 1.2 button (or on older browsers not supporting JavaScript 1.2), the information is submitted to the server. The Perl scripts perform the verification and the server returns the not-yet-completed page with the error message displayed within the survey page. Disabling JavaScript verification requires the Web server to process more pages and generally work harder. This can result in slower performance for your respondents.

Customizing Error Messages

When you click the Error Messages tab, the Error Message list is displayed. You can choose the color, styles, size and justification for the error messages. Default text is provided, that you can customize for the purposes of your survey. The default text provided refers respondents to a certain question number on the current page. However, you can also customize your text to refer to specific question names (Note: if you refer respondents to specific question names, you should include the question name as part of the header text of that question). These are the following error type classes: Page Error The general message that appears at the top of the page when there is an error of any sort. Missing Answers Reminds respondents to complete a question before continuing. Minimum Checks Tells respondents that the minimum number of checks required in a multiple-response select question (check box) has not been satisfied. Maximum Checks Tells respondents that the maximum number of checks permitted in a multiple-response select question (check box) has been exceeded. Out of Range (Numeric) Lets respondents know that the value they specified for a numeric question is outside the specified range. You specify the minimum and maximum range when you compose numeric questions using the Numeric Question dialog. Invalid Input (Numeric) This error message appears if respondents type an invalid character into a numeric entry box. Invalid characters are values other than numbers or the period and/or comma permitted for decimal answers. You may want to expand the error message text to explain these other conditions to respondents. Minimum Characters (Openend Questions)

General Web Interviewing (CiW) Help

101

Informs respondents that they haven't typed enough characters in an open-end question. You can specify a minimum number of characters when you compose open-end questions using the Open-end Question dialog. Maximum Characters (Openend Questions) Informs respondents that they have typed too many characters in an open-end question. You can specify a maximum number of characters when you compose open-end questions using the Open-end Question dialog. Password Closed Appears when a valid password combination is used, but the password quota cell is full (e.g. the username/password combination has already been used.) Finished Informs respondents that their survey already has been submitted. Invalid Login Lets respondents know that the password combination they've specified is invalid. Rank Informs respondents that the answers to a ranking question are not valid. Rank Count Informs respondents that they have not yet fully ranked enough items in a ranking style question to continue. None of the Above Informs respondents that they cannot select an "exclusive" answer (such as "None of the above") together with other responses for the same question. This message is only shown with Perl verification. Constant Sum Informs respondents that the responses to a constant sum question do not total the valid amount. Submit Only Once If respondents click the submit button multiple times, this message asks them to wait for a moment while their responses are being processed. This message only appears if JavaScript 1.2 is enabled for the respondent's browser. MaxDiff Unique Answers If respondents click the same response for the "best" and "worst" items in a MaxDiff question, this message is displayed. ACBC Prohibited Warns respondents that they cannot select levels in the BYO question of an ACBC survey that are prohibited. ACBC Price Adjustments Warning If additional price adjustments are specified, and the respondent changes a BYO selection that would affect the price of other selected options within the BYO, this warning is displayed.

Error Message Arguments The text of the error messages includes Functions that are used to display numeric values or text

102

SSI Web v7 strings relevant to each error in the error message. The available Functions are: ErrField ( ) Specifies the field (question) number on the most recent interview page, where the first question on the page is question number 1. ErrMin ( ) Specifies the minimum value for a numeric question, or the minimum number of characters for an open-end question. ErrMax ( ) Specifies the maximum value for a numeric question, or the maximum number of characters for an open-end question. ErrQName ( ) Returns the question name for the question on the current page causing the error. Use this argument if you prefer to refer respondents to specific question names rather than a question sequence number on the current page. (Note: if you refer respondents to specific question names, you should include the question name as part of the header text of that question). ErrText ( ) Returns the error text provided under the Advanced... button for questions, or in the error text field for Free Format questions. If no text is provided in the error text field, the ErrText variable will be replaced by nothing (be missing) when run over a server. ErrTotal ( ) Returns the total number of items to be ranked in a ranking-style question, or the total number of points to be allocated in a constant-sum question. ErrCurSum ( ) Returns the current number of points that have been allocated in a constant sum question. The actual error messages that a respondent sees when taking the questionnaire (with numeric values substituted for the string arguments above) can only be viewed when your survey is running on the Local Test Server or on a remote web server.

General Web Interviewing (CiW) Help

103

1.3.4.1.7 Server Settings

Administrative Module Access Survey administrators log into the Online Administrative Module using a browser (the login page is studynameadmin.html on the web server) to manage studies in the field and download data. A User Name and Password are required to access the Administrative Module. This area is used to specify the passwords you use to access the Admin Module. Default user names and random passwords are provided for your convenience (new random passwords are created for each new study), but you can change these if you like. User Name Lets you provide a name with up to 12 characters. Read-Only Password Lets you provide a password with up to 12 characters (passwords are case-sensitive and can only use numbers or letters). Read-only passwords convey limited administrative rights (e.g. you cannot download the data, delete or accumulate disqualified/incomplete data, edit the passwords file, or reset the survey.) Read-Modify Password Lets you provide a password with up to 12 characters (passwords are case-sensitive and can only use numbers or letters). Read-modify passwords convey full administrative rights. We encourage you to use strong passwords that include a combination of alphabetic and numeric characters, and are at least eight characters long.

Server Directory Paths Sometimes you may find that your ISP doesn't let you set up directories the way we've outlined. Or, you may simply have other preferences regarding your study's architecture on your server. You can change the names and locations of the directories on this dialog. This dialog lets you specify relative paths for the Administrative, CGI Scripts and Data directories. The five paths are as follows: Administrative Directory: CGI Scripts Directory: Data Directory: Relative Path from CGI Scripts to Graphics: Relative Path from login.html to Graphics: Path from Perl scripts to the /admin directory Path from STUDYNAMElogin.html and STUDYNAMEadmin.html to the directory that contains ciwweb.pl and admin.pl Path from Perl scripts to the /data directory Path from Perl scripts to the /graphics directory Path from STUDYNAMElogin.html to the /graphics directory

When you use the Add Image icon to add graphics to your study, your <img> tags then contain the function [%GraphicsPath( )%] in their paths. [%GraphicsPath( )%] will be replaced on the web server with the path that you provided here in the Server Settings tab of the Survey Settings dialog. For example if you set your graphics path to "../My Graphics/" and you included <img src="[%GraphicsPath( )%]picture.jpg" BORDER="0"> in your questionnaire, then on the web server

104

SSI Web v7 that path would dynamically change to <img src="../MyGraphics/picture.jpg" BORDER="0">. If you rename the directories that the files are stored in on your server or use absolute file references, put the new MyGraphics file in your CGI Scripts directory so that the Perl scripts can find your study. If you can only run Perl scripts from one directory and you want to have multiple studies running at the same time. The paths in the Server Settings tab of the Survey Settings dialog must be set correctly for each study and then each study's STUDYNAMEpath.cgi file must be placed in the cgi-bin directory. For example, if two studies "T1" and "T2" share the same Perl files, then the cgi-bin directory would need to contain the T1path.cgi and T2path.cgi files.

Customizing the Admin Module You may customize the look of the Admin Module by creating your own CSS file. In the on the Server Settings tab, enter the name of your custom CSS file. You can place this CSS file in the graphics folder or you can refer to a remote file by listing the full URL (i.e. http:// ). If the CSS file is placed in the graphics folder, you will need to include the graphics path. For example: [%GraphicsPath()%]your_style.css By including a custom CSS file we will remove all "Sawtooth Software" branding from the Admin Module. To remove our title and include your company's logo include the following CSS: #title_text { display: none; } #title { background:url('your_logo.gif') no-repeat center center; height: 50px; }

General Web Interviewing (CiW) Help

105

1.3.4.1.8 Advanced

Save Respondent Information

This section lets you disable SSI Web's automatic storage of information about the respondent's browser, operating system, and IP address. There may be some instances in which you are prohibited from collecting such information, and you can un-check the information that should not be saved. The information collected on respondent's browser, operating system, and IP address may be exported with your other survey data. Even if you decide not to collect the identifying information described here, a variable indicating whether Javascript is enabled within the respondent's browser is always collected. Save Respondent Time Per Page Check this box to store the time (in milliseconds) spent per page. When you enable this, variables are stored into the data file that store the time. You can export these times under the File | Export Data option. To convert times to seconds, divide by 1000. If the respondent backs up and re-answers a page (using SSI Web's Previous button), new time spent answering that page will be added to the previous time recorded. If respondents use the Back button on the browser to re-answer a page, the total time spent on the re-answered page may not be accounted for correctly.

Additional Settings
HTML DOCTYPE Checking Include HTML DOCTYPE adds the following line to the top of each of the web pages generated by SSI Web: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> This line declares the HTML version used on the web page. Many web browsers will read this line and make a decision how to display the HTML. The DOCTYPE declares that the HTML is following standards. You can check the HTML of the page for errors by going to http://validator.w3.org/. Using a DOCTYPE can alter the way that the web page is displayed in some browsers. Some of your custom HTML might not be formatted quite the same in some browsers with the DOCTYPE at the top as it is without it. You might consider un-checking Include HTML DOCTYPE if you are not pleased with the slight visual changes on the web page caused by the DOCTYPE. Question Verification Using Javascript 1.2 Any browser that interprets standard HTML can work to take surveys. More recent browsers support JavaScript 1.2. You can reduce the processing effort for your Web server by letting JavaScript, running locally on your respondents' computers, perform response verification (making sure required questions are answered, numeric values fall within prescribed ranges, etc.). If you specify JavaScript verification and the respondent's browser does not support or does not have JavaScript 1.2 or later enabled, SSI Web will use the Perl error checking script built into SSI Web instead. Under Perl verification, responses to the survey page are submitted to the server, the Perl scripts perform the response verification, and any error messages are sent within the HTML

106

SSI Web v7 code back to the respondent. We generally recommend you check the Use JavaScript 1.2 box. If you want to turn off JavaScript verification altogether, you can un-check it, although in most cases there is no reason why you would want JavaScript to be off.

Decimal Point Character

If you are including numeric questions in your survey and have specified that decimal answers are permitted for those questions, you need to specify the character to be reserved for the decimal point. In the US, a period (default) is used. Other countries may use a comma. If a period is selected, only numbers and a single period are acceptable answers. For example, the number "one thousand one and one half" must be specified as: "1001.5", not "1,001.5". If other characters are used, a warning message appears. If you specify that a comma or period is to be used as the decimal point character, both periods AND commas are acceptable as decimal points. In other words, the number "one thousand one and one half" can be specified as: "1001.5" or "1001,5", but not "1,001.5", or "1.001,5". (Note: the Marginals program within the Admin Module truncates all decimal answers, treating them as integers, prior to computing frequencies, min, max, and means.)

General Web Interviewing (CiW) Help 1.3.4.2

107

HTML Elements to Enhance Your Survey

(This section gives a brief overview of using HTML elements within SSI Web. More HTML help is included in the HTML Cheat-Sheet .) HTML (HyperText Markup Language) is the language of the Web. You don't need to know any HTML code to use this software. But users who know (or are willing to learn) a few simple HTML tricks can gain some nice benefits. You can include HTML instructions to: Add styles to specific words or letters (like bold, italics, or underlining) Change font sizes of specific words or letters Display graphic images Nearly any HTML instruction can be included in your survey. You simply type HTML elements into the text entry fields in the SSI Web interface. Notes: Be careful with the HTML elements you add within your surveys! You should only include HTML instructions that belong between the <BODY> </BODY> tags of the HTML script. It would be a mistake to use <HTML> or <FORM> tags in the HTML code you add to your questionnaires, as it will likely interfere with the <HTML> and <FORM> section instructions that our software automatically generates when it runs on the server. Also, you probably wouldn't want to add a hyperlink that would send respondents out of the survey, turn off the forms mode using </FORM>, create a submit button or additional input field that would interfere with the survey progress. When using the Free Format question type, you will probably need to know more than just the "basics" of HTML. The Free Format question type turns all of the question-writing responsibility over to you, including specifying the <input> tags. We have some very simple examples documented in the Free Format section. Any number of books and websites can instruct the interested user on HTML basics. Here is a partial list of HTML elements that can be useful (and safe to use) in your Internet surveys:

Adding Styles
Italics to turn on: to turn off: Type: to create: Bold to turn on: to turn off: Underline to turn on: to turn off: <i> </i> Please be <i>very</i> careful! Please be very careful! <b> </b> <u> </u>

108

SSI Web v7

Adding Hard Returns

Internet browsers typically wrap your text if it doesn't fit into the allotted space. Sometimes you may want to control where your phrases break. You can insert hard returns using the <br> element. Type: to create: Type: to create: Please be <br>very careful! Please be very careful! <p> hard return plus blank line

Inserting Horizontal Lines Type: to create: <hr> Horizontal line spanning the current text cell, or spanning the current line.

General Web Interviewing (CiW) Help 1.3.4.3

109

HTML Cheat-Sheet Text Tags

<pre></pre> Creates preformatted text <h1></h1> Creates the largest headline <h6></h6> Creates the smallest headline <b></b> Creates bold text <u></u> Creates underline text <i></i> Creates italic text <font size=?></font> Sets size of font, from 1 to 7 <font color=?></font> Sets font color, using name or hex value

Links
<a href="URL"></a> Creates a hyperlink <a href="mailto:EMAIL"></a> Creates a mailto link <a name="NAME"></a> Creates a target location within a document <a href="#NAME"></a> Links to that target location from elsewhere in the document

Formatting
<p></p> Creates a new paragraph <p align=?> Aligns a paragraph to the left, right, or center <br> Inserts a line break

110

SSI Web v7 <blockquote> </blockquote> Indents text from both sides <dl></dl> Creates a definition list <dt> Precedes each definition term <dd> Precedes each definition <ol></ol> Creates a numbered (ordered) list <li></li> Precedes each list item, and adds a number <ul></ul> Creates a bulleted (unordered) list

Graphical Elements
<img src="name"> Adds an image <img src="name" align=?> Aligns an image: left, right, center; bottom, top, middle <img src="name" border=?> Sets size of border around an image <hr> Inserts a horizontal rule <hr size=?> Sets size (height) of rule <hr width=?> Sets width of rule, in percentage or absolute value <hr noshade> Creates a rule without a shadow

Tables
<table></table> Creates a table <tr></tr> Sets off each row in a table

General Web Interviewing (CiW) Help <td></td> Sets off each cell in a row

111

Table Attributes
<table border=#> Sets width of border around table cells <table cellspacing=#> Sets amount of space between table cells <table cellpadding=#> Sets amount of space between a cell's border and its contents <table width=# or %> Sets width of table -in pixels or as a percentage of document width <tr align=?> or <td align=?> Sets alignment for cell(s) (left, center, or right) <tr valign=?> or <td valign=?> Sets vertical alignment for cell(s) (top, middle, or bottom) <td colspan=#> Sets number of columns a cell should span <td rowspan=#> Sets number of rows a cell should span (default=1) <td nowrap> Prevents the lines within a cell from being broken to fit

Forms
<select multiple name="NAME" size=?></select> Creates a scrolling menu. Size sets the number of menu items visible before you need to scroll. <option> Sets off each menu item <select name="NAME"></select> Creates a pulldown menu <textarea name="NAME" cols=40 rows=8></textarea> Creates a text box area. Columns set the width; rows set the height. <input type="checkbox" name="NAME"> Creates a check box. Text follows tag. <input type="radio" name="NAME" value="x"> Creates a radio button. Text follows tag <input type="text" name="foo" size=20>

112

SSI Web v7 Creates a one-line text area. Size sets length, in characters. <input type="submit" value="NAME"> Creates a Submit button <input type="image" border=0 name="NAME" src="name.gif"> Creates a Submit button using an image <input type="reset"> Creates a Reset button

General Web Interviewing (CiW) Help 1.3.4.4

113

Including Graphics
You can display graphics most any place within an SSI Web study. Those who know some HTML recognize that graphics can be added using the IMG tag. SSI Web automates the way that SSI Web manages graphics for testing (under Local Test Server) and for running over servers. We recommend you use SSI Web's automatic ability to manage the "path" to your graphics, and that you follow the instructions below for inserting graphics within your questionnaire. (Note: Please see the section on Graphics Management for more information regarding how SSI Web manages graphics, and how you can customize how graphics are used and stored on the server.)

Adding Graphics
To add a graphic, click the Add Image icon on the toolbar accessed by first clicking the "Pencil" edit

icon . Then, browse to the folder containing your graphic, and double-click the graphics file. When you do this, the following happens: 1. An HTML <img> tag including SSI Web scripting is inserted within your questionnaire: <img src="[%GraphicsPath( )%]yourgraphic.???" border="0"> where "yourgraphic.???" is the name of the graphic file you added, and [%GraphicsPath( )%] is the SSI Web script with tags "[% %]" and system function GraphicsPath( ) that returns the path from the cgi-bin directory to your graphics directory (or from the login.html file to your graphics, if the graphic is on the login screen), as specified under Compose | Survey Settings | Server Settings... 2. The file "yourgraphic.???" is copied into a graphics directory directly off of your study directory. You may be tempted to type the <img> tag contents yourself for a new graphic, or cut-and-paste the <img> tag contents to another part of your survey and alter the referenced graphic. If you do this, make sure to copy the new graphic into the graphics directory, or the graphic will not appear when testing your survey. For this reason, you may discover that it is easier to just use the Add Image icon. We suggest you use the system function GraphicsPath( ) for managing the paths to graphics for your survey. However, you can edit other aspects of the <img> tag, such as providing size parameters or changing the border width. Different browsers support different graphics formats. For best results across different browsers, your image should probably be saved in .GIF,.JPG., or .PNG format. Internet browsers display images in their original size; they do not automatically size them to fit within your interview or in relation to your question area. Plan ahead and size your images to look good under different screen resolutions.

114 1.3.4.5

SSI Web v7

Graphics Management
SSI Web assumes a server installation that contains a separate graphics directory. We use the function [%GraphicsPath( )%] to specify the path to that graphics directory. When the survey is run on a server, the text "[%GraphicsPath( )%]" is replaced by the paths specified under Survey Settings | Server Settings. There are two paths related to graphics: If the graphic is located on the login page, then the function [%GraphicsPath( )%] is replaced with the text supplied in the Relative Path From login.html to Graphics field. If the graphic is located anywhere other than the login page, then the function [%GraphicsPath( )%] is replaced with the text supplied in the Relative Path From CGI Scripts to Graphics field. If you use the [%GraphicsPath( )%] function, Local Test Server always looks to the graphics subdirectory off of your study directory regardless of what you have specified under Survey Settings | Server Settings. But, when running on the server, SSI Web uses the paths to graphics specified under Compose | Survey Settings | Server Settings. One benefit to using the [%GraphicsPath( )%] function is that if you need to change the path definitions for graphics, you only need to change them in one place (under Compose | Survey Settings | Server Settings). If you want to, you can choose not to use the system variable when defining the paths to graphics. You can provide an <img> tag with a relative or absolute path of your choice (where the path is the path from the Perl scripts to the directory containing the graphic, or the login.html file to the graphic). However, customizing the paths will most likely mean that the graphics will not be able to be found when testing with Preview or the Local Server.

General Web Interviewing (CiW) Help 1.3.4.6

115

Randomizing Questions/Pages/Blocks
There are times where the researcher wants to randomize a series of pages, questions on a page, or blocks of questions. This is most often done to reduce response bias due to question order. Question order bias occurs when respondents answer differently to later questions because of the influence of previously answered questions. If questions are displayed in random order, question order bias still exists at the individual level. But, because individuals each receive a different question ordering, the bias is nearly eliminated when viewing aggregate results. You can randomize the order that questions are asked within the same page. You can also randomize the order of pages. You cannot randomize the order of questions that span from one page to another, when there are multiple questions per page.

Randomizing Questions on a Page

To randomize the order that questions are asked on a page: 1) Click Compose | Write Questionnaire and click the Randomize... | Questions on a Page button. From the Randomize Questions on a Page dialog, click the Add... button. 2) Select the beginning and ending question. All questions from the beginning to the ending question (including the beginning and ending questions) are displayed in random order when the questionnaire runs on a Web server.

Randomizing the Order of Pages

1) Click Compose | Write Questionnaire and click the Randomize... | Pages button. From the Randomize Pages dialog, click the Add... button. 2) Select any question from the first and last pages in the randomized sequence. All pages including and between the anchor questions are displayed in random order when the questionnaire runs on a Web server.

Randomizing Blocks of Questions

Example: imagine a questionnaire with three blocks of questions Q1-Q3 (block 1), Q4-Q10 (block 2), and Q11-Q20 (block 3). We may want to randomize the order that those three blocks of questions occur. For example, one possible ordering is Q4-Q10, Q1-Q3, Q11-Q20. 1) Click Compose | Write Questionnaire and click the Randomize... | Blocks button. From the Randomize Blocks dialog, click the Add Set button. 2) Add each block of questions by clicking Add Block... and then choosing a beginning and ending question for each block. 3) Select to either randomize the order that the blocks are asked, or to ask the blocks in the order as indicated in a constructed list. For example, if the blocks were to be asked in the order as represented within a constructed list, and that list contained elements ordered 3, 2, 1 then the blocks would be asked in order: block 3, block 2, block1. Notes: Each block must be separated by at least one page break. Questions can be additionally

116

SSI Web v7 randomized within blocks.

Showing a Random Subset of Questions/Blocks

Example: imagine a questionnaire with 10 questions (or blocks of questions). You wish to randomize their presentation and have respondents complete a randomly selected 5 of the 10 questions. 1) Within Compose | Lists..., create a list called list10 with 10 members (you may just use integers 1 through 10 as the list member text). Add a constructed list called randlist5, that uses parent list list10. Use the following constructed list logic to randomize the list10 members, and trim the list to just 5 members: ADD (list10) RANDOMIZE () SETLISTLENGTH (5) 2) Click Compose | Write Questionnaire and click the Randomize... | Blocks button. From the Randomize Blocks dialog, click the Add Set button. 3) Add each question, or block of questions, by clicking Add Block... and then choosing a beginning and ending question for each block. (If you are selecting random subset of single questions, the beginning and ending question for each block will refer to the same one question.) 4) Select to radio button (Use Constructed List) to randomize the order that the blocks are asked according to the constructed list, randlist5. When the questionnaire runs, if the constructed list contained elements 4, 2, 8, 7, and 1 the question blocks would be asked in order: block 4, block 2, block 8, block 7, and block 1. Notes: Each block must be separated by at least one page break.

General Web Interviewing (CiW) Help

117

1.3.5

Skip Logic
Skip patterns (also known as branching) let respondents skip over sections of the questionnaire that don't apply, typically based on previous answers. SSI Web provides flexible and powerful skip logic. Skip instructions can either be executed before showing a question, or after the question is answered. We refer to these classifications as pre-skips and post-skips: Pre-Skips - The skip instruction is evaluated and, if true, executed just prior to loading a question. The typical use is to conditionally skip a question if a condition is true. Otherwise, the question is displayed. Post-Skips - The skip instruction is evaluated and, if true, executed after a respondent has answered the question and submitted the data on the page containing this question. You have the option of providing a target question to skip to, or to skip to the "next question" (in the case of pre-skips). A comment field is associated with each skip logic entry you provide, so you can make any notations to help you as you construct and test the questionnaire. Skip patterns only operate when the survey runs on a web server (not during preview mode).

Skip Settings
Skip from Specify the question to skip from using this drop-down box. Skip Type This control is for specifying whether this is a Pre-Skip (skip is executed Before the question is shown) or Post-Skip (skip is executed After the question is answered). Always skip Indicates an unconditional skip. The skip is always executed, and no skip logic is necessary. Skip if the following logic evaluates to true Check this button if the skip is only to be executed if certain skip logic is true (provided in the text field directly below). Skip to Specify the question to skip to using this drop-down box. Comment You can optionally provide notes within this field to assist you in composing and testing your survey.

Skip Logic Syntax

SSI Web provides powerful skip logic syntax for executing skip patterns. You type this logic into the text field directly below the Skip if the following logic evaluates to true radio button. Supported Logical Operators: and or not

118

SSI Web v7 Supported Mathematical Operators: =, <, >, <=, >=, <> "not equal to" +, / "division" * "multiplication" ^ "raise to the power of" mod "modulo"

and all other SSI Web Functions. You can check whether the instructions (syntax) you provide are valid by clicking checked when you prepare your survey for local or remote web server. (Note: skip logic has been implemented using SSI Script.) . Syntax is also

Example Syntax
Below are some examples of valid skip syntax. If these statements evaluate to true, then the skip is executed. Q1 = 23 Q1_3 = 1
Q1 <= 10

Q1 <> 99

"Skip if Q1 is equal to 23" "Skip if the third response option is checked in multi-response question Q1" "Skip if Q1 is less than or equal to 10" "Skip if Q1 is not equal to 99"

Age > 17 (Age >= 18) and (Gender = 1) (Age >= 18) or (Q1 = 23) or (Q2 > 3.534)

((ageQuestion > 17) Or (Not Shown(ageQuestion))) "Shown" is a function that returns true if the question specified in the parenthesis has been seen
Q23/(Q24+Q25) >= 23.5

(Q23*Q1)^0.5 < 10

"^" means "raised to the power of"

Q1OpenEnd="hello world"

Shown(Q1) "Skip if the respondent saw Q1, even if the respondent left Q1 blank" ListHasParentMember(Q1List, 4) "Skip if Q1List contains list member #4 from its parent list" If you change the name of a question or list in your questionnaire, the question or list name is NOT automatically updated in your skip logic. You must do this manually.

Skip Logic Flow of Execution

Pre-skips are evaluated before a page loads. If the questions on the page are randomized, any skip patterns associated with these questions are evaluated in their unrandomized order. If a question is skipped, any post skip logic associated with the skipped question is ignored. (Note that questions can be skipped due to skip logic, or also due to containing an empty constructed list.) Post-skips are evaluated after the page is submitted, according to the unrandomized order of questions on the page. The first post-skip logic that evaluates to true is executed, and the rest are ignored. Warning: a pre-skip that has a destination to a different page may cause other questions on the same

General Web Interviewing (CiW) Help

119

page not to be displayed. For example, consider a page containing questions Q10, Q11 and Q12. If a pre-skip on Q12 skips to a later page, if the pre-skip logic associated with Q12 is true, it causes the entire page to be skipped. In this case, one would want to place Q12 on its own page to ensure that Q10 and Q11 are displayed. Hint: You can create HTML/Filler questions that contain skip instructions (and perhaps no actual text to be shown on the screen). This can help in some instances when you want to place skip instructions on a page, but do not want the skip instructions to be directly associated with a specific question (in case that question was conditionally skipped).

Performing Skips Based on "Select All That Apply" (Multi-Select) Questions

You can perform skip patterns based on "select all that apply" questions. Assume a multiple-response question named Q1 that has 5 options. Respondents can register up to 5 answers. If item number three on the list was selected, then that item is referenced using "underscore notation": Q1_3 = 1. If item three was not selected, then Q1_3 = 0. If item number 1 was selected, then Q1_1 = 1, etc. (Note for Ci3 users: the "underscore" notation here acts quite differently from "dot notation" for multiple selects as is used in Ci3. With our web-based interviewing system, you cannot detect which item in a multiple-select was selected first, second, etc. as you can with Ci3.) Constant Sum and Ranking questions also employ underscore notation. For example, in a ranking question named Q10, if the third item is ranked 2nd, then Q10_3 = 2. Grid-based questions can involve "double-underscore" notation, to indicate both row and column (for example Q10_r1_c2). (Hint: if you are uncertain regarding how a variable involving underscore notation is named, you can preview the associated question and view its source HTML to confirm the variable's name.) Randomizing items within a multiple-response list does not affect the operation of the system variable Question_#. The item number after the underscore always refers to that item's position within the original, unrandomized list.

Unconditional Skips
Sometimes you want to perform a skip that has no conditions (it is always executed). The section below "Randomly Displaying a Subset of a Block of Questions" is an example that uses an unconditional skip. Unconditional skips can also be useful, for example, if displaying a follow-up message when the response is somehow unacceptable. Consider a numeric question (Age) that is used to indicate the respondent's age. You can create an Explanatory Text question (no response required) on the next page directly following that numeric question. That question might be named LessThan18 and the text might state: "You must be 18 to qualify for this survey, please have the individual in your household who is 18 or older and whose birthday is closest to today continue this survey." If the numeric value for Age is greater than 17, we want to skip to the next question in the survey (Q1). But, if the value is 17 or less, LessThan18 is displayed with the text asking for an individual 18 or older. An unconditional skip is created from LessThan18 back to Age, as follows: Skip from: Skip Type: Always skip Skip to: Comment: LessThan18 Post-Skip (After the question is answered) (checked) Age Always skip back to Age from this question.

120

SSI Web v7 The survey cannot continue until a valid age is entered. (We aren't advocating this approach from a survey research methodology standpoint, but have used this example as an illustration of unconditional skips.)

Skip Example: Randomly Displaying One out of n Pages

Let's assume you have ten pages of questions, but you want respondents to randomly see just one page. You could place an unconditional skip after the last question on each page, that skips to the question directly following the ten pages of questions. Then, simply randomize the pages. When the questionnaire runs, a random page appears, and the skip is executed (skipping out of the series of 10 questions) when the page is submitted.

Skip Example: Displaying a Random Subset of Questions on the Same Page

This is a somewhat advanced example that illustrates additional capabilities of SSI Web. This example assumes you have a working knowledge of constructed lists. Consider a situation in which there were 20 questions, but you wanted respondents to see just five of these questions, selected randomly. This could be done as follows: 1. 2. Create a predefined list called (for illustration) Integers using Compose | Lists... (the list has 20 items, integers 1 through 20, or letters a through t, etc.). Under Compose | Lists, create a constructed list called NewIntegers with parent list Integers. Add the Integers list to NewIntegers, and then randomize the elements using the Randomize instruction. Drop all but the first five list elements using the ListMax instruction. Place a pre-skip on each of the 20 questions, which skips the question if:

3.

ListValue(NewIntegers,1)<>X And ListValue(NewIntegers,2)<>X And ListValue(NewIntegers,3)<>X And ListValue(NewIntegers,4)<>X And ListValue(NewIntegers,5)<>X

Where "X" is 1 for the first question, 2 for the second question, etc. Additionally, if you place all 20 questions on separate pages, you can randomize the order of the pages by clicking Randomize... from the Write Questionnaire dialog.

Prohibited "Skip Backs"

You cannot skip to any question on the first page.

General Web Interviewing (CiW) Help

121

Notes:
Questions not answered by a respondent evaluate to zero or "" (empty string). Therefore, a skip based on Q5A < 5 will be true if Q5A was not seen by this respondent (Not Shown( )) or was seen but not answered (Not Answered( )). If no questions are shown on a page due to pre-skips, the entire page is skipped as well. You can conduct skip patterns based on JavaScript status. If JavaScript( ) evaluates to true, then the respondent has JavaScript enabled. If JavaScript( ) evaluates to false, then JavaScript is disabled. For respondents that do not have JavaScript enabled, you may want to skip them to a page that says "To enable JavaScript in Microsoft Internet Explorer, go to Tools, Internet Options, Security, Custom Level. Under Scripting, Active Scripting, choose Enable. To enable JavaScript in Netscape Navigator, go to Edit, Preferences, Advanced, and then check Enable JavaScript." To do that, you would skip if not JavaScript( ).

122

SSI Web v7

1.3.6

List Building
Introduction
List Functions Reference Market research questionnaires almost always include lists of brands, statements, response options (scale points), etc. Sometimes, lists of brands or response options need to be used over and over again within the same questionnaire. It is useful if the list is established and simply reused when writing subsequent questions. SSI Web saves all lists of response options for your questionnaire, and lets you select from this list when creating new questions. Some questionnaires require more sophisticated use of lists. For example, one might display a list of brands and ask respondents which brands they have used recently. Then, only those brands selected are used in a subsequent question, such as for indicating which of those brands being used is the "favorite." (Some researchers refer to this as piping.) Such lists need to be customized ("constructed") for each respondent. SSI Web offers these capabilities. There are two general classifications of lists in SSI Web: 1) Predefined List - a list that always remains the same for all respondents 2) Constructed List - a list built individually for each respondent (according to instructions you specify), depending on answers to previous questions or randomization. Whenever you create a list of response options for a select-type, ranking, constant sum, or grid-style question, you are creating a list that you can reuse in a later question. For example, if you create a select-type question Q1: Q1: What is your gender? Male Female Refused the responses are automatically stored as a predefined list named Q1List that includes the following list members: Male Female Refused This list, once established, can be used in other questions. Note that if you change the list, all questions using that list are affected. For straightforward questionnaires, thinking about lists may be unnecessary. But, for complex questions, the ability to build constructed lists is indispensable. We've described how predefined lists are automatically stored whenever you create a select-type question. There is also a general Lists dialog where you can add or edit predefined and constructed lists. You access this list manager by clicking Compose | Lists from the main menu. You must use the Lists dialog to create constructed lists. Time-Saving Hint: You can cut-and-paste lists of response options from Microsoft Word or Excel by highlighting the array of items in Word or Excel, pasting to the clipboard using Ctrl+C or the Copy button in Word or Excel, and then (within SSI Web) by clicking the Paste from word processor or

General Web Interviewing (CiW) Help

123

spreadsheet icon

In our documentation, we'll use the following terminology to refer to aspects of lists:

How Constructed Lists Work

Constructed lists are customized lists of brands, statements, response options, etc. that are built "onthe-fly" based on the answers that a respondent gives (and that were submitted on a previous page) or that are built based on randomization or other user-defined instructions. They often include a subset of the original members in a list. For example, consider a question that asks respondents which colors they like. The researcher might first create a predefined (parent) list that contains the colors: Red Green Blue Purple Yellow Next, the researcher might create a question called Q1 using that predefined list of colors. A respondent might answer that question in this manner: Q1. Which colors do you like? Red Green Blue Purple Yellow

This is a multiple response select question, and the data are stored internally within SSI Web as follows: Q1_1, 0 Q1_2, 1 Q1_3, 0 Q1_4, 1 Q1_5, 0 It is useful to review this so you can understand how the system stores data and how to refer to answers from previous questions when using skip patterns or displaying the responses to previous questions on the screen. With multiple-select questions, any checked options are indicated with a "1" and non-checked options with a "0". "Underscore" notation is used to refer to the list members. For

124

SSI Web v7 example, Q1_4 refers to the fourth member in the list (Purple). Even if you decided to randomize the list, "_4" always refers to the fourth member on the original predefined list. SSI Web always uses indices that refer to the original list order so that no matter what randomization or sub-selection is employed, you can easily keep track of which members have been seen or answered. With SSI Web's list-building capabilities, you can create a new constructed list containing the colors chosen in Q1. (Examples and instructions for this are given later in this section.) The constructed list for this respondent, would contain the members this respondent chose in Q1: Green Purple In Q2 (placed on a page following Q1), we might use this list when asking this respondent which of the colors previously selected is her favorite: Q2. Which of the colors you previously said you liked is your favorite? Green Purple

If this respondent answers that Purple is her favorite color, the data are stored internally within SSI Web as follows: Q2, 4 Note that the response to Q2 is coded as a "4," because Purple is member #4 on the original parent list of colors.

Displaying the Text for List Members in Questions

Most of the time, you will just show an entire predefined or constructed list in a question. To display the list as response options to a question, you'll simply select it from the drop-down control of available lists while editing the question. Occasionally, you may want to display the label for a particular list member on the screen. The following SSI Web script returns the label for the list member: [%ListLabel (ListName, member#)%] When the survey runs, SSI Web replaces the entire expressions above with the requested label.

Skip Patterns for Questions That Use Constructed Lists

Constructed lists must be built using list members from only a single predefined ("parent") list. When referring to responses to a question involving a constructed list, we just refer to the list values on the original parent list. For example, consider the following question called Vehicles: <Question: Vehicles>

General Web Interviewing (CiW) Help Which of the following vehicles would you consider buying? that uses a predefined list called VehiclesList: Honda Accord Mazda 626 Saturn SL2 Toyota Corolla Ford Focus

125

We might build a new constructed list called ConsideredList that contained only those models that the respondent chose in question Vehicles. Then, we might ask a new question called MostLikely: <Question: MostLikely> Of those vehicles you are considering buying, which do you think you are most likely to purchase? <Display ConsideredList, containing only those vehicles chosen in question Vehicles.> Let's assume we needed to execute a skip if the respondent chose Saturn SL2 as the most likely vehicle in question MostLikely. You do not know how many vehicles are in ConsideredList for each respondent or what position Saturn SL2 might occupy on that list. But this is no matter, as you simply need to refer to the original value from the parent predefined list Vehicles. Saturn SL2 is associated with a value of 3 on the parent list. Therefore, we would skip if MostLikely = 3. With multi-response select questions that use constructed lists, you may want to skip based on whether a particular list member was chosen. Again, you refer to values on the original predefined parent list. If we wanted to skip if a member with a value of 12 from the original parent list was chosen in multi-select question Q23, we would skip if Q23_12 = 1. Since constructed lists draw members from a single predefined parent list, there is no ambiguity.

Summary of List-Building Instructions

Below, we first provide a summary of commonly-used list-building instructions, followed by a more detailed description of these commands. Following that, we provide some questionnaire-programming examples. Constructed lists must be built using list members from only a single predefined ("parent") list. ADD Adds members from a specified list to the end of the constructed list. AIC ("Add If Chosen"), ANC ("Add if Not Chosen") Adds members chosen (AIC) or not chosen (ANC) in a select-type question onto a new constructed list. AIE ("Add If Equal"), ANE ("Add if Not Equal"), AIL ("Add If Less"), AIG ("Add if Greater") Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers equal to (AIE), not equal to (ANE), less than (AIL) or greater than (AIG) a specific value. INSERT Inserts members from a specified list to an exact position within a constructed list. LISTMAX , LISTMIN, SETLISTLENGTH Controls the maximum or minimum length of a list.

126

SSI Web v7

RANDOMIZE Randomizes the members of a constructed list. REMOVE Removes the members in the specified list from the constructed list. REVERSE Reverses the order of the members in the specified list. SORTBYVALUE Sorts members of a constructed list into their natural order (as in the pre-defined list)

Rules Governing List Operations

1. All constructed lists are initially empty. 2. All members added to a constructed list must come from a common parent list. 3. You may add members from another constructed list to a constructed list, as long as both constructed lists share the same parent predefined list. 4. A member can only appear once in the constructed list (if you try to add a member after it already exists on a constructed list, it will not be added again). 5. List building commands are executed sequentially. 6. A question using an empty constructed list (as response options) is skipped. When you view questions involving constructed lists in Preview mode, all members from the parent list are displayed (since we don't know how the respondent has answered previous questions).

List Instructions
In this section, we describe each list building instruction in more detail. These instructions build or modify constructed lists. Remember, a key rule for building lists is that the members must originate from the same predefined list.

ADD ADD is used to add a specific member or members from a list (either predefined or constructed) to the constructed list. The formats are: ADD (ListName) ADD (ListName, Member#) ADD (ListName, StartMember#, EndMember#) The referenced list can be either a predefined or constructed list. The member numbers refer to the current member positions on the referenced list. (If you are adding members from a constructed list, the member numbers refer to the current member positions on that constructed list, not the members' original positions in the predefined list.) StartMember# and EndMember# are optional and are used to designate a range of list members to add to the constructed list. If these two arguments are omitted, all list members are added to the constructed list. If only one argument is supplied, then only that one list member is added to the constructed list.

General Web Interviewing (CiW) Help

127

If the current constructed list already contains the list member to be added, then the duplicate list member is not added to the list. The original member remains in its position. See Examples B and E at the end of this section for illustration.

AIC and ANC Adds members chosen (AIC) or not chosen (ANC) in a select-type or grid question onto the constructed list. AIC stands for "Add If Chosen" and ANC stands for "Add if Not Chosen." The formats are: AIC (QuestionName) AIC (QuestionName, Member#) AIC (QuestionName, StartMember#, EndMember#) ANC (QuestionName) ANC (QuestionName, Member#) ANC (QuestionName, StartMember#, EndMember#) The question named must be a select-type question. If from a grid-style question, appropriate column numbers or row numbers must be included in the question label, such as Q1_c1 (for column-based grid question Q1) or Q1_r1 (for row-based grid question Q1). If from a MaxDiff question, the question label follows these conventions: ExerciseName_TaskNumber_b (for "best" response) or ExerciseName_TaskNumber_w (for "worst" response). The last two arguments are optional and are used to designate a range of list members to add to the constructed list. If these two arguments are omitted, all qualifying list members are added to the constructed list. If only one argument is supplied, then only that one list member (if chosen or not chosen in questionname) is added to the constructed list. The member numbers in the arguments always refer to the original predefined parent list (the parent list indices), not the particular position number that member happens to occupy in the list used in the referenced questionname. Members are added to the constructed list in the order they were displayed to the respondent (unless using question-level randomization with pre-defined lists). If the current constructed list already contains the list member to be added, then the duplicate list member is not added to the list. The original member remains in its position. If the member to be added to a constructed list is an Other Specify response, the text supplied by the respondent is added. This is an automatic and helpful feature. Technical Note: if there is just one Other Specify response in a predefined list, but two or more questions (using the same predefined list) collect multiple answers for this same Other Specify member, there is a potential conflict if you try to add the Other Specify members across multiple questions onto a constructed list. As functions are executed sequentially, the other response referred to by the last function is used. Using ADD or INSERT always forces the open-end box to appear with Other Specify responses. See Examples A, D, and E at the end of this section for illustration.

AIE, ANE, AIL, AIG Adds members from a question that were given answers equal to (AIE), not equal to (ANE), less than

128

SSI Web v7 (AIL) or greater than (AIG) a specific value. The formats are: AIE ANE AIL AIG (QuestionName, (QuestionName, (QuestionName, (QuestionName, Value) Value) Value) Value)

Where QuestionName can be: constant-sum (including within grids) ranking (including within grids) grid-style radio (where all rows or columns are radio) grid-style numeric If using a grid-style question that only uses radio buttons, the questionname is referred to as the original grid name, without any "underscore" notation to indicate row or column numbers. If it is a rowbased question, all rows members meeting the qualification are added to the constructed list. If column-based, members in columns are added to the constructed list. Sometimes, a grid question might use numeric entry fields. When numeric entries are part of a gridstyle question (including ranking and constant-sum questions within grid questions), the row or column must be referenced in the AIE, ANE, AIL, or AIG instruction using underscore notation. For example: Q1. How many of these types of fruits do you typically eat at work and at home during an entire week? Home Apples Bananas Peaches _____ _____ _____ Work _____ _____ _____

Let's assume we needed to create a new list containing the fruits that an individual eats at least 3 times or more at work. We'd use the instruction: AIG (Q1_c2, 2) Since this is a column-based grid question, row members are added to the constructed list. Members are added to the constructed list in the order they were displayed to the respondent (unless using question-level randomization with pre-defined lists). If the current constructed list already contains the list member to be added, then the duplicate list member is not added to the list. The original member remains in its position. Please note that when referring to numeric answers (grid numerics, constant sums, ranking, grid radios), if no answer is supplied for one of the list members (if an entry is left empty), the missing response is treated the same as zero for AIL, AIE, AIG, and ANE instructions. For example, for constant sum question "Q1", if respondents do not supply an answer for a list member (skip the entry box), that member would be forced onto a list if using the AIE (Q1, 0) instruction. As another illustration, assume you collect a series of non-required numeric responses for list members, and you want to force all members onto a new list that are rated less than 5. You might not want to use AIL (QuestionName, 5) in this case, as any numeric responses left blank evaluate to 0, and these items

General Web Interviewing (CiW) Help would also be added to the new list. See Example C at the end of this section for illustration.

129

INSERT Inserts members from a specified list (either predefined or constructed) to an exact position within a constructed list. If a member to be inserted already exists on the constructed list, it is first deleted from the constructed list. Then, it is added at the indicated position. The formats are: INSERT (TargetPosition#, ListName) Adds a list (either predefined or constructed) to the constructed list starting in the target position number indicated. INSERT (TargetPosition#, ListName, Member#) The member from the referenced list (either predefined or constructed) is inserted into a specific position on the constructed list. For example, consider inserting the color "Red" into the constructed list [Purple, Green] at the second position. The result is [Purple, Red, Green]. INSERT (TargetPosition#, ListName, StartMember#, EndMember#) Here, a range of elements are moved from the referenced list (either predefined or constructed) to a target position in the constructed list.

LISTMAX , LISTMIN, SETLISTLENGTH Controls the maximum or minimum length of a list. The format is: LISTMAX (Value) LISTMIN (Value) SETLISTLENGTH (Value) The LISTMAX instruction is useful if you want a list to contain no more than a certain number of members. If a list has more members than is specified in the LISTMAX instruction, the extra members are removed from the bottom of the list. For example, assume a predefined list with 50 European cities, of which we want respondents to rate exactly 10 of the cities chosen randomly. We first would ADD all fifty cities onto the new constructed list. Next, we would RANDOMIZE( ) the list of cities. Finally, we would use the LISTMAX instruction: LISTMAX (10) to remove the bottom 40 members from the constructed list, leaving the first 10 for respondents to rate. The LISTMIN instruction ensures that a minimum number of elements exist on a constructed list by adding members (if needed) chosen randomly from the parent list.

130

SSI Web v7 For example, assume that we'll ask respondents to rate 10 cities from a total list of 50 cities. Further assume that we'll want to ask respondents to rate cities with which they are familiar. We might first ask respondents to check such familiar cities. If they check fewer than 10, we want to randomly add additional cities to the list to be rated. To accomplish this, we use the AIC instruction to add any cities to the list that were chosen in a selecttype familiarity question. Finally, we would use the LISTMIN instruction: LISTMIN (10) to add new members (if needed) onto the list so that there were at least 10 members on the constructed list. If any members are added, they are chosen randomly from the parent list among members not already existing on the constructed list that are not "None of the above" or "Other specify" items (SSI Web never permits duplicate members on a constructed list). You can set a list length to a certain value by using the LISTMAX and LISTMIN commands together (with the same fixed integer as the argument). However, a simpler way to accomplish this is to use the SETLISTLENGTH instruction. Elements that need to be deleted from or added to the list are treated in the same way as with LISTMAX and LISTMIN instructions (if list members need to be randomly added they will not be chosen if they are "None of the above" or "Other Specify"). See Example D at the end of this section for illustration.

RANDOMIZE Randomizes the members of a constructed list. The formats are: RANDOMIZE RANDOMIZE RANDOMIZE RANDOMIZE ( ) (SeedValue#) (StartMember#, EndMember#) (SeedValue#, StartMember#, EndMember#)

Many times, you'll want list members to be presented randomly to avoid order bias. Specifying RANDOMIZE ( ) randomizes all elements on the constructed list. Specifying a starting and ending member randomizes a subset of the list. (Remember, a constructed list initially is empty. Prior to randomizing a list of members, they must first be added to the constructed list.) If you do not provide a seed value, then the list name is used as a seed. The seed value and the internal respondent number are used to ensure that the randomization will be different across lists and respondents. The seed value can range from (1 to 99999). Using a particular seed value ensures that you can obtain the same randomization across different lists (containing the same number of members) for the same respondent. (Warning: if you use multiple RANDOMIZE instructions when building a constructed list, and if you are explicitly specifying seed values, take care to use different seed values. Using the same seed value in this case will likely result in poor randomization of the constructed list.) There is a more simple way to randomize members of a list: by clicking the Randomize Response Options box on the Response Options tab when editing a select-type question. This method does not involve constructed lists, but it also doesn't let you store and later analyze the randomized order of the members (as is done when you randomize constructed lists). Also, when you re-use a predefined list of members and click the Randomize Response Options box, you obtain a different random sequence. Sometimes, you may want to re-use a list in a survey with the same randomized order for each respondent. Using constructed lists and the RANDOMIZE instruction can accomplish that.

General Web Interviewing (CiW) Help

131

Once RANDOMIZE has been used to randomize elements of a constructed list, other list-building instructions can be used to add or delete members from this new list. See Examples B, C, D, and E at the end of this section for illustration.

REMOVE Removes the members in the specified list from the constructed list. The formats are: REMOVE (ListName) REMOVE (ListName, Member#) REMOVE (ListName, StartMember#, EndMember#) The REMOVE instruction removes or deletes a member or members in the specified list (either predefined or constructed) from the current constructed list. If the members to be removed are not in the list, then nothing happens. A common use for REMOVE involves removing a single member (such as a "None of the above" response from a constructed list. Consider a predefined list named hotels, whose 20th member is "None of the above." Assume you wanted to remove the "None of the above" response from the new constructed list we are building (whose parent list is hotels). You would use the following instruction: REMOVE (hotels, 20) You can also use the REMOVE command to "subtract" a series of members of one list from another. The list to be subtracted can be a predefined list or a constructed list. For example, if you are constructing a list and want to subtract elements 5 through 10 of another constructed list named Q17Sel from your constructed list, you would specify: REMOVE (Q17Sel, 5, 10) If the REMOVE instruction refers to a constructed list, the parent list for both that referenced list and the constructed list you are building must be the same. See Examples A and E at the end of this section for illustration.

Example A: Asking a follow-up question using brands selected previously

Learning Objectives: a) Illustrates the following functions: - AIC "Add If Chosen" - REMOVE b) Demonstrates how to take brands previously selected forward to a follow-up select-type question. c) Show how respondent-supplied text for Other Specifies can be used in follow-up questions d) Illustrates how follow-up questions using an "empty" constructed list are automatically

132

SSI Web v7 skipped. The Questionnaire: <Question: LastSix> Which of the following fast-food restaurants have you visited in the last 6 months? <Randomize order for options 1 through 9. "None of the above" is exclusive.> 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. A&W Arby's Burger King Dairy Queen Hardee's McDonald's Subway Taco Bell Wendy's Other Specify (1) ______________________ Other Specify (2) ______________________ None of the above

<Question: MostOften> Of those you have visited in the last 6 months, which one do you visit most often? <Show restaurants chosen in previous question. Include Other Specify responses.> Solution: For this illustration, we assume you already know the basics of building questions in CiW. Therefore, we only provide details regarding the more advanced functions of list building. 1. LastSix uses a list of twelve response members (fast-food restaurants) that you must specify as a predefined list. Specify this predefined list by adding LastSix to your questionnaire as a selecttype question (check-box select format), and while editing LastSix using the Response Options tab, or you can directly add the predefined list under Compose | Lists (the List Manager dialog). In either case, you are establishing a predefined list of members that we can use for LastSix and elsewhere in the questionnaire. We'll use this predefined list as a source "parent list" for building the constructed (customized) list of fast-food restaurants we'll use for question MostOften. Call the predefined list LastSixList (which is the default name for the list if adding it from the Response Options tab for LastSix). CiW is not case sensitive with respect to list names. 2. After specifying the 12 list members in LastSixList, highlight member 10 ("Other Specify (1)") and check the available box to indicate that it is an "other specify" response. Repeat for the additional Other Specify list member 11. For list member 12, highlight it and check the box to indicate that it is "Exclusive" (meaning that if respondents check this category, it is the only check allowed). We also need to randomize members 1 through 9, and we can do this by clicking Randomize Response Options from the Response Options tab. (Note: the "None of the above" and "Other specify" members automatically will not be randomized if they are the last members on the list.) Add a page break after LastSix. Remember, MostOften must be displayed on a page after LastSix, since the respondent needs to submit the answers from LastSix before we can know how to customize the list of members for MostOften. 3. The list of members to be displayed in MostOften must be customized (constructed) for each respondent, depending on the answers given in question LastSix. To accomplish this, we create a new constructed list. From the main menu, click Compose | Lists to access the Lists dialog. Click

General Web Interviewing (CiW) Help

133

Add, and then you should select Constructed List. Name the new list ChosenRestaurantsList. The key rule to remember when constructing new lists is that all the response options (list members) moved into this new list must originate from the same predefined (parent) list. For this questionnaire, all the response options you'll use to build ChosenRestaurantsList come from LastSixList. Using the drop-down Parent List control, select the parent list LastSixList. To begin with, the constructed list is empty even though you have specified that you'll be using members from LastSixList. Type the list building instructions that create the new ChosenRestaurantsList: AIC (LastSix) REMOVE (LastSixList, 12) Any instructions you use are processed one-at-a-time, in a linear fashion, started from the first instruction and working through the last instruction. You can check your syntax by clicking the "Check" icon.

The first instruction tells SSI Web to "Add If Chosen" any members chosen in the question LastSix onto the new constructed list. The second instruction tells SSI Web to remove the 12th member (None of the above) of the predefined list LastSixList from the new constructed list. This remove step is necessary because we don't want to display the MostOften question with the member "None of the above" to respondents who didn't visit any restaurants in the last six months. SSI Web automatically skips a question that uses an empty constructed list, so respondents who did not visit any restaurant are not asked the follow-up question MostOften. No additional skip logic is required. (Alternatively, we could have used the single instruction AIC (LastSix, 1, 11) to indicate that only members 1 through 11 should be moved onto the new constructed list.) 4. Add the question MostOften as a select-type (radio) question. From the Response Options tab, select that this question should use as response options the existing list ChosenRestaurantsList.

Example B: Asking a question for each member of a randomized list, using a grid-style question
Learning Objectives: a) Illustrates the following functions: - ADD - RANDOMIZE b) Demonstrates how to randomize a list of members to be shown in a question. c) Demonstrates how to use a constructed list in a grid-style question. The Questionnaire: <Show grid, where rating scale is in the columns, and statements are in the rows. Randomize order of statements.>

134

SSI Web v7

Solution: For this illustration, we assume you already know the basics of building questions in CiW. Therefore, we only provide details regarding the more advanced functions of list building. This example does not need to employ constructed lists at all, since you can simply request that the members be randomized within grid questions. However, to illustrate constructed lists, we provide this as a simple example. Using constructed lists in this case does provide an advantage over simply requesting that the rows be randomized in a grid-type question. If you create a constructed list with the randomized members, information describing the order of the members for respondents is saved and can be analyzed. Also, if you wanted to show this same randomized list in another part of the questionnaire (preserving the randomized order within respondents), you could do so when using constructed lists. 1. The question in this example uses a list of six statements to describe a commercial respondents have just seen. You'll need to specify these six members as a predefined (parent) list. You can either specify the list while writing the question, or from the main menu by clicking Compose | Lists (accessing the List Manager dialog). In either case, you are establishing a predefined list of members that you can use in this question and elsewhere in the questionnaire. For this example, we suggest you use the list manager (Compose | Lists, then select Predefined List) to build the predefined list named StatementsList containing the six statements. (CiW is not case sensitive with respect to list names.) 2. Next, we need to create a new constructed list that includes all the elements from our predefined list StatementsList, but in randomized order. From the main menu, click Compose | Lists to access the Lists dialog. Click Add, and then select Constructed List. Name the new list RandStatementsList. The key rule to remember when constructing new lists is that all the response options (list members) moved into this new list must originate from the same predefined (parent) list. For this questionnaire, all the response options you'll use to build RandStatementsList come from StatementsList. Using the drop-down Parent List control, select the parent list StatementsList. To begin with, the constructed list is empty even though you have specified that you'll be using members from StatementsList. Type the list building instructions that create the new constructed list: ADD (StatementsList)

General Web Interviewing (CiW) Help RANDOMIZE ()

135

These instructions add all members from the predefined list StatementsList onto the current constructed list (RandStatementsList). Then, all members are randomized. 3. Finally, you need to add the grid-style question to the questionnaire, which uses row-based question direction, with all radio buttons. While editing the question, on the Rows tab, select that you want to use the constructed list RandStatementsList.

Example C: Adding brands to a list that were rated highly in a grid question
Learning Objectives: a) Illustrates the following functions: - AIG ("Add If Greater") - RANDOMIZE The Questionnaire: <Question: FamRx>

<Question: Effective>

Solution: For this illustration, we assume you already know the basics of building questions in CiW. Therefore,

136

SSI Web v7 we only provide details regarding the more advanced functions of list building. 1. The question uses a list of five drugs. Specify this predefined list by adding this question (call it FamRx) to your questionnaire as a grid-type question (row-based question direction), and while editing FamRx and within the Rows tab, adding the list members. Alternatively, you can directly add the predefined list under Compose | Lists (the List Manager dialog). In either case, you are establishing a predefined list of members that we can use for FamRx and elsewhere in the questionnaire. We'll use this predefined list as a source "parent list" for building the constructed (customized) list of familiar drugs. Call the predefined list FamRxRowList (which is the default name for the list if adding members from the Rows tab for FamRx). CiW is not case sensitive with respect to list names. Add a page break after question FamRx. 2. The list of members to be displayed in Effective must be customized (constructed) for each respondent, depending on the answers given in question FamRx. To accomplish this, we create a new constructed list. From the main menu, click Compose | Lists to access the Lists dialog. Click Add, and then select Constructed List. Name the new list FamiliarDrugsList. The key rule to remember when constructing new lists is that all the response options (list members) moved into this new list must originate from the same predefined (parent) list. For this questionnaire, all the response options you'll use to build FamiliarDrugsList come from FamRxRowList. Using the dropdown Parent List control, select the parent list FamRxRowList. To begin with, the constructed list is empty even though you have specified that you'll be using members from FamRxRowList. Type the list building instructions that create the new FamiliarDrugsList: AIG (FamRx, 2) RANDOMIZE () These instructions add any members from question FamRx rated greater than a 2 to the constructed list, and then randomize their order. (Note: Remember that SSI Web codes the values for grid questions sequentially from 1 on the leftmost column to 4 in the fourth column. If you had reversed your labels to display a 4 on the left and a 1 on the far right, SSI Web would not pay attention to your labeling and would still consider the ratings as running from 1 to 4 from left to right.) 3. Finally, add a new grid question after the page break named Effective (row-based question direction) with radio buttons, and specify that it uses the new constructed list FamiliarDrugsList from the Rows tab.

Example D: Asking follow-up grid questions about brands chosen in a previous question
Learning Objectives: a) Illustrates the following functions: - AIC ("Add If Chosen") - LISTMAX - RANDOMIZE b) Demonstrates how to include a subset of brands selected in follow-up grid-type questions.

General Web Interviewing (CiW) Help

137

c) Demonstrates how to limit the number of brands used in subsequent questions to no more than a maximum number. Shows how to force the client's brand onto the list, if respondents have visited the client's restaurant. The Questionnaire: Q10. Which of the following fast-food restaurants have you visited in the last 6 months? <Randomize order for options 1 through 9. "None of the above" is exclusive.> A&W Arby's Burger King Dairy Queen Hardee's McDonald's Subway Taco Bell Wendy's Other Specify (1) ______________________ Other Specify (2) ______________________ None of the above Next, we'd like to ask you to evaluate the restaurants you have visited in the last six months. Q11a. How would you rate the restaurants you have visited in terms of Cleanliness? 1 = Poor 2 = Fair 3 = Good 4 = Excellent Q11b. How would you rate the restaurants you have visited in terms of High Quality Food? Q11c. How would you rate the restaurants you have visited in terms of Good Value for Money? Q11d. How would you rate the restaurants you have visited in terms of Enjoyable Atmosphere? <For Q11a through Q11d, use grid layout, with restaurants chosen in Q10 in the rows, and the rating scale in the columns. Randomize order of presentation of restaurants, with a maximum of 5 restaurants to be rated. Always include Subway if respondent chooses Subway in Q10. Do not ask respondents to rate any "Other Specify" restaurants from Q10. Randomize the order of presentation of questions Q11a through Q11d, and place each on a separate page.> Solution: For this illustration, we assume you already know the basics of building questions in CiW. Therefore, we only provide details regarding the more advanced functions of list building. 1. Q10 uses a list of twelve response members (fast-food restaurants) that you must specify as a predefined list. Specify this predefined list by adding Q10 to your questionnaire as a select-type question (check box select format), and while editing Q10 using the Response Options tab, or you can directly add the predefined list under Compose | Lists (the List Manager dialog). In either case, you are establishing a predefined list of members that we can use for Q10 and elsewhere in the questionnaire. We'll use this predefined list as a source "parent list" for building the constructed (customized) list of fast-food restaurants we'll use for Q11. Call the predefined list RestaurantsList. CiW is not case sensitive with respect to list names.

138

SSI Web v7 2. After specifying the 12 list members in RestaurantsList, highlight member 10 ("Other Specify (1)") and check the available box to indicate that it is an "other specify" response. Repeat for the additional Other Specify list member 11. For list member 12, highlight it and check the box to indicate that it is "Exclusive/None of the above" (meaning that if respondents check this category, it is the only check allowed). We also need to randomize members 1 through 9, and we can do this by clicking Randomize Response Options from the Response Options tab. (Note that the "None of the Above" and the "Other Specify" are automatically not randomized if they are at the bottom of the list.) Add a page break after Q10. 3. For Q11, we need to show the text "Next, we'd like to ask you to evaluate the restaurants you have visited in the last six months." at the top of the question, but we'll be randomizing the presentation of the remaining sections (subsections a through d). Specify a Text/HTML Filler question called Q11intro, and include the introductory sentence "Next, we'd like to ask you to evaluate the restaurants you have visited in the last six months." 4. The most challenging aspect of this example is to create the customized list of restaurants to include in Q11a through Q11d. We need to: Include restaurants chosen in Q10 (up to a maximum of 5) Randomize order of presentation of restaurants Always include Subway if respondent chooses Subway in Q10 Do not ask respondents to rate any Other Specify restaurants from Q10.

To accomplish this, we create a new constructed list to use in Q11a. From the main menu, click Compose | Lists to access the Lists dialog. Click Add, and then you should select Constructed List. Name the new list VisitedRestaurantsList. The key rule to remember when constructing new lists is that all the response options (list members) moved into this new list must originate from the same predefined (parent) list. For this questionnaire, all the response options you'll use to build VisitedRestaurantsList come from RestaurantsList. Using the drop-down Parent List control, select the parent list RestaurantsList. To begin with, the constructed list is empty even though you have specified that you'll be using members from RestaurantsList. Type the list building instructions that create the new VisitedRestaurantsList: AIC (Q10, 7) AIC (Q10, 1, 9) RANDOMIZE (2, 9) LISTMAX (5) RANDOMIZE () We describe these instructions one at a time below. AIC (Q10, 7) First, we force Subway (the 7th member on the parent list) onto the constructed list if it is chosen (you'll see further below why it was important to force it initially into the first position). AIC (Q10, 1, 9) Next, we add any other members (but not including the "Other Specifies" or "None of the above") chosen in Q10. These new members are placed on the constructed following Subway, if Subway was chosen. Note that if a list member (Subway) already exists on the constructed list, if a new instruction attempts to add the same member, it will not be added again (no duplicates are permitted).

General Web Interviewing (CiW) Help RANDOMIZE (2, 9)

139

We always want to include Subway on the final list (if chosen). Subway, if chosen, is in the first position on the constructed list. Other brands (following in positions 2 up to 9) are randomized. ListMax (5) We only want to ask about a maximum of 5 restaurants in Q11, so we trim the list using the ListMax instruction (which always trims from the bottom). Randomize ( ) Finally, we randomize the remaining elements on the list, so that Subway (if chosen) isn't always in the first position. 4. Add Q11a through Q11d as grid-style questions to the questionnaire. Each of these uses the same setup and constructed list. Remember for each question to use the constructed list VisitedRestaurantsList as the row members. The only aspect that changes across the questions is the performance attribute referred to in the heading text (such as Cleanliness, High Quality Food, Good Value for Money, Enjoyable Atmosphere). The easiest way to accomplish this is by copying and pasting Q11a within the list of questions. Highlight Q11a and click the Copy icon (or

press Ctrl-C), then click the Paste icon (or press Ctrl-V). A copy of the question is placed in the question list, but with a numeric index after the name. You can change the default name while editing the question, by clicking the Rename button. Create three more copies of Q11a, and change the wording in the header of each to insert "High Quality Food," "Good Value for Money," and "Enjoyable Atmosphere." Insert a page break between the Q11a to Q11d questions, and randomize the order of these questions by clicking the RandomizePages button on the Write Questionnaire dialog and selecting Q11a and Q11d as the anchor questions for the randomization sequence.

Example E: Gas station questionnaire

Learning Objectives: a) Illustrates the following functions: - ADD - AIC ("Add If Chosen") - ANC ("Add if Not Chosen") - RANDOMIZE - REMOVE b) Uses skip patterns to skip a question and to terminate disqualified respondents c) Demonstrates how respondent-supplied "other specify" answers can be used as response members in subsequent questions. The Questionnaire: Q1. Which of the following gas stations have you visited in the last year to purchase fuel for your

140

SSI Web v7 vehicle? <Select all that apply. Randomize options 1 through 5. Response option 9 is exclusive. Terminate if don't ever purchase fuel.> 1 2 3 4 5 6 7 8 9 Chevron Shell Sinclair Texaco Union 76 Other Specify (1) _________________________ Other Specify (2) _________________________ Other Specify (3) _________________________ I don't ever purchase fuel (none of the above)

Q2. Of those that you have visited, which gas station do you use most often? <Select one. Show gas stations chosen in Q1, including any "other specifies". > Q3. Among those gas stations you have not visited in the last year, which of these are available in your community? <Show all gas stations not selected in Q1. Select all that apply.> Q4. What are the main reasons you haven't visited Texaco, even though it is available in your community? <Open-end. Ask if not visited Texaco in last year and is available within respondent's community> Q5. Thank you, those are all the questions we have for you today. Solution: For this illustration, we assume you already know the basics of building questions in CiW. Therefore, we only provide details regarding the more advanced functions of list building and skip patterns. 1. Q1 uses a list of nine response members (gas stations) that you must specify as a predefined list. Specify this predefined list by adding Q1 to your questionnaire as a select-type question (check box select format), and while editing Q1 using the Response Options tab, or you can directly add the predefined list under Compose | Lists (the List Manager dialog). In either case, you are establishing a predefined list of members that we can use for Q1 and elsewhere in the questionnaire. We'll use this predefined list as a source "parent list" for building subsequent constructed (customized) lists we'll need for later questions in this example. Call the predefined list StationsList. CiW is not case sensitive with respect to list names. 2. After specifying the 9 list members in StationsList, highlight member 6 ("Other Specify (1)") and check the available box to indicate that it is an "other specify" response. Repeat for the additional Other Specify list members 7 and 8. For list member 9, highlight it and check the box to indicate that it is "Exclusive/None of the Above" (meaning that if respondents check this category, it is the only check allowed). Also note that you need to make Q1 a multi-select (Check Box) question, and this is specified on the Settings tab. 3. Prior to showing the response members in Q1, you need to randomize the order of the gas stations (members 1 to 5). There are two ways to do this. You can either randomize the members for this question only (question-level randomization, from the Response Options tab while editing question Q1), or you can create a new list (a "constructed" list) to use in Q1 using the response members from your predefined list StationsList, and then randomize the subset of members on that new list. The advantages of using the list-building approach to randomizing the list of gas stations for Q1 are that you can use this same randomized list later in the questionnaire, should you want to present this list again using the same randomized order for that respondent. Also, should you want

General Web Interviewing (CiW) Help

141

to know the order of the randomized members for each respondent, that information would be available in the data file. Although it is easier to use question-based randomization, let's use list building operations with constructed lists for illustration. From the main menu, click Compose | Lists to open the Lists dialog. Click Add, and then you should select Constructed List. Name the new list RandStationsList. The key rule to remember when constructing new lists is that all the response options (list members) moved into this new list must originate from the same predefined (parent) list. For this questionnaire, all the response options you'll use to build RandStationsList come from StationsList. Using the drop-down Parent List control, select the parent list StationsList. Next, type the list building instructions that create the new RandStationsList. To begin with, the constructed list is empty even though you have specified that you'll be using members from StationsList. For this example, you first need to move the members from the parent list onto the new list, and following that need to randomize members 1 through 5. The following instructions typed into the Constructed List Instructions edit box provided accomplish this: ADD (StationsList) RANDOMIZE (1, 5) Any instructions you use are processed one-at-a-time, in a linear fashion, started from the first instruction and working through the last instruction. You can check your syntax by clicking the "Check" icon.

"ADD (StationsList)" takes all list members from the predefined StationsList and adds them to the constructed list you are building "RandStationsList". "RANDOMIZE (1, 5)" randomizes elements 1 through 5 of the current constructed list.

Note that switching the order of these instructions would matter. The randomize instruction would be meaningless in the first position, as there would be no members yet in the constructed list to randomize. (ADD adds the list members.) 4. Now that you have created the list containing the response options to use in Q1 (the constructed list, RandStationsList), you need to tell CiW to use that list within Q1. If you haven't already done so, create a new select-type question called Q1. While editing Q1, click the Response Options tab. Click the Existing List button and using the drop-down control select to use RandStationsList as response options for Q1. The list members from the parent list are displayed as a reference. 5. In select-type question Q2, you need to show gas stations that were chosen in Q1. This list of response options will obviously vary per person, so we also must build a constructed list for Q2. Click Compose | Lists to access the Lists dialog. Add a new constructed list named VisitedStationsList. The parent (predefined) list that this new constructed list will draw from is StationsList. Use the AIC ("Add If Chosen") command to add any members chosen in Q1 to this new constructed list: AIC (Q1) Then, when editing Q2 in the Write Questionnaire area, select the VisitedStationsList constructed list as the response options. Note that when the AIC operates on an Other Specify response, SSI Web automatically adds the respondent's typed response as a member on the constructed list rather than the original list text. The original list member value is also recorded in the data file. As you add new questions to this questionnaire, insert a page break between each question by clicking the Add Page Break button.

142

SSI Web v7 6. In select-type question Q3, you need to show gas stations that were not chosen in Q1. This list of response options obviously varies per person, and so we must build another constructed list. Click Compose | Lists to access the Lists dialog. Add a new constructed list named NotVisitedList. The parent (predefined) list that this new constructed list will draw from is StationsList. Use the ANC command "Add if Not Chosen" to add any members not chosen in Q1. We would not want to display any not-selected Other Specify or "none of the above" responses in Q3, so we'll use the REMOVE instruction to remove members 6-9 (the "other specifies" and "none of the above") from the list: ANC (Q1) REMOVE (StationsList, 6, 9) "ANC (Q1)" takes any members not selected in Q1 and adds them to this new constructed list. All members not selected in Q1 have been added to the constructed list with the previous instruction. Since you don't want any of the "other specify" or "none of the above" response members to appear on this new list, use the instruction "REMOVE (StationsList, 6, 9)" to remove members 6 through 9 of StationsList from this new constructed list.

7. Q4 is an open-end question and Q5 is a terminating question type. They are involved in skip patterns, and it makes sense to add these questions to the questionnaire before specifying the related skip logic. We only want to ask Q4 if the respondent did not visit Texaco in the last year and it was available within their community. We could place a skip instruction on Q4 to determine whether Q4 needs to be skipped. This is called a "pre-skip." Before the question loads on a page, SSI Web determines whether this question should be displayed or skipped. For this questionnaire, a "pre-skip" could be associated with Q4 that skips Q4 if: Q1_4=1 OR Q3_4=0. In other words, skip Q4 if Texaco was chosen in multiple-select question Q1 (Texaco was visited) or if Texaco was not chosen in Q3 (Texaco was not available in the community). Make sure to use the drop-down box on the skip logic tab to indicate that the skip type is Pre-Skip (executed before the question is shown) and to skip to the Next Question. 8. Finally, we need to create a skip from Q1 to the final question if respondents don't ever purchase fuel. While editing Q1, click the Skip tab and specify to skip from Q1 to Q5 if the following is true: Q1_9=1. (The ninth item in Q1 is "checked".)

General Web Interviewing (CiW) Help

143

1.3.7

SSI Script
SSI Script is a scripting language that you can use within SSI Web. You can use SSI Script to write small sections of customized code that will enhance your web survey. Where can SSI Script be Used? Displaying Responses to Previous Questions Functions Unverified Perl

Where can SSI Script be Used?

SSI Script can be used in the following areas: Skip Logic - In Skip Logic skips are executed if the SSI Script that you provide evaluates to true. If, for example, the expression: ((Q1 + 3) > Q2) And (Q5 < 8) evaluates to true then the skip is taken. Note that no beginning and ending [% %] SSI Script tags should be used when specifying SSI Script within Skip Logic. List Building - Constructed lists are created by using SSI Script List Functions. For example: AIC(Q1) INSERT(5, ColorList) Note that no beginning and ending [% %] SSI Script tags should be used when specifying SSI Script within List Building. Inline Inline scripting involves inserting sections of SSI Script into your text or HTML. To enter a section of SSI Script into your text it must be wrapped by [% and %]. For example the following text could be entered into the header of a question: Why do you spend [% (PhoneHours / TotalHours) * 100 %] percent of your time at work on the phone? SSI Script is processed/executed on the server before it is sent down to the respondent's browser. The script between [% and %] is executed and the result is inserted into its place. If the respondent had answered 18 for the question PhoneHours and 40 for the question TotalHours, then the above text would be displayed to the respondent as: Why do you spend 45 percent of your time at work on the phone? Question Settings You can enter SSI Script in some question settings . For example, the Total setting in a Constant Sum question accepts the answer to a previous question. So, if [% Q1 %] were entered into the Totals field, then the respondent's answer to Q1 would be used as the total required for the Constant Sum question. Note that beginning and ending [% %] SSI Script tags are needed when specifying SSI Script within question setting fields. Quota Control You can enter SSI Script in the qualification logic that indicates whether a respondent qualifies to count toward a Cell Limit. Note that beginning and ending [% %] SSI Script tags should not be used when specifying SSI Script within quota control qualification logic.

We admit that it might be confusing to remember when [% %] tags are needed or not when adding SSI

144

SSI Web v7 Scripting to your questionnaire--or whether SSI Scripting is permitted at all in specific areas. Fortunately, SSI Web will give you an error message if your syntax is incorrect. When in doubt, just try using SSI Script and then click the "Check" icon to have SSI Web check your syntax. For example, it may tell you that the "[" character is not valid (which indicates that the tags are not needed). If you see the "Pencil" icon next to a field in SSI Web, that is an indicator that SSI Scripting is permitted for that field.

Comments
You might find it useful to add comments to your SSI Script. To add a comment, use a single quote character prior to the comment text. For example: 'Here I'm adding the items that were chosen AIC(Q1) or [% 4 + Q1 'Here we are adding 4 to the answer to Q1 %] Everything after the single quote on the same line (except for the closing "%]" ) is a comment and is ignored when the SSI Script is executed.

General Web Interviewing (CiW) Help 1.3.7.1

145

Displaying Responses Given to Previous Questions

You can display responses given to previous questions by referring to the question names (in skip logic, list building logic, or inline SSI Scripting). (Some researchers refer to this as piping.) If you are displaying text to the screen (for example, returning the answer for a previous question within a Header1 field for a question), you must enclose the question name within [% ... %] tags, such as: [% Q1 %] returns the response recorded in the data file for Q1

If you are referring to the answers in other areas of SSI Web, skip logic, and quota control, you do not include the [% ... %] tags. If the response is not in the data file, a blank is returned. In some cases, you might not want to return the answer in the data file, but the label that is associated with it. With single select type questions such as select, radio, combo, or grid, you can use LABEL(Q1) to return the label. Example: [% LABEL(Q1) %] returns the label associated with the response recorded in the data file for Q1

Example: Showing Response Given to a Previous Numeric Question Let's assume that you wanted to display the numeric value that the respondent provided earlier in the questionnaire in a question called "housesize." For example, in the Header1 text section of a selecttype question called "numminors", you might type: You previously said that you have [% housesize %] people in your household. How many of those individuals are minors (17 or younger)? When the questionnaire runs over the server, the numeric value that was provided in the question named "housesize" is displayed instead of "[% housesize %]".

Example: Showing the Label for a Single-Select Question Let's assume you asked respondents to indicate their favorite color, using a select-type (radio) button question called "FavoriteColor." For example, in the Header1 text question of a follow-up question, you might type: You previously said that your favorite color was [%Label(FavoriteColor)%] .

Example: Showing the Label for a Single-Select Question from a Grid Let's assume you asked respondents to indicate their favorite color, using a select-type (radio) button question within the third column in a grid with column-based questions called "Colors."

146

SSI Web v7

To show the respondent's favorite color on the screen in a later question, you might specify: You previously said that your favorite color was [%Label(Colors_c3)%] . Because Colors is a column-based grid question, the underscore notation referring to column "_c3" returns the row label associated with the answer to column 3. If the grid question were row-based, underscore notation returns the column label associated with the response to the referenced row.

General Web Interviewing (CiW) Help 1.3.7.2

147

Functions
SSI Web includes many functions that can be used in various areas of the software:

List Functions System Functions Quota Control Functions Math Functions String Functions Error Functions CBC Functions ACA Functions MaxDiff Functions CVA Functions ACBC Functions

Functions can be used in skip logic and list-building syntax. They can also be used in the middle of questionnaire text (such as in a Header1 section of a question), in error messages sent to respondents, or in Unverified Perl. As an example, one might want to skip to a certain question if the length of an open-end response is fewer than 2 characters (a String Function called Length). In the question skip syntax, you might indicate to skip to Q10 if the following is true: Length(Q8) < 2

148

SSI Web v7

1.3.7.2.1 List Functions

SSI Script:
ADD (ListName) ADD (ListName, Member#) ADD (ListName, StartMember#, EndMember#) AddSorted (QuestionName) AddSorted (QuestionName, intDirection)

Description:
Adds members from a specified list to the end of the constructed list. More info. Adds members rated/ranked within a multi-part radio/rank/numeric question to a constructed list, in ascending or descending order. Specify a 0 for intDirection to indicate descending order; specify 1 for ascending order. Add If Chosen: Adds members chosen in a select-type question onto a new constructed list. More info. Add If Equal: Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers equal to a specific value. More info. Add If Greater: Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers greater than a specific value. More info. Add If Less: Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers less than a specific value. More info.

AIC (QuestionName) AIC (QuestionName, Member#) AIC (QuestionName, StartMember#, EndMember#)

AIE (QuestionName, Value)

AIG (QuestionName, Value)

AIL (QuestionName, Value)

General Web Interviewing (CiW) Help ANC (QuestionName) ANC (QuestionName, Member#) ANC (QuestionName, StartMember#, EndMember#)

149

Add if Not Chosen: Adds members not chosen in a selecttype question onto a new constructed list. More info. Add if Not Equal: Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers not equal to a specific value. More info. Returns a string with every list item separated by "SeparatorText" with "FinalSeparatorText" prior to the last list item. For example: DisplayListLabels (fruitlist, "," , "and") returns: grape, peach, and cherry. Returns a string with all list items joined together with "BeforeText" before each item and "AfterText" after each item. For example "<li>" and "</li>" as BeforeText and AfterText returns a bulleted list. Inserts members from a specified list to an exact position within a constructed list. More info. Is true if the specified parent member exists in the specified list. More info Returns the label of a specified member number in a list. Returns a comma-separated list of member labels enclosed in quotes.

ANE (QuestionName, Value)

DisplayListLabels (ListName, "SeparatorText", "FinalSeparatorText")

FormatListLabels (ListName, "BeforeText", "AfterText")

INSERT (TargetPosition#, ListName) INSERT (TargetPosition#, ListName, Member#) INSERT (TargetPosition#, ListName, StartMember#, EndMember#) ListHasParentMember (ListName, ParentMember#)

ListLabel (ListName, Member#) ListLabelsArray (ListName)

150

SSI Web v7 ListLength (ListName) ListMax (Value)

Returns the number of members in the list.

Controls the maximum length of a list. More info. Controls the minimum length of a list. More info. Returns the list member position from the original parent list for the specified member# on the specified constructed list. Returns an array of list positions from the original parent list for the specified constructed list (example: [1, 4, 6]). Randomizes the members of a constructed list. More info. Removes the members in the specified list from the constructed list. More info. Reverses the members in a list. For example, list members A, B, C, D when reversed become D, C, B, A. Remove If Chosen. Removes items from the list that were chosen in QuestionName. Controls the length of a list. More info. Sorts a list based on list labels. Numbers are sorted below letters. Sorts the members in a constructed list with respect to their natural order (lowest to highest value) from their parent predefined list. For example, list members with values 5, 3, 2, 7 would be sorted as 2, 3, 5, 7.

ListMin (Value)

ListValue (ListName, Member#)

ListValuesArray (ListName)

Randomize Randomize Randomize Randomize

() (SeedValue) (StartMember#, EndMember#) (SeedValue, StartMember#, EndMember#)

Remove (ListName) Remove (ListName, Member#) Remove (ListName, StartMember#, EndMember#) Reverse ()

RIC (QuestionName) RIC (QuestionName, Member#) RIC (QuestionName, StartMember#, EndMember#) SetListLength () SortByLabel () SortByValue ()

General Web Interviewing (CiW) Help

151

1.3.7.2.2 System Functions

SSI Web includes a number of functions that can be used to display information to the screen, within skip logic, within constructed list logic, and in other fields within the SSI Web menu system. The "SSI Script" column below gives the syntax for System Functions.

SSI Script:
Answered(QuestionName)

Description:
Is true if the question was answered by the respondent, false otherwise. More info Returns the current block number when used within a Randomized Block sequence. Converts a variable of type "Boolean" to type "Number" for use in SSI Scripting. Returns information on respondent's browser. For use within Free Format questions. Returns the HTML for a checkbox, including the specific HTML compatible with Graphical Select buttons. More info. Returns debug information that assists you when testing the questionnaire. Returns a grayed out box displaying the total of all questions beginning with FirstQuestionName and ending with LastQuestionName. More info. Converts characters that are not valid for use in URLs to URL-supported strings. Returns the system graphics path.

BlockPosition()

BooleanToNumber(Boolean)

Browser() CheckSelect(QuestionName, Value)

Debug() DisplayTotal (FirstQuestionName, LastQuestionName, Width)

EncodeForURL(QuestionName)

GraphicsPath()

152

SSI Web v7 IPAddress() JavaScript() Returns respondent's IP Address. Is true if they have JavaScript, false otherwise. More info Returns the label that is associated with the answer to a single select response type question. Returns an HTML "submit" button (this helps in linking surveys). Converts a variable of type "Number" to type "Boolean" for use in SSI Scripting. Converts a variable of type "Number" to type "String" for use in SSI Scripting. Returns number of response options checked. QuestionName can be a select checkbox (i.e. Q1), a row or column of checkboxes (i.e. Q1_r1 or Q1_c1), or a Free Format question (i.e. Q1_checked). Returns information on Respondent's operating system. Returns current page number. Returns the time (in milliseconds, or one one-thousandths of a second) between Startpage and Endpage (integer values). More info

Label(QuestionName)

LinkButton("URL") NumberToBoolean(Number)

NumberToString(Number)

NumChecked(QuestionName)

OperatingSystem() PageNumber() PageTime(StartPage) PageTime(StartPage, EndPage)

General Web Interviewing (CiW) Help PopUp("Link Text", "File.htm", Width, Height)

153

Returns the HTML for a JavaScript pop-up. File.htm is assumed to be in the graphics folder. Or, File.htm can be elsewhere if referenced with full http:// path. More info Returns the progress bar to the screen. Typically placed in the Global Header or Global Footer. Turns off the progress bar on the page where it is placed. Returns the name of the current question. May be used in Header 1, etc. sections, and also in skip patterns. For use within Free Format questions. Returns the HTML for a radio button, including the specific HTML compatible with Graphical Select buttons. More info. Returns a random number. If one parameter is given, returns a uniform random decimal value between 0 and 1. If three parameters are given, returns a random integer value from Min# to Max#. Seed must be an integer. More info. Removes the Previous button from the page where it is placed. Returns the internal respondent number. Is true if the question has been seen by the respondent, or false if the question was not seen by the respondent. More info

ProgressBar()

ProgressBarOff() QuestionName()

RadioSelect(QuestionName, Value)

RandNum(Seed) RandNum(Seed, Min#, Max#)

RemovePrevious() RespNum() Shown(QuestionName)

154

SSI Web v7 StringToNumber(String) Studyname() SysRand() SysRand(Min#, Max#) SystemTime() Converts a variable of type "String" to type "Number" for use in SSI Scripting. Returns the name of a study. Returns a random number seeded by the system clock (different seed every time instruction is encountered). Returns a number representing the current time (number of non-leap seconds since January 1, 1970). Returns the global tab width specified in global settings Returns total number of pages in study. Returns respondent's User Agent. Returns the value of the question as it is recorded in the STUDYNAMEdat.cgi file. More Info.

TabWidth() TotalPages() UserAgent() Value(QuestionName)

For additional flexibility in SSI Scripting, see the Unverified Perl documentation.

Notes on use of Value (QuestionName) function:

The Value (QuestionName) function returns the value of the question as recorded in the data file. It can be used with various questions: Numeric, Open-End or Select (radio) question types: [%Value(G3)%] where G3 is the name of a question Select (check box) question types: [%Value(G3_1)%] where G3 is the name of a question, the number after the underscore indicates the response item#, and the value returned is "1" (checked) or "0" (not checked) Grid question type [%Value(G3_r2_c3)%] where G3 is the name of a grid question, and the response is in a particular row (_r#) and column (_c#)

General Web Interviewing (CiW) Help

155

1.3.7.2.3 Quota Control Functions

SSI Web includes a number of functions that can be used when employing Quota Control. The standard quota control options available in SSI Web's interface give a great deal of control. However, advanced users may wish to use additional functions having to do with quota control. The "SSI Script" column below gives the syntax for Quota Control Functions.

SSI Script:
Value(QuestionName)

Description:
Returns the quota cell value to which the respondent has been assigned for the quota question referred to by QuestionName; returns 0 if unassigned. Returns the quota cell name to which the respondent has been assigned; returns empty string (nothing) if unassigned Returns true (1) if the cell is open, false (0) if the cell is closed

QuotaCellName(QuestionName)

IsQuotaCellOpen (QuestionName, QUOTA_CELL_NAME) IsQuotaCellOpen (QuestionName, QUOTA_CELL_VALUE) QuotaCellRemaining (QuestionName, QUOTA_CELL_NAME) QuotaCellRemaining (QuestionName, QUOTA_CELL_VALUE) QuotaCellLimit (QuestionName, QUOTA_CELL_NAME) QuotaCellLimit (QuestionName, QUOTA_CELL_VALUE) QuotaCellCompletes (QuestionName, QUOTA_CELL_NAME) QuotaCellCompletes (QuestionName, QUOTA_CELL_VALUE) AreAllQuotasClosed()

Returns the number of completes left to fill the quota cell. Returns 0 if quota cell is full. Returns the quota cell limit

Returns number of completes in the quota cell

Returns true if all cells have been closed for all quotas. This might be useful if you don't want to show a "survey closed" message but would like to skip the respondent to a different survey. Returns true if all cells have been closed for the specified quota. Returns true if all cells have been closed in any quota.

AreAllQuotaCellsClosed (QuestionName) AreAnyQuotasClosed()

156

SSI Web v7

1.3.7.2.4 Math Functions

SSI Web includes a number of math functions that can be used to display information to the screen, within skip logic, within constructed list logic, for use with Hidden Variables, and in other fields within the SSI Web menu system. The "SSI Script" column below gives the syntax for Math Functions.

SSI Script:
Abs (*NUMBER) Ceiling (NUMBER) Exp (NUMBER) Floor (NUMBER) FORMATNUMBER (NUMBER, "ThousandsSeparator", "DecimalSeparator", NumDecimalPlaces) Log10 (NUMBER) Ln (NUMBER) POWER (NUMBER1, NUMBER2) ROUND (NUMBER) ROUND (NUMBER, #DECIMALS) ROUNDTONUMBER (NUMBER, ROUNDTO) ROUNDTONUMBER (NUMBER, ROUNDTO, #DECIMALS) Sqrt (NUMBER)

Description:
Returns the absolute value. Rounds the NUMBER up to the closest integer. Returns e (2.71828183) raised to the power of NUMBER Rounds the NUMBER down to the closest integer. Formats a number with commas and decimal places. More Information.

Returns the log (base 10) of NUMBER. Returns the natural log of NUMBER. Returns NUMBER1 raised to the NUMBER2 power. Rounds NUMBER to the #DECIMALS places. If no decimal places are specified, rounds to nearest integer. Rounds NUMBER to the nearest ROUNDTO value, such as to the nearest 10, 25, 50, 100, 250, 500, 1000, 5000, 10000, etc. Returns the square root of NUMBER.

*For all math functions, NUMBER can be any QuestionName that contains a numeric value or any numeric expression (including functions and mathematical operators) that returns a numeric value. For additional flexibility in SSI Scripting, see the Unverified Perl documentation.

General Web Interviewing (CiW) Help

157

1.3.7.2.5 String Functions

SSI Web includes a number of string functions for operations involving text variables (such as openends, etc.). The "SSI Script" column below gives the syntax for String Functions.

SSI Script:
IsNumber (QuestionName)

Description:
Is true if the entire string is all digits, false otherwise. More info Returns the number of characters contained in the response. Extracts a substring of a string given by QuestionName. The substring is extracted starting at the character number specified in BeginPosition (1based counting) with length specified in Length. Searchers for "TextToLookFor" in the answer of QuestionName. If found, a 1 is returned; if not found, a 0 is returned. Returns a lowercased version of the response. Returns an uppercased version of the response.

Length (QuestionName) Mid (QuestionName, BeginPosition, Length)

TextExists (QuestionName, "TextToLookFor") ToLower (QuestionName) ToUpper (QuestionName)

To concatenate strings, use the & operator. For example: [% "Hello " & Name %] If Name is an open-end question and the respondent answered "Bob" for Name, then the SSI Script would return: Hello Bob

158

SSI Web v7

1.3.7.2.6 Error Functions

Error Functions are used in displaying a message to the respondent that a question has not been completed properly, and that they must remedy the problem before continuing. The Error Functions only work when specified within the Compose | Survey Settings | Error Messages tab.

SSI Script:
ErrCursum () ErrField () ErrMax () ErrMin () ErrQName () ErrText ()

Description:
Returns the current sum of the constant sum question. Returns the question number on the page. Returns the maximum value allowed for the question. Returns the minimum value allowed for the question. Returns the question name. Returns the question-specific error message that is optionally specified at the question level, under the Advanced... button. Returns the total amount that the responses to the constant sum question should add to.

ErrTotal ()

For additional flexibility in SSI Scripting, see the Unverified Perl documentation.

General Web Interviewing (CiW) Help

159

1.3.7.2.7 CBC Functions

CBC Functions are only applicable in CBC (Choice-Based Conjoint) questionnaires. The CBCDesign function is useful when advanced researchers want to customize the format for their CBC questions.

SSI Script:
CBCAttributeLabel (ExerciseName_CBCQuestionName, AttributePosition#)

Description:
Returns the attribute label for the given attribute row position. Useful if the attribute position is randomized in the design. Returns the actual concept number from the design that is being displayed to the screen in ConceptPosition#. Position numbers will only differ if concept sorting is in place. This function is used when researchers want to customize the format of their CBC questions. Returns the level text for the specified CBC question in the specified concept position and level position within the task. This function is used when researchers want to customize the format of their CBC questions. Returns the "None" text for the CBC question (traditional "None" option only) and is only used in Shelf Facing Display CBC questions. Returns the design version number and question name. This function is useful when creating paper-and-pencil interviews. (Example: D-1, TCBCRAN1). Only available within a CBC task.

CBCDesignConceptValue (ExerciseName_CBCQuestionName, ConceptPosition#)

CBCDesignLevelText (ExerciseName_CBCQuestionName, ConceptPosition#, LevelTextPosition#)

CBCNone ()

CBCVersion ()

For additional flexibility in SSI Scripting, see the Unverified Perl documentation.

160

SSI Web v7

1.3.7.2.8 ACA Functions

ACA Functions are only applicable in ACA (Adaptive Conjoint Analysis) questionnaires.

SSI Script:
ACAAttribute ()

Description:
Returns the text for the attribute label within the ACARating or ACAImportance questions. Within the ACAImportance question, returns the level text for the "best" level rated from the previous ACARating question.

ACABest ()

ACABestLevelLabel Returns label for best attribute level of (ExerciseName, Attribute#, Rank#) Attribute #. Rank# optional. Rank# of 2 returns second-best attribute level. ACABestLevelValue Returns level number for best attribute (ExerciseName, Attribute#, Rank#) level of Attribute #. Rank# optional. Rank# of 2 returns second-best attribute level number. ACACalMin () ACACalMax () ACAImportance (ExerciseName, Attribute#) Used in ACACalibration to display minimum ACACalibration value used. Used in ACACalibration to display maximum ACACalibration value used. Returns the importance score for the attribute, as specified by Attribute#. May be used after the ACAImportance question section. (See also ACAStrictImportance.) Returns label for the least important attribute. Rank# optional. Rank# of 2 returns second-least important attribute. Returns attribute number of the least important attribute. Rank# optional. Rank# of 2 returns second-least important attribute number. Returns label for the most important attribute. Rank# optional. Rank# of 2 returns second-most important attribute. Returns attribute number of the most important attribute. Rank# optional. Rank# of 2 returns second-most important attribute number.

ACALeastImpAttLabel (ExerciseName, Rank#) ACALeastImpAttValue (ExerciseName, Rank#)

ACAMostImpAttLabel (ExerciseName, Rank#) ACAMostImpAttValue (ExerciseName, Rank#)

General Web Interviewing (CiW) Help ACAPriorsBestLevelLabel (ExerciseName, Attribute#) ACAPriorsBestLevelValue (ExerciseName, Attribute#) ACAPriorsWorstLevelLabel (ExerciseName, Attribute#) ACAPriorsWorstLevelValue (ExerciseName, Attribute#) ACAStrictImportance (ExerciseName, Attribute#) Returns the best level text for attribute#, based only on Priors information. Returns the best level number for attribute#, based only on Priors information. Returns the worst level text for attribute#, based only on Priors information. Returns the worst level number for attribute#, based only on Priors information. Returns the strict importance score for the attribute, as specified by Attribute#. May be used after the ACAImportance question section. Returns the part-worth utility for the attribute and level, as specified by Attribute# and Level#. May be used after the ACAImportance question section. More info. Within the ACAImportance question, returns the level text for the "worst" level rated from the previous ACARating question.

161

ACAUtility (ExerciseName, Attribute#, Level#)

ACAWorst ()

ACAWorstLevelLabel Returns label for worst attribute level of (ExerciseName, Attribute#, Rank#) Attribute #. Rank# optional. Rank# of 2 returns second-worst attribute level. ACAWorstLevelValue Returns level number for worst attribute (ExerciseName, Attribute#, Rank#) level of Attribute #. Rank# optional. Rank# of 2 returns second-worst attribute level number. For additional flexibility in SSI Scripting, see the Unverified Perl documentation.

162

SSI Web v7

1.3.7.2.9 CVA Functions

CVA Functions are only applicable in CVA (Conjoint Value Analysis) questionnaires.

SSI Script:
CVAVersion ()

Description:
Returns the design version number and question name. This function is useful when creating paper-and-pencil interviews involving multiple versions. (Example: D-1, TCVA1). Only available within a CVA task.

For additional flexibility in SSI Scripting, see the Unverified Perl documentation.

General Web Interviewing (CiW) Help

163

1.3.7.2.10 MaxDiff Functions

MaxDiff Functions are only applicable in studies containing MaxDiff exercises. The functions below are useful when advanced researchers want to customize the format for their MaxDiff questions.

SSI Script:
MaxDiffDesignLabel (ExerciseName_MaxDiffQuestionName, Position#)

Description:
Returns the item text for the specified MaxDiff question for the specified Position#. This function is used when researchers want to customize the format of their MaxDiff questions. Returns the item value for the specified MaxDiff question for the specified Position#. This function is used when researchers want to customize the format of their MaxDiff questions. Returns the list item label for the item with the rank specified. So a rank of 1 would return the list item label with the highest maxdiff logit score. More info. Returns the list item value for the item with the rank specified. So a rank of 1 would return the list item value with the highest maxdiff logit score. More info. Returns the maxdiff logit score for the ItemIndex of the list associated with ExerciseName. More info.

MaxDiffDesignValue (ExerciseName_MaxDiffQuestionName, Position#)

MaxDiffRankAttLabel(ExerciseName, Rank#)

MaxDiffRankAttValue(ExerciseName, Rank#)

MaxDiffScore(ExerciseName, ItemIndex)

For additional flexibility in SSI Scripting, see the Unverified Perl documentation.

164

SSI Web v7

1.3.7.2.11 ACBC Functions

ACBC Functions are only applicable in studies containing Adaptive CBC exercises.

SSI Script:
ACBCBYOLabel(ExerciseName, Attribute#) ACBCBYOValue(ExerciseName, Attribute#) ACBCCurrentCalibration()

Description:
Returns the level label selected in the BYO question for the indicated attribute#. Returns the level value selected in the BYO question for the indicated attribute#. Returns the index of the current Calibration Concept. Needs to be used inside of an ACBC Calibration. Returns the index of the current Choice Task. Needs to be used inside of an ACBC ChoiceTask. Returns the index of the current Screening Task. Needs to be used inside of an ACBC Screener. Returns True if the attribute and level specified has been declared a "Must Have." Returns True if the attribute and level specified has been declared "Unacceptable." If referencing a Summed Price attribute, specify a price value rather than the Level#. Returns the total number of Calibration Concepts. Can use this inside of a ACBCCalibration. If used outside of an ACBCCalibration needs to have the ACBCExerciseName passed as a parameter.

ACBCCurrentChoiceTask()

ACBCCurrentScreener()

ACBCIsMustHave (ExerciseName, Attribute#, Level#) ACBCIsUnacceptable (ExerciseName, Attribute#, Level#)

ACBCNumCalibrations()

General Web Interviewing (CiW) Help ACBCNumChoiceTasks()

165

Returns the total number of Choice Tasks. Can use this inside of a ACBCChoiceTask. If used outside of an ACBCChoiceTask needs to have the ACBCExerciseName passed as a parameter. Returns the total number of Product Concepts marked "a possibility" by the respondent. Returns the total number of Screening Tasks. Can use this inside of a ACBCScreener. If used outside of an ACBCScreener needs to have the ACBCExerciseName passed as a parameter. Returns the level label for the choice tournament winner for the indicated attribute#. Returns the level value for the choice tournament winner for the indicated attribute#. Within a BYO question, returns the conditional text/graphic element for the CondText# relationship from the conditional relationship table. If XPos and YPos (values expressed in Pixels) are given, then the conditional element is shown at that fixed position on the screen (hovering graphic).

ACBCNumScreenedInConcepts(ExerciseName)

ACBCNumScreeners()

ACBCWinnerLabel(ExerciseName, Attribute#)

ACBCWinnerValue(ExerciseName, Attribute#)

BYOCondText(ExerciseName, CondText#) BYOCondText(ExerciseName, CondText#, XPos,YPos)

For additional flexibility in SSI Scripting, see the Unverified Perl documentation.

166 1.3.7.3

SSI Web v7

More Detail on Boolean (True/False) Functions

Some functions in SSI Web are Boolean (true/false) functions that do not return a specific "value". They simply evaluate to true or false. You can use the syntax "not" prior to the Boolean function (as shown in the examples below).

ANSWERED ANSWERED is true if the question was answered by the respondent. Note that ANSWERED evaluates to false if the respondent did not ever see the question (use SHOWN if you are interested in whether the question was seen). (Note: check-box questions are an exception to these rules; checkbox questions, if ever displayed to the respondent are always considered "answered," with a value of 1 or 0.) The format is: Answered(QuestionName) For example, you may specify a skip in SSI Web that executes if the following is true: Answered(Q1) In other words, skip if Q1 was answered by the respondent. You could also execute a skip if Not Answered(Q1), which would execute if Q1 was not answered.

ISNUMBER ISNUMBER is true if the entire string is made up of numeric values (0 to 9), false otherwise. For example, the string "01605" contains only numeric characters. The format is: IsNumber(QuestionName) For example, you may specify a skip in SSI Web that executes if the following is true: IsNumber(Q1) In other words, skip if the respondent's answer to Q1 contains only numbers. You could also execute a skip if not IsNumber(Q1), which would execute if Q1 contained characters other than numbers 0 to 9.

JAVASCRIPT JAVASCRIPT is true if the respondent's browser has JavaScript 1.2 enabled (and you have turned on JavaScript use under Survey Settings), false otherwise. The format is: JavaScript( ) For example, you may specify a skip in SSI Web that executes if the following is true: JavaScript( )

General Web Interviewing (CiW) Help

167

In other words, skip if the respondent's system has JavaScript enabled. You could also execute a skip if not JavaScript( ), which would execute if the respondent's system did not have JavaScript enabled.

LISTHASPARENTMEMBER LISTHASPARENTMEMBER is true if the specified parent member exists in the specified list, false otherwise. The format is: ListHasParentMember(ListName, ParentMember#) For example, you may specify a skip in SSI Web that executes if the following is true: ListHasParentMember(BrandAwareList, 12) In other words, skip if the constructed list named BrandAwareList contains item #12 (from the original predefined parent list for BrandAwareList). You could also execute a skip if not ListHasParentMember(BrandAwareList, 12), which would execute if the BrandAwareList did not contain item #12.

SHOWN SHOWN is true if the question was seen by the respondent, or false if the question was not seen by the respondent. The format is: Shown(QuestionName) For example, you may specify a skip in SSI Web that executes if the following is true: Not Shown(Q1) In other words, skip if Q1 was not seen by the respondent. You could also execute a skip if Shown(Q1), which would execute if Q1 was seen by the respondent (even if it was not answered). Note: for questions such as multi-select, constant sum, Free Format questions and grid questions, the full variable name must be specified for the QuestionName, such as Q1_1, FF_name or GridQ1_r1_c1.

168 1.3.7.4

SSI Web v7

Additional Details on Certain Functions

POPUP Returns the HTML for a JavaScript pop-up. The HTML file that contains the text to be shown in the popup (File.htm) is assumed to be in the graphics folder. Or, the HTML file can be elsewhere if referenced with the full http:// path. The format is: PopUp("Link Text", "File.htm", Width, Height) Example 1: Assume that you have saved an HTML file containing the help in the study graphics folder. The text below is placed within the question text, such as within Header 1 or Header 2. [% PopUp("Click Here for Survey Help", "SurveyHelp.htm", 500, 500) %] When the survey is run, the above function displays a link that the respondent may click to launch a popup window containing the information in SurveyHelp.htm. The respondent can retire the popup window and return to the survey by clicking the "X" in the upperright corner of the popup window. Example 2: In this example, we reference an HTML page not in the graphics folder, but somewhere else on your web site. To do this, we include the full URL path to the HTML file. [% PopUp("Click Here for Survey Help", "http://www.mysurvey.com/mystudy/help/SurveyHelp.htm", 500, 500) %] Example 3: In this example, we replace the "Click Here for Survey Help" text with a graphic. [% PopUp("<img src=' " & GraphicsPath() & "help.jpg' alt=' ' border='0'>", "Help.htm", 500, 500) %]

FormatNumber This function formats numbers with thousands separators, decimal separators, and decimal places. The format is: FormatNumber (Number, ThousandsSeparator, DecimalSeparator, #DecimalPlaces) The ThousandsSeparator and DecimalSeparator are enclosed in double-quotes. For example, to make the ThousandsSeparator a comma, you should specify "," as the argument. Example 1: In the US, the value 7123456 would be formatted as 7,123,456.00. The following script formats this value to US standards: FormatNumber(7123456, ",", ".", 2)

General Web Interviewing (CiW) Help

169

Example 2: In Germany, the value 7123456 would be formatted as 7.123.456,00. The following script formats this value to German standards: FormatNumber(7123456, ".", ",", 2) Example 3: In most cases, you'll be formatting a value contained in a variable (such as a question asked earlier in the survey): FormatNumber(Q28, ",", ".", 0) Technical notes: DecimalPlaces can be replaced by a variable that returns a value. Thousands separators and decimal separators can be replaced by a variable that returns text.

Strict Importance If you use ACA's "Strict" importance function, we do not assume that importance is defined simply by computing the range from the most and least preferred levels within an attribute. We refer to the spread in utility between this respondent's prior best and worst levels (as specified in the attribute a priori order, or by the respondent in the ACARAT questions). If the best minus worst utility is negative, we set the range equal to zero. Strict importance keeps virtually unimportant attributes that potentially have reversals (out-of-order utilities) from getting inflated importance weights. It reduces the effect of random error on importance calculations. Consider the following part-worth utilities: $10 $15 $20 $25 1.45 1.50 0.75 0.50

Under standard importance calculations, we take the full spread in utilities for this attribute (1.50 - 0.50 = 1). With Strict importance, we take the spread in importance between the prior best and worst levels (1.45 - 0.50 = 0.95). As another example, consider the following part-worth utilities: Has widget "A" 0.20 No widget "A" 0.30 If having widget "A" is expected to be preferred to not having widget "A," based on prior rank-order (or from the respondent's ACARAT question), these utilities suggest a "reversal." The standard importance calculation takes the full range of 0.10 into the computation. Strict importance recognizes this as a reversal, and sets the importance equal to 0 for this attribute. We assume that the reversal is due to this attribute having virtually no importance, with random error overcoming any correct signal from the respondent regarding his/her true preference for levels within this attribute. Note: this function only applies within the ACA questionnaire. The market simulator (SMRT software) uses the standard definition of attribute importance.

PageTime PageTime(Startpage, Endpage) returns a value (in milliseconds) between Startpage and Endpage. Endpage is optional. If no Endpage is supplied, PageTime returns the total time spent on Startpage.

170

SSI Web v7 You can specify integers as arguments (the page number for the current question is shown at the bottom of the Write Questionnaire dialog when you highlight any question on the list). For example, you may also use: [%PageTime(PageNumber()-3, PageNumber())%] to return the time spent on the previous three pages, without worrying about hardcoding integer values. You might apply that script within a skip pattern to display a warning message if a respondent answered the previous pages too quickly. If you check the Save Respondent Time per Page setting in the Survey Settings + General Settings tab dialog, the time for each page (in milliseconds) is stored in the data file with variable naming convention sys_pagetime_#, where # is the page number. When you export the data, you can export these variables, and they appear in the export list below any constructed list items. To convert to seconds, divide by 1000. If the respondent backs up and re-answers a page (using SSI Web's Previous button), new time spent answering that page will be added to the previous time recorded. If respondents use the Back button on the browser to re-answer a page, the total time spent on the re-answered page may not be accounted for correctly.

RadioSelect This function returns HTML for a radio button, for use within Free Format questions. When Graphical Select buttons are enabled under the Survey Settings, General Format tab, the HTML required for displaying graphical radio buttons is relatively complex. The RadioSelect function returns the HTML necessary for displaying regular or graphical select questions. The syntax for this instruction is: RadioSelect(QuestionName, Value) Where QuestionName is the full QuestionName + Variable extension for a Free Format variable; and Value is the numeric value that is stored when the respondent clicks this particular radio button. Example: <table> <tr class="clickable"> <td class="input_cell"> [%RadioSelect(freeformat_radio, 1)%] </td> <td> ResponseOption_1 </td> </tr> <tr class="clickable"> <td class="input_cell"> [%RadioSelect(freeformat_radio, 2)%] </td> <td> ResponseOption_2 </td> </tr> <tr class="clickable"> <td class="input_cell"> [%RadioSelect(freeformat_radio, 3)%] </td> <td> ResponseOption_3

General Web Interviewing (CiW) Help </td> </tr> </table>

171

In the example above, the Free Format variable is named freeformat_radio. The RadioSelect instruction (highlighted in blue above) is called for each of the three response options (radio buttons), and the appropriate value is saved into the data file for each option. The HTML needed for showing Graphical Radio buttons is supplied when the survey runs on the server. Notice that we've declared classes "clickable" and "input_cell" within the <tr> and <td> tags. This makes the graphical select feature and the highlight text feature (when respondents mouse-over the response item text) work.

CheckSelect This function returns HTML for a checkbox, for use within Free Format questions. When Graphical Select buttons are enabled under the Survey Settings, General Format tab, the HTML required for displaying graphical radio buttons and checkboxes is relatively complex. The CheckSelect function makes the Free Format question appear and function like the other Select-type questions in your questionnaire. The syntax for this instruction is: CheckSelect(QuestionName, Value) Where QuestionName is the full QuestionName + Variable extension for a Free Format variable; and Value is the numeric value associated with each item in the checkbox question. Example: <table> <tr class="clickable"> <td class="input_cell"> [%CheckSelect(freeformat_check, 1)%] </td> <td> ResponseOption_1 </td> </tr> <tr class="clickable"> <td class="input_cell"> [%CheckSelect(freeformat_check, 2)%] </td> <td> ResponseOption_2 </td> </tr> <tr class="clickable"> <td class="input_cell"> [%CheckSelect(freeformat_check, 3)%] </td> <td> ResponseOption_3 </td> </tr> </table>

172

SSI Web v7 In the example above, the Free Format variable is named freeformat_check. The CheckSelect instruction (highlighted in blue above) is called for each of the three response options (check boxes). The CheckSelect instruction will return HTML that declares the variables freeformat_check_1, freeformat_check_2, and freeformat_check_3, and sets all variables equal to 1 if selected, and 0 otherwise. The HTML needed for showing Graphical Checkboxes is supplied (if needed) when the survey runs on the server. Notice that we've declared classes "clickable" and "input_cell" within the <tr> and <td> tags. This makes the graphical select feature and the highlight text feature (when respondents mouse-over the response item text) work.

DisplayTotal Returns a grayed out box displaying the total of all questions beginning with FirstQuestionName and ending with LastQuestionName. The syntax for this instruction is: DisplayTotal (FirstQuestionName , LastQuestionName, Width) The FirstQuestionName and LastQuestionName question names need to have the same base question names with a starting number at the end of FirstQuestionName and a last question number at the end of LastQuestionName. DisplayTotal will sum the values for all question names in between. For example: [%DisplayTotal (Num1, Num5, 5) %] will display the total of Num1 + Num2 + Num3 + Num4 + Num5. Notice that all question names must contain the same base name, in this case "Num". DisplayTotal is useful when using Free Format questions to implement custom constant sum CBC questions. May also be used for non-CBC questions.

General Web Interviewing (CiW) Help 1.3.7.5

173

Generating Random Numbers

Often, you may wish to generate a random number during a questionnaire. For example, you may wish to randomly branch to one question or another. Or, you may wish to use a random number in a mathematical function. SSI Web uses the RandNum and SysRand functions to generate random numbers.

Randnum Function
The random number generator is seeded first by the internal respondent number (plus any seed you specify). That keeps different respondents from receiving the same random value. If you provide a seed value in the RandNum function, you can call the RandNum function again for that same respondent and receive the same value each time. The formats are: RandNum(Seed) This returns a random decimal value greater than or equal to 0 and less than 1. The random values are distributed rectangularly, meaning that there is an equal probability of receiving a value between 0 and 0.1, 0.1 and 0.2, 0.3 and 0.4, etc. The Seed must be an integer zero or greater, and is used to seed the random number generator. The seed value used is actually the internal respondent number plus the specified seed. Therefore, if you call the RandNum function again for this respondent with the same given seed you will receive the same random value. But, different random values are generated for different respondents. RandNum(Seed, Min#, Max#) This returns a random integer in the range of Min# to Max#. Using a specific seed integer is useful if you want to be able to generate the same random value later in the questionnaire. The Seed must be an integer zero or greater, and is used to seed the random number generator. The seed value used is actually the internal respondent number plus the specified seed. Therefore, if you call the RandNum function again for this respondent with the same given seed you will receive the same random value. But, different random values are generated for different respondents.

SysRand Function
The SysRand function is used for generating truly random values (not repeatable), where the random number generator is always seeded by the system clock. Caution: when respondents back up in the survey, a new random number will be chosen the next time the SysRand function is encountered. The formats are: SysRand( ) This returns a random decimal value greater than or equal to 0 and less than 1. The random values are distributed rectangularly, meaning that there is an equal probability of receiving a value between 0 and 0.1, 0.1 and 0.2, 0.3 and 0.4, etc. The seed value used is the system clock at the moment the instruction is encountered. SysRand(Min#, Max#) This returns a random integer in the range of Min# to Max#. The seed value used is the system clock at the moment the instruction is encountered.

174 1.3.7.6

SSI Web v7

Unverified Perl
Unverified Perl can be used for additional flexibility in SSI Scripting. SSI Script has been implemented using the Perl Programming Language. When [% Q1 + 3 %] is specified in SSI Web, the server side Perl scripts actually receive that as [% VALUE("Q1") + 3; %] which is valid Perl source code. The Perl scripts evaluate this code and return the result. If additional power and flexibility are needed in your SSI Script, you can enter straight Perl code to accomplish the task. This is definitely an advanced area. SSI Web does not check your Unverified Perl for errors. You must thoroughly test your Perl logic. Unverified Perl Rules Unverified Perl Functions Perl Programming Language Basics Examples

Unverified Perl Rules

Unverified Perl can be entered by using the key words "Begin Unverified Perl" and "End Unverified". For example:
[% Begin Unverified Perl VALUE("Q1") + 3; End Unverified %]

Your Unverified Perl code is not checked for errors. Unverified Perl is executed on the server before the web page is sent down to the browser. So in order to execute this:
[% Begin Unverified Perl VALUE("Q1") + 3; End Unverified %]

Q1 must have been answered on a previous page. You cannot refer to the answer to a question if the question is being asked on the same page. Each statement must end with a semicolon. For example:
[% Begin Unverified Perl my $intVar = VALUE("Q1") + 3; $intVar = $intVar + (VALUE("Q3") / 3); return $intVar; End Unverified %]

Notice the semicolons at the end of each line. The last statement of the Unverified Perl is evaluated and the result returned. For example:
[% Begin Unverified Perl my $intVar = VALUE("Q1") + 3;

General Web Interviewing (CiW) Help

$intVar = $intVar + (VALUE("Q3") / 3); End Unverified %]

175

returns $intVar after all the math is completed. You can also return explicitly using the "return" keyword. For example:
[% Begin Unverified Perl my $intVar = VALUE("Q1") + 3; $intVar = $intVar + (VALUE("Q3") / 3); return $intVar; End Unverified %]

To compare two values, use == for numbers and eq for text. (!= means not equal for numbers, and ne means not equal for text.) For example:
Begin Unverified Perl if(VALUE("Q1") == 5) { AIC("Q3"); } if(VALUE("QName") eq "Sally") { ADD("ColorList", 4); } End Unverified

Functions must be in upper case. (There are a few exceptions to this rule. Please see Unverified Perl Functions for details). For example, while creating a constructed list you can specify:
Begin Unverified Perl if(VALUE("Q1") > 5) { AIC("Q3"); } End Unverified

Notice the functions VALUE and AIC are upper case. Question names and list names are case-sensitive, and must be enclosed in double quotes when used as function parameters. For example:
Begin Unverified Perl ADD("ColorsList"); AIC("q1"); End Unverified

The list ColorsList is enclosed in double quotes. "colorslist" is not valid, because the case does not match the case of the name of the defined list. Question names are similar. Avoid using single quotes in unverified Perl. If you need to use a single quote, you can do so in a separate Perl script file that you include.

176

SSI Web v7

Unverified Perl Functions

List Functions: SSI Script:
ADD (ListName) ADD (ListName, Member#) ADD (ListName, StartMember#, EndMember#) AddSorted (QuestionName) AddSorted (QuestionName, intDirection)

"Unverified" Perl:
ADD ("ListName") ADD ("ListName", Member#) ADD ("ListName", StartMember#, EndMember#) ADDSORTED ("QuestionName") ADDSORTED ("QuestionName", intDirection)

Description:
Adds members from a specified list to the end of the constructed list. More info. Adds members rated/ranked within a multi-part radio/rank/numeric question to a constructed list, in ascending or descending order. Specify a 0 for intDirection to indicate descending order; specify 1 for ascending order. Add If Chosen: Adds members chosen in a select-type question onto a new constructed list. More info.

AIC (QuestionName) AIC (QuestionName, Member#) AIC (QuestionName, StartMember#, EndMember#)

AIC ("QuestionName") AIC ("QuestionName", Member#) AIC ("QuestionName", StartMember#, EndMember#)

AIE (QuestionName, Value)

AIE ("QuestionName", Value)

Add If Equal: Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers equal to a specific value. More info. Add If Greater: Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers greater than a specific value. More info. Add If Less: Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers less than a specific value. More info.

AIG (QuestionName, Value)

AIG ("QuestionName", Value)

AIL (QuestionName, Value)

AIL ("QuestionName", Value)

General Web Interviewing (CiW) Help

ANC (QuestionName) ANC (QuestionName, Member#) ANC (QuestionName, StartMember#, EndMember#) ANE (QuestionName, Value) ANC ("QuestionName")

177

Add if Not Chosen: Adds members not chosen in a select-type question onto a new constructed list. More info. Add if Not Equal: Adds members in a grid numeric, grid constant sum, grid ranking, grid with all radio buttons, constant sum, or ranking question that were given answers not equal to a specific value. More info. Returns a string with every list item separated by "SeparatorText" with "FinalSeparatorText" prior to the last list item. For example: DisplayListLabels (fruitlist, "," , "and") returns: grape, peach, and cherry. Inserts members from a specified list to an exact position within a constructed list. More info.

ANE ("QuestionName", Value)

DisplayListLabels (ListName, SeparatorText", "FinalSeparatorText")

DISPLAYLISTLABELS ("ListName", "SeparatorText", "FinalSeparatorText")

INSERT (TargetPosition#, ListName) INSERT (TargetPosition#, ListName, Member#) INSERT (TargetPosition#, ListName, StartMember#, EndMember#) FormatListLabels (ListName, "BeforeText", AfterText")

INSERT (TargetPosition#, "ListName") INSERT (TargetPosition#, "ListName", Member#) INSERT (TargetPosition#, "ListName", StartMember#, EndMember#) FORMATLISTLABELS ("ListName", "BeforeText", AfterText")

Returns a string with all list items joined together with "BeforeText" before each item and "AfterText" after each item. For example "<li>" and "</li>" as BeforeText and AfterText returns a bulleted list. Is true if the specified parent member exists in the specified list. More info Returns the label of a specified member number in a list. Returns a comma-separated list of member labels enclosed in quotes.

ListHasParentMember (ListName, ParentMember#)

LISTHASPARENTMEMBER ("ListName", ParentMember#)

ListLabel (ListName, Member#) ListLabelsArray (ListName)

LISTLABEL ("ListName", Member#) LISTLABELSARRAY ("ListName")

178

SSI Web v7
ListLength (ListName) ListMax (Value) LISTLENGTH ("ListName") LISTMAX (Value) Returns the number of members in the list. Controls the maximum length of a list. More info. Controls the minimum length of a list. More info. Returns the value of the specified member number in a list. Returns a list of list values (example: [1, 4, 6]). Randomizes the members of a constructed list. More info.

ListMin (Value)

LISTMIN (Value)

ListValue (ListName, Member#) ListValuesArray (ListName) Randomize () Randomize (SeedValue) Randomize (StartMember#, EndMember#) Randomize (SeedValue, StartMember#, EndMember#) Remove (ListName) Remove (ListName, Member#) Remove (ListName, StartMember#, EndMember#) Reverse ()

LISTVALUE ("ListName", Member#) LISTVALUESARRAY ("ListName") RANDOMIZE () RANDOMIZE (SeedValue) RANDOMIZE (StartMember#, EndMember#) RANDOMIZE (SeedValue, StartMember#, EndMember#) REMOVE ("ListName") REMOVE ("ListName", Member#)

REMOVE ("ListName",
StartMember#, EndMember#) REVERSE ()

Removes the members in the specified list from the constructed list. More info. Reverses the members in a list. For example, list members A, B, C, D when reversed become D, C, B, A. Remove If Chosen. Removes all items from the list chosen in QuestionName. Controls the length of a list. More info. Sorts a list based on list labels. Numbers are sorted below letters. Sorts the members in a constructed list with respect to their natural order (lowest to highest value) from their parent predefined list. For example, list members with values 5, 3, 2, 7 would be sorted as 2, 3, 5, 7.

RIC (QuestionName) RIC (QuestionName, Member#) RIC (QuestionName, StartMember#, EndMember#) SetListLength () SortByLabel ()

RIC ("QuestionName") RIC ("QuestionName", Member#) RIC ("QuestionName", StartMember#, EndMember#) SETLISTLENGTH () SORTBYLABEL ()

SortByValue ()

SORTBYVALUE ()

General Web Interviewing (CiW) Help System Functions: SSI Script:

Answered(QuestionName)

179

"Unverified" Perl:
ANSWERED("QuestionName")

Description:
Is true if the question was answered by the respondent, false otherwise. More info Returns the current block number when used within a Randomized Block sequence. Returns information on respondent's browser. For use within Free Format questions. Returns the HTML for a checkbox, including the specific HTML compatible with Graphical Select buttons. More info. Returns debug information that assists you when testing the questionnaire. Returns a grayed out box displaying the total of all questions on the same page beginning with FirstQuestionName and ending with LastQuestionName. More info. Converts characters that are not valid for use in URLs to URL-supported strings. Returns respondent's IP Address. Returns the system graphics path.

BlockPosition()

BLOCKPOSITION()

Browser() CheckSelect (QuestionName, Value)

BROWSER() CHECKSELECT ("QuestionName", Value)

Debug()

DEBUG()

DisplayTotal (FirstQuestionName, LastQuestionName, Width)

DISPLAYTOTAL ("FirstQuestionName", "LastQuestionName", Width)

EncodeForURL(QuestionName)

ENCODEFORURL(QuestionName)

IPAddress() GraphicsPath()

IPADDRESS() GRAPHICSPATH()

180

SSI Web v7
JavaScript() JAVASCRIPT() Is true if they have JavaScript, false otherwise. More info Returns the label that is associated with the answer to a single select response type question. Returns an HTML "submit" button (this helps in linking surveys). Returns number of response options checked. QuestionName can be a select checkbox (i.e. Q1), a row or column of checkboxes (i.e. Q1_r1 or Q1_c1), or a Free Format question (i.e. Q1_checked). Returns information on respondent's operating system. Returns current page number. Returns the time (in milliseconds, or one onethousandths of a second) between Startpage and Endpage (integer values). Returns the HTML for a JavaScript pop-up. File.htm is assumed to be in the graphics folder. Or, File.htm can be elsewhere if referenced with full http:// path. More info

Label(QuestionName)

LABEL("QuestionName")

LinkButton("URL")

LINKBUTTON("URL")

NumChecked(QuestionName)

NUMCHECKED("QuestionName")

OperatingSystem()

OPERATINGSYSTEM()

PageNumber() PageTime(Startpage) PageTime(Startpage, Endpage)

PAGENUMBER() PAGETIME(Startpage) PAGETIME(Startpage, Endpage)

PopUp("LinkText", "File.htm", Width, Height)

POPUP("LinkText", "File.htm", Width, Height)

General Web Interviewing (CiW) Help

ProgressBar() PROGRESSBAR()

181

Returns the progress bar to the screen. Typically placed in the Global Header or Global Footer. Turns off the progress bar on the page where it is placed. Removes the Previous button from the page where it is placed. Returns the name of the current question. May be used in Header 1, etc. sections, and also in skip patterns. For use within Free Format questions. Returns the HTML for a radio button, including the specific HTML compatible with Graphical Select buttons. More info. Returns a random number. If one parameter is given, returns a uniform random decimal value between 0 and 1. If three parameters are given, returns a random integer value from Min# to Max#. Seed must be an integer. If Seed is 0, the system clock is used to seed the random number generator. More info. Returns the internal respondent number. Is true if the question has been seen by the respondent, or false if the question was not seen by the respondent More info Returns the name of the study. Returns a random number seeded by the system clock (different seed every time instruction is encountered). Returns a number representing the current time (number of non-leap seconds since January 1, 1970). Returns the global tab width specified in global settings . Returns the total number of pages in the study. Returns respondent's User Agent. Returns the value of the question as it is recorded in the STUDYNAMEdat.cgi file. More Info.

ProgressBarOff() RemovePrevious()

PROGRESSBAROFF() REMOVEPREVIOUS()

QuestionName()

QUESTIONNAME()

RadioSelect (QuestionName, Value)

RADIOSELECT ("QuestionName", Value)

RandNum(Seed) RandNum(Seed, Min#, Max#)

RANDNUM(Seed) RANDNUM(Seed, Min#, Max#)

RespNum() Shown(QuestionName)

RESPNUM() SHOWN("QuestionName")

Studyname() SysRand() SysRand(Min#, Max#)

STUDYNAME() SYSRAND() SYSRAND(Min#, Max#)

SystemTime()

time()

TabWidth() TotalPages() UserAgent() Value(QuestionName)

TABWIDTH() TOTALPAGES() USERAGENT() VALUE("QuestionName")

182

SSI Web v7 Quota Control Functions: SSI Script:

Value(QuestionName)

"Unverified" Perl:
VALUE("QuestionName")

Description:
Returns the quota cell value to which the respondent has been assigned for the quota question referred to by QuestionName; returns 0 if unassigned. Returns the quota cell name to which the respondent has been assigned; returns empty string (nothing) if unassigned Returns true (1) if the cell is open, false (0) if the cell is closed

QuotaCellName (QuestionName)

QUOTACELLNAME ("QuestionName")

IsQuotaCellOpen (QuestionName, QUOTA_CELL_NAME) IsQuotaCellOpen (QuestionName, QUOTA_CELL_VALUE) QuotaCellRemaining (QuestionName, QUOTA_CELL_NAME) QuotaCellRemaining (QuestionName, QUOTA_CELL_VALUE) QuotaCellLimit (QuestionName, QUOTA_CELL_NAME) QuotaCellLimit (QuestionName, QUOTA_CELL_VALUE) QuotaCellCompletes (QuestionName, QUOTA_CELL_NAME) QuotaCellCompletes (QuestionName, QUOTA_CELL_VALUE) AreAllQuotasClosed()

ISQUOTACELLOPEN ("QuestionName", QUOTA_CELL_VALUE)

QUOTACELLREMAINING ("QuestionName", QUOTA_CELL_VALUE)

Returns number of completes left to fill the quota cell. Returns 0 if quota cell is full.

QUOTACELLLIMIT ("QuestionName", QUOTA_CELL_VALUE)

Return the quota cell limit

QUOTACELLCOMPLETES ("QuestionName", QUOTA_CELL_VALUE)

Returns number of completes in the quota cell

AREALLQUOTASCLOSED()

Returns true if all cells have been closed. This might be useful if you don't want to show a "survey closed" message but would like to skip the respondent to a different survey. Returns true if all cells have been closed for the specified quota. Returns true if all cells have been closed for any quota.

AreAllQuotaCellsClosed (QuestionName) AreAnyQuotasClosed()

AREALLQUOTACELLSCLOSED ("QuestionName") AREANYQUOTASCLOSED()

General Web Interviewing (CiW) Help Math Functions: SSI Script:

Abs (*NUMBER) Ceiling (NUMBER) Exp (NUMBER) Floor (NUMBER) FORMATNUMBER (NUMBER, "ThousandsSeparator", "DecimalSeparator", NumDecimalPlaces) Log10 (NUMBER) Ln (NUMBER) POWER (NUMBER1, NUMBER2) ROUND (NUMBER) ROUND (NUMBER, #DECIMALS)

183

"Unverified" Perl:
ABS (NUMBER) CEILING (NUMBER) EXP (NUMBER) int (NUMBER) FORMATNUMBER (NUMBER, "ThousandsSeparator", "DecimalSeparator", NumDecimalPlaces) LOG10 (NUMBER) LOG (NUMBER) POWER (NUMBER1, NUMBER2) ROUND (NUMBER, 0) ROUND (NUMBER, #DECIMALS)

Description:
Returns the absolute value of NUMBER. Rounds the NUMBER up to the closest integer. Returns e (2.71828183) raised to the power of NUMBER Rounds the NUMBER down to the closest integer. Formats a number with commas and decimal places. More Information. Returns the log (base 10) of NUMBER. Returns the natural log of NUMBER. Returns NUMBER1 raised to the NUMBER2 power. Rounds NUMBER to the #DECIMALS places. If no decimal places are specified, rounds to nearest integer. Rounds NUMBER to the nearest ROUNDTO value, such as to the nearest 10, 25, 50, 100, 250, 500, 1000, 5000, 10000, etc. Returns the square root of NUMBER.

ROUNDTONUMBER (NUMBER, ROUNDTO) ROUNDTONUMBER (NUMBER, ROUNDTO, #DECIMALS) Sqrt (NUMBER)

ROUNDTONUMBER (NUMBER, ROUNDTO) ROUNDTONUMBER (NUMBER, ROUNDTO, #DECIMALS) sqrt (NUMBER)

*For all math functions, NUMBER can be any QuestionName that contains a numeric value or any numeric expression (including functions and mathematical operators) that returns a numeric value.

184

SSI Web v7 String Functions:

SSI Script: IsNumber (QuestionName) "Unverified" Perl: ISNUMBER ( VALUE("QuestionName")) length ( VALUE("QuestionName")) MID (VALUE("QuestionName"), BeginPosition, Length) Description: Is true if the entire string is all digits, false otherwise. More info Returns the number of characters contained in the response. Extracts a substring of a string given by QuestionName. The substring is extracted starting at the character number specified in BeginPosition (1based counting) with length specified in Length. Searchers for "TextToLookFor" in the answer of QuestionName. If found, a 1 is returned; if not found, a 0 is returned. Returns a lowercased version of the response. Returns an uppercased version of the response.

Length (QuestionName)

Mid (QuestionName, BeginPosition, Length)

TextExists (QuestionName, "TextToLookFor")

TEXTEXISTS (VALUE "QuestionName", "TextToLookFor")

ToLower (QuestionName) ToUpper (QuestionName)

lc (VALUE("QuestionName")) uc (VALUE("QuestionName"))

Error Functions:
SSI Script: ErrCursum () ErrField () ErrMax () ErrMin () ErrQName () ErrText () "Unverified" Perl: ERRCURSUM () ERRFIELD () ERRMAX () ERRMIN () ERRQNAME () ERRTEXT () Description: Returns the current sum of the constant sum question. Returns the question number on the page. Returns the maximum value allowed for the question. Returns the minimum value allowed for the question. Returns the question name. Returns the question-specific error message that is optionally specified at the question level, under the Advanced... button. Returns the total amount that the responses to the constant sum question should add to.

ErrTotal ()

ERRTOTAL ()

General Web Interviewing (CiW) Help CBC Functions:

SSI Script: CBCAttributeLabel (CBCQuestionName, AttributePosition#) "Unverified" Perl: CBCATTRIBUTELABEL ("CBCQuestionName", AttributePosition#) Description:

185

Returns the attribute label for the given attribute row position. Useful if the attribute position is randomized in the design. Returns the actual concept number from the design that is being displayed to the screen in ConceptPosition#. Position numbers will only differ if concept sorting is in place. This function is used when researchers want to customize the format of their CBC questions. Returns the level text for the specified CBC question for the specified concept position and level text position. This function is used when researchers want to customize the format of their CBC questions. Returns the "None" text for the CBC question (traditional "None" option only) and is only used in Shelf Facing Display CBC questions. Returns the design version number and question name. This function is useful when creating paper-and-pencil interviews. (Example: D-1, TCBCRAN1)

CBCDesignConceptValue (CBCQuestionName, ConceptPosition#)

CBCDESIGNCONCEPTVALUE ("CBCQuestionName", ConceptPosition#)

CBCDesignLevelText (CBCQuestionName, ConceptPosition#, LevelTextPosition#)

CBCDESIGNLEVELTEXT ("CBCQuestionName", ConceptPosition#, LevelTextPosition#)

CBCNone ()

CBCNONE ()

CBCVersion ()

CBCVERSION ()

186

SSI Web v7 ACA Functions:

SSI Script: ACAAttribute () "Unverified" Perl: ACAATTRIBUTE () Description: Returns the text for the attribute label within the ACARating or ACAImportance questions. Within the ACAImportance question, returns the level text for the "best" level rated from the previous ACARating question. Returns label for best attribute level of Attribute #. Rank# optional. Rank# of 2 returns second-best attribute level. Returns level number for best attribute level of Attribute #. Rank# optional. Rank# of 2 returns second-best attribute level number. Returns the importance score for the attribute, as specified by Attribute#. May be used after the ACAImportance question section. (See also ACAStrictImportance.) Returns label for the least important attribute. Rank# optional. Rank# of 2 returns second-least important attribute. Returns attribute number of the least important attribute. Rank# optional. Rank# of 2 returns second-least important attribute number. Returns label for the most important attribute. Rank# optional. Rank# of 2 returns second-most important attribute. Returns attribute number of the most important attribute. Rank# optional. Rank# of 2 returns second-most important attribute number. Returns the best level text for attribute#, based only on Priors information. Returns the best level number for attribute#, based only on Priors information.

ACABest ()

ACABEST ()

ACABestLevelLabel (Attribute#, Rank#)

ACABESTLEVELLABEL (Attribute#, Rank#)

ACABestLevelValue (Attribute#, Rank#)

ACABESTLEVELVALUE (Attribute#, Rank#)

ACAImportance (Attribute#)

ACAIMPORTANCE (Attribute#)

ACALeastImpAttLabel (Rank#)

ACALEASTIMPATTLABEL (Rank#)

ACALeastImpAttValue (Rank#)

ACALEASTIMPATTVALUE (Rank#)

ACAMostImpAttLabel (Rank#)

ACAMOSTIMPATTLABEL (Rank#)

ACAMostImpAttValue (Rank#)

ACAMOSTIMPATTVALUE (Rank#)

ACAPriorsBestLevelLabel (Attribute#) ACAPriorsBestLevelValue (Attribute#)

ACAPRIORSBESTLEVELLABEL (Attribute#) ACAPRIORSBESTLEVELVALUE (Attribute#)

General Web Interviewing (CiW) Help

ACAPriorsWorstLevelLabel (Attribute#) ACAPriorsWorstLevelValue (Attribute#) ACAStrictImportance (Attribute#) ACAPRIORSWORSTLEVELLABEL (Attribute#) ACAPRIORSWORSTLEVELVALUE (Attribute#) ACASTRICTIMPORTANCE (Attribute#)

187

Returns the worst level text for attribute#, based only on Priors information. Returns the worst level number for attribute#, based only on Priors information. Returns the strict importance score for the attribute, as specified by Attribute#. May be used after the ACAImportance question section. Returns the part-worth utility for the attribute and level, as specified by Attribute# and Level#. May be used after the ACAImportance question section. More info. Within the ACAImportance question, returns the level text for the "worst" level rated from the previous ACARating question. Returns label for worst attribute level of Attribute #. Rank# optional. Rank# of 2 returns second-worst attribute level. Returns level number for worst attribute level of Attribute #. Rank# optional. Rank# of 2 returns second-worst attribute level number.

ACAUtility (Attribute#, Level#)

ACAUTILITY (Attribute#, Level#)

ACAWorst ()

ACAWORST ()

ACAWorstLevelLabel (Attribute#, Rank#)

ACAWORSTLEVELLABEL (Attribute#, Rank#)

ACAWorstLevelValue (Attribute#, Rank#)

ACAWORSTLEVELVALUE (Attribute#, Rank#)

CVA Functions:
SSI Script: CVAVersion () "Unverified" Perl: CVAVERSION () Description: Returns the design version number and question name. This function is useful when creating paper-and-pencil interviews with multiple versions. (Example: D-1, TCVA1)

188

SSI Web v7 MaxDiff Functions:

SSI Script: MaxDiffDesignLabel (ExerciseName_ MaxDiffQuestionName, Position#) "Unverified" Perl: MAXDIFFDESIGNLABEL ("ExerciseName_ MaxDiffQuestionName", Position#) Description: Returns the item text for the specified MaxDiff question for the specified Position#. This function is used when researchers want to customize the format of their CBC questions. Returns the item value for the specified MaxDiff question for the specified Position#. This function is used when researchers want to customize the format of their CBC questions. Returns the list item label for the item with the rank specified. So a rank of 1 would return the list item label with the highest maxdiff logit score. More info. Returns the list item value for the item with the rank specified. So a rank of 1 would return the list item value with the highest maxdiff logit score. More info. Returns the maxdiff logit score for the ItemIndex of the list associated with ExerciseName. More info.

MaxDiffDesignValue (ExerciseName_ MaxDiffQuestionName, Position#)

MAXDIFFDESIGNVALUE ("ExerciseName_ MaxDiffQuestionName", Position#)

MaxDiffRankAttLabel (ExerciseName, Rank#)

MAXDIFFRANKATTLABEL ("ExerciseName", Rank#)

MaxDiffRankAttValue (ExerciseName, Rank#)

MAXDIFFRANKATTVALUE ("ExerciseName", Rank#)

MaxDiffScore (ExerciseName, ItemIndex)

MAXDIFFSCORE ("ExerciseName", ItemIndex)

General Web Interviewing (CiW) Help ACBC Functions:

SSI Script: ACBCBYOLabel (ExerciseName, Attribute#) ACBCBYOValue (ExerciseName, Attribute#) ACBCCurrentCalibration() "Unverified" Perl: ACBCBYOLABEL ("ExerciseName", Attribute#) ACBCBYOVALUE ("ExerciseName", Attribute#) ACBCCURRENTCALIBRATION() Description:

189

Returns the level label selected in the BYO question for the indicated attribute#. Returns the level value selected in the BYO question for the indicated attribute#. Returns the index of the current Calibration Concept. Needs to be used inside of an ACBC Calibration. Returns the index of the current Choice Task. Needs to be used inside of an ACBC ChoiceTask. Returns the index of the current Screening Task. Needs to be used inside of an ACBC Screener. Returns True if the attribute and level specified has been declared a "Must Have". Returns True if the attribute and level specified has been declared "Unacceptable". If referencing a Summed Price attribute, specify a price value rather than the Level#. Returns the total number of Calibration Concepts. Can use this inside of a ACBCCalibration. If used outside of an ACBCCalibration needs to have the ACBCExerciseName passed as a parameter.

ACBCCurrentChoiceTask()

ACBCCURRENTCHOICETASK()

ACBCCurrentScreener()

ACBCCURRENTSCREENER()

ACBCIsMustHave (ExerciseName, Attribute#, Level#)

ACBCISMUSTHAVE ("ExerciseName", Attribute#, Level#)

ACBCIsUnacceptable (ExerciseName, Attribute#, Level#)

ACBCISUNACCEPTABLE ("ExerciseName", Attribute#, Level#)

ACBCNumCalibrations()

ACBCNUMCALIBRATIONS()

190

SSI Web v7
ACBCNumChoiceTasks() ACBCNUMCHOICETASKS() Returns the total number of Choice Tasks. Can use this inside of a ACBCChoiceTask. If used outside of an ACBCChoiceTask needs to have the ACBCExerciseName passed as a parameter. Returns the total number of Product Concepts marked "a possibility" by the respondent. Returns the total number of Screening Tasks. Can use this inside of a ACBCScreener. If used outside of an ACBCScreener needs to have the ACBCExerciseName passed as a parameter. Returns the level label for the choice tournament winner for the indicated attribute#. Returns the level value for the choice tournament winner for the indicated attribute#. Within a BYO question, returns the conditional text/graphic element for the CondText# relationship from the conditional relationship table. If XPos and YPos (values expressed in Pixels) are given, then the conditional element is shown at that fixed position on the screen (hovering graphic).

ACBCNumScreenedInConcepts (ExerciseName) ACBCNumScreeners()

ACBCNUMSCREENEDINCONCEPTS ("ExerciseName") ACBCNUMSCREENERS()

ACBCWinnerLabel (ExerciseName, Attribute#) ACBCWinnerValue (ExerciseName, Attribute#) BYOCondText(ExerciseName, CondText#) BYOCondText(ExerciseName, CondText#, XPos, YPos)

ACBCWINNERLABEL ("ExerciseName", Attribute#) ACBCWINNERVALUE ("ExerciseName", Attribute#) BYOCONDTEXT ("ExerciseName", CondText#) BYOCONDTEXT ("ExerciseName", CondText#, XPos, YPos)

Perl Programming Language Basics

There are many books and web sites devoted to teaching Perl. An introductory Perl tutorial can be found at: http://perldoc.perl.org/perlintro.html. Variables Flow Control Custom Functions Variables Scalars Scalars can hold single values such as an integer, real number, or text. Scalar names must always begin with a $ sign. All variables in Unverified Perl must be declared using the word "my". For example:
my $numDays = 12; my $numTotalSeconds = 0; my $txtMessage = "The total number of seconds is:"; $numTotalSeconds = $numDays * 24 * 60 * 60; return $txtMessage . $numTotalSeconds;

General Web Interviewing (CiW) Help

191

Notice that before you can use $numDays, it needs to be declared using my $numDays. The first line declares $numDays as a scalar variable and then assigns it to contain the value 12. When run, the above script returns: The total number of seconds is: 1036800 (The "." in $txtMessage . $numTotalSeconds concatenates two values.) Arrays Arrays are variables that hold a collection of items. Array names must begin with an @ sign. For example:
my @Scores = (55, 22, 4, 17); my $FirstScore = 0; $FirstScore = $Scores[0];

The first line above creates an array named Scores. It then initializes @Scores to a list of scores: 55, 22, 4, and 17. To extract the elements of the array, you must replace the @ with a $ and reference the element using [ NUMBER ] where NUMBER is the ith list item, zero based. So [0] returns the 1st item, [1] the 2nd and so on. Hashes Hashes are variables that hold a collection of items that consist of key value pairs. The values are accessed by referring to their key. Hash names begin with a % sign. For example:
my %ScoresLookup = (); %ScoresLookup = ( 'Bob' => 55, 'Sally' => 22, 'Frank' => 4, 'Pam' => 17 ); return "Sally has a score of: " . $ScoresLookup{'Sally'};

The above code declares a hash named ScoresLookup. It assigns a set of key value pairs to ScoresLookup. Finally it returns the sentence: Sally has a score of: 22 Notice that to access an item's value, we replace the % in the hash name with $ and use {'KEY'} where KEY is the key that is associated with a certain value. Flow Control The "if" statement can be used to conditionally execute code. For example:
if(VALUE("Q1") > 5) { AIC("Q3"); }

The AIC("Q3") function is only executed if the answer to Q1 is greater than 5. The following statement is also valid:
if(VALUE("Q1") > 5) { AIC("Q3"); } elsif(VALUE("Q1") < 5) {

192

SSI Web v7
AIC("Q2"); } else { AIC("Q5"); }

If the answer to Q1 is greater than 5, then AIC("Q3") is executed. If it is less than 5, then AIC("Q2") is executed. If it is equal to 5, the first two tests fail and AIC("Q5") is executed. In conditional logic, use the following symbols: "&&" for "And" "||" for "Or" "==" for "=" For example:
if (VALUE("Q1") == 3 && (VALUE("Q2") > 4 || VALUE("Q2") < 100)) { AIC("Q5"); }

The "for" loop allows you to execute a section of code over and over again. For example:
my my my my @Scores = (55, 22, 4, 17); $TotalOfScores = 0; $i = 0; $FirstScore=0;

for($i = 1; $i <= 4; $i++) { $TotalOfScores = $TotalOfScores + $Scores[$i 1]; } $FirstScore = $Scores[0];

Another way that you could do this is with a "foreach." For example:
my @Scores = (55, 22, 4, 17); my $TotalOfScores = 0; my $SingleScore = 0; foreach $SingleScore (@Scores) { $TotalOfScores = $TotalOfScores + $SingleScore; }

The "foreach" fills $SingleScore with each value inside @Scores one at a time until it reaches the end of @Scores. Custom Functions You can create your own Perl functions by using the "sub" keyword. For example:
sub AddTwoNumbers { my($NumOne, $NumTwo) = @_; my $TheSum = $NumOne + $NumTwo; return $TheSum; }

The above code creates a new function named "AddTwoNumbers". The line:
my($NumOne, $NumTwo) = @_;

General Web Interviewing (CiW) Help

193

is the syntax required to pass two parameters to the function. To use this new function, use:
my $Result = 0; $Result = AddTwoNumbers(5, 6);

Result would then be assigned the number 11. When "AddTwoNumbers" is called, 5 is assigned to $NumOne and 6 to $NumTwo. The "return" keyword passes the result from the function back to the caller. Examples Conditional List Logic In constructed list logic, you might conditionally call a function. For example:
Begin Unverified Perl AIC("q1"); if(LISTHASPARENTMEMBER("ListA", 5)) { ADD("ParentList", 7); } End Unverified

This example begins by adding all of the items that were chosen from "q1". An Unverified Perl "if" statement is then used to add list item #7 if list item #5 is in the list. Dynamic Text - This example shows how to use Unverified Perl to show dynamic text to the screen. It also shows how to concatenate .
[% Begin Unverified Perl if(VALUE("q1") > 1) { return "Tell us about each of your " . VALUE("q1") . " cars"; } else { return "Tell us about your car"; } End Unverified %]

If the answer to "q1" is greater than 1 (say 5), the question header is in the plural form: Tell us about each of your 5 cars If "q1" is 1, the singular form is used: Tell us about your car Notice the line:
return "Tell us about each of your " . VALUE("q1") . " cars";

The period concatenates values together.

Working with Variables and Saving them to the Data File First, create a Free Format question with a variable of type "hidden." Then, include the following text in the HTML section of the Free Format question:

194

SSI Web v7
<input type="hidden" name="ff_income" value=" [% Begin Unverified Perl my $TakeHome = 0; $TakeHome = VALUE("QIncomeVar") * .85;

return $TakeHome; End Unverified %] ">

The text above defines an HTML hidden input field. The name of the input field is "ff_income" and the value is computed by SSI Script. In the SSI Script, a variable named "$TakeHome" is defined. "$TakeHome" is then assigned the value of the question "QIncomeVar" (answered on a previous page) multiplied by .85 and the result is returned. When the SSI Script runs on the server, it might return:
<input type="hidden" name="ff_income" value="42500">

When the form is submitted, the data field "ff_income" now contains 42500. You can then use this value throughout your survey and it is saved in your data file. Calling Other Perl Scripts You can use Unverified Perl to call other Perl scripts on your server. For example:
[% Begin Unverified Perl require "CustomPerl.pl"; my $Num = 5; my $strResult = ""; $strResult = CustomPerl::MyFunction($Num); End Unverified %]

The "require" statement above makes all of the Perl code in "CustomPerl.pl" available for you to call in your SSI Web survey. The function "MyFunction" which is declared in "CustomPerl.pl" is now available. Connecting to a Database The following code includes the code contained in "dbrun.pl" with the keyword "require". It then calls a function defined in "dbrun.pl" called "RunDBCommand" and returns the result from a database query.
[% Begin Unverified Perl require "dbrun.pl"; my $intRecordNum = 1; my $strCommand = "SELECT * FROM User_Directory WHERE id ='" . $intRecordNum . "';"; my $strResult = ""; eval { my $sth = 0; my $dbh = 0; ($sth, $dbh) = dbrun::RunDBCommand($strCommand, 0); if ($sth != 0 && $dbh != 0)

General Web Interviewing (CiW) Help

{ my $row = $sth->fetchrow_arrayref;

195

if ($row) { $strResult .= "<font color=\"blue\"><b>Record #</b></font>" . $row->[0]; $strResult .= "<h3>" . $row->[1] . "</h3>"; } else { $strResult .= "<br><b>No users were found. again.</b>"; } } $dbh->disconnect; }; if ($@) { $strResult .= "<br><br>" . $@ . "<br><br>"; exit(); } $strResult .= "<br><br>\n\nSQL:" . $strCommand; End Unverified %] Please try

196

SSI Web v7

1.3.8

Surveys in Non-English Languages

SSI Web may be used for developing surveys in languages other than English (such as Spanish, German, Japanese, Chinese, Korean, etc.).

Using the XML .XLF/XLIFF Format Import/Export

The most efficient way to develop surveys in multiple languages is to first program the entire survey (including all skip patterns, etc.) within SSI Web in the source language (for this explanation, we'll assume English). Then, using File | Translation | Export Text for Translation... export all the English text of the survey to an XML .xlf/XLIFF file format. Translation houses commonly use software tools that support this format. The XLIFF file includes each text string twice: once in the source language, and a second time for the translated version of that text. For example, the translation house initially sees the following information within the .xlf file: <source language> Don't eat at Pepe's house <target language> "Needs-Translation" Using their software tools, the translation house translates the target language text. The .xlf file they return to you might include: <source language> Don't eat at Pepe's house <target language> No comes en la casa de Pepe Use File | Save As to create a copy of your existing English (source language) project, giving the new SSI Web project a new name such as "mystudy_spanish." You would open the mystudy_spanish study, and using File | Translation | Import Translated Text..., bring all the translated text into your new Spanish study. All of the <target language> phrases found in the .xlf file overwrite the existing English phrases. A backup of your survey (prior to importing and overwriting the text) is saved in your project's Recovery folder. This results in a separate, parallel study for each language. But, the variable names are identical across different versions of the questionnaire. So, the data files (dat.dat) could be appended and exported in a single step (from the base tongue study), if desired.

Create the XLIFF File

When you select File | Translation | Export Text for Translation... the following is displayed:

General Web Interviewing (CiW) Help

197

A separate .XLF file must be created for each Target Language you wish to translate the questionnaire into. In the example above, we are translating from English (Source Language) into Spanish (Target Language). Each language is identified by either a two-character or three-character code ("en" for English, etc.). These codes are called the ISO 639-1 or ISO 639-2 codes. These standardized codes are required by many programs that operate on the .XLF file, so they can perform some automated processes. If the language you are using is not identified in the list above, you should visit: http://www.loc.gov/standards/iso639-2/php/code_list.php to find the appropriate code. Type the codes directly into the text box interface above.

Directly Typing or Copy-Pasting Text into SSI Web

We assume that you have the ability to enter the survey text (using a keyboard or other input device) or that you have the translated text in a Word document, or some other program that displays the text correctly (the characters display as they are intended to be shown in the survey). When authoring surveys in SSI Web, you should ensure that the characters appearing in the input boxes in SSI Web's interface are the characters that respondents should see. (They may not be formatted with the correct size, color, or font, but they should be the correct character. If you see a "?",

198

SSI Web v7 a box character, or just the wrong character(s), then the respondents are most likely not going to see the correct character either.) SSI Web uses a Unicode representation of the characters, and the STUDYNAMEqst.cgi file uses UTF-8 Unicode character encoding. You do not need to do anything additional to make SSI Web display your text correctly. Do NOT add a <META> tag specifying the character set. (SSI Web automatically includes a <META> tag indicating that the UTF-8 character set is being used.) SSI Web supports double-byte characters in file names for graphics and also in files you name during export. If you use Excel to open a .txt or .csv file that you have exported that contains double-byte characters, you should first open Excel, and then open the file using File | Open. This brings up a import wizard where you can specify to use Unicode encoding. You should have Excel 2003 or later to open files properly containing UTF-8 Unicode encoded characters. Many languages work without any special considerations (such as Spanish, German, French, etc.). Just enter the characters or copy and paste them into SSI Web, and they will appear correctly. If the characters do not appear correctly, then try the procedures outlined below. Test your survey carefully. Be careful to check the survey in Preview mode against your original Word document to ensure that the displayed text is correct. Be sure to test your survey on multiple browser types and using older versions of browsers since they all display HTML slightly different.

Before You Begin

First Steps A. Add Windows support for the language in which you'll be working. a. Windows XP Users: 1. Click Start + Control Panel + Regional and Language Options. 2. Under the Languages tab, check the two boxes for Install files for complex script and right-to-left languages (including Thai) and Install files for East Asian languages. 3. Restart your computer when prompted. You may also be asked to provide your Windows installation disks. You may need to have administrative rights to make these changes. b. Windows Vista Users (Ultimate or Enterprise Only): 1. Click Start + Settings + Control Panel + Clock, Language and Region + Regional and Language Options 2. Go to the Keyboard and Languages tab. Under Display Language, click Install/uninstall languages, and then follow the steps. If you are prompted for an administrator password or confirmation, type the password or provide confirmation. B. (This step is for users who will be copying from Microsoft Office into SSI Web.) If it is not already installed, add language support in Microsoft Office for the languages you'll be using. This may require you to download additional components from the Microsoft website. Click Start + Control Panel + Add or Remove Programs. Click Microsoft Office and click Change. When the setup dialog appears, select Add or Remove Features and then Next >. Select Office Shared Features International Support and make sure that all font support is installed.

General Web Interviewing (CiW) Help

199

1.3.9

Print Survey
Many researchers want to print a hard-copy representation of their survey. Typical reasons for doing this are: 1. To share a copy of the questionnaire with a client, 2. For internal review (quality assurance check). SSI Web's Print Survey (File | Print Study) capability can print/save a copy of your questionnaire in two formats: a) Text and Settings View: a text-only format intended for internal review that includes many details about each question: question text, response options, answer settings (such as min, max, number of responses, randomization, etc.), skip logic, lists, constructed list logic, attribute and level lists, and conjoint settings. b) Graphical View: a graphical PDF or HTML format that shows what the questionnaire looks like to the respondent as displayed by a web browser, with question name labels alongside each question for easy reference.

Treeview Navigation
You can navigate the "Treeview" menu to select which items to include in your printout. Click the "+" icons to expand and view the items under each node. The selected questions on the Question List are shared between the Text and Graphical views. Thus, if you select a subset of questions to print, if you switch from one view to the other, the selection applies to both views. Selecting a subset of questions from the question list. Often, you may want to print questionnaire for just a subset of the questions. First, de-select all questions in your list. Expand the Question List node by clicking its "+" icon. Right-click the label Question List and select uncheck all. Then, you can individually select the questions by checking the box next to each question label.

200

SSI Web v7

1.4
1.4.1

Respondent Passwords
Introduction to Passwords
Posting your questionnaire online gives anyone connected to the Internet potential access to your survey. The potential imposter/repeater problem is well-known. By default, SSI Web's password functionality is turned off, meaning that respondents may access the survey as many times as they want. You can limit or control this problem by assigning passwords within the Questionnaire Access and Passwords dialog. Password Fields You can require the respondent to type a single or multiple passwords (SSI Web supports up to 125* passwords for each respondent, each with up to 255 characters). You can also specify how many times each password combination may be used to access the survey. Most of the time, each password combination is specified for single use (Max Respondents=1). This has two benefits: each respondent may only take the survey once, and if respondents return to an incomplete survey, SSI Web can restart them at the point they left off. Restarting is not allowed for a password where Max Respondents permitted is greater than 1 (unless you use Cookies). Providing respondents with passwords permits them to access just their survey; it does not give them access to other aspects of your Web server, including the stored data. Merged Fields: Sometimes you already have some respondent data prior to fielding a survey (such as descriptive data for online panelists) and you want this information to be available during the questionnaire to display certain prior-known information or to execute skip patterns. SSI Web lets you supply up to 250 Merged Fields (variables) in addition to passwords that will be automatically merged into each respondent's record upon accessing the survey. Both the Password Fields and the Merged Fields are stored within the same database file. Pass-In Fields: It is common to "pass-through" data into the survey on the URL link that respondents click to start the survey. For example, you might be launching an SSI Web survey from a third-party application and want to pass-through the respondent ID# to be stored as well in the respondent's record in SSI Web. You define these Pass-In Fields within the Passwords area. The number of pass-in fields is only limited by the constraints on length of URL strings imposed by browsers. Cookies: By default, we do not write cookies to respondents' hard drives. However, you can turn cookies on, and if respondents do not block them (through their browser configuration settings), they can be used to restart incomplete questionnaires. The cookie will also keep a respondent from completing more than one survey (unless, of course, the respondent disables the cookie). If you ask SSI Web to write cookies, they will be used when the respondent could not otherwise be uniquely identified by the password specified to start the survey (e.g. passwords with Max Respondent>1, or no passwords in place at all). User-Defined Passwords: When this option is enabled, if the password that the respondent uses is not found, the respondent is still permitted to take the survey and the password(s) the respondent used are stored (so that the respondent could restart the survey at a later point if needed). Case sensitivity is ignored with User-Defined Passwords. You can employ User-Defined Passwords in addition to pre-defined passwords within the same study. You can use the Questionnaire Access and Passwords dialog to generate thousands or even millions of random passwords, or you can import passwords (as well as Merged and Pass-In fields) from text files. * The actual number of passwords that may be used is (2 x NumPasswords) + NumMergedVariables <= 250.

General Web Interviewing (CiW) Help

201

1.4.2

Start Question Layout

Every SSI Web study includes a Start question, with associated Questionnaire Access and Passwords. The Start question is commonly used to welcome respondents and ask them to type passwords prior to accessing the rest of the survey. If you aren't requiring passwords, this question may just be used to display introductory text (without a required response). Moreover, if you are bypassing the first page of the survey using One-Click Respondent Access to the Survey, respondents will never see the page including the Start question. The Start question has four main sections: 1. 2. 3. 4. Header 1 Header 2 Password Fields (optional) Footer

If you use passwords, they are indented according to the Question Indent Width you specify in the Survey Settings | Page Format dialog. The Questionnaire Access and Passwords dialog has four buttons: Advanced, Preview, OK and Cancel. Advanced... Preview OK Cancel This button accesses advanced settings for this question. Lets you preview how the Internet browser displays the current question. Saves changes to the current question and returns to the Write Questionnaire dialog. Returns to the Write Questionnaire dialog without saving changes to the current question.

Text entry fields and format controls (Color, Size, Bold, Italic, Underline, Justification) are provided for formatting question text. Editing Text Fields You specify header and footer text by typing directly into the Text fields within the dialog or by Clicking the "pencil" icon to edit the text within a larger window. You can include HTML instructions in any of the Text fields to enhance your surveys with additional styles or graphics. The toolbar within the text editor offers a number of buttons to automatically add HTML instructions within your survey. Format controls (Color, Size, Bold, Italic, Underline, and Justification) are available for formatting text using the paintbrush icon.

202

SSI Web v7

1.4.3

Password Fields
The Password Fields tab lets you specify any password fields (variables) that respondents are required to provide prior to accessing the survey. Up to 125 Password Fields (variables) may be specified. If multiple passwords are in place, all information must match prior to accessing the survey (for example, the respondent cannot access the survey by matching two out of three password fields correctly). Typically, if passwords are used, only one or two passwords are defined in this table (for example, a username and password). The table is displayed by default starting with a single blank row, where you may type a password Field Name, and supply additional settings. As you type information into this table, additional blank rows open beneath your last entry:

To specify a password field, type a label in the first row of this table, such as myusername. (The label must conform to question naming conventions in SSI Web.) Choose the data type (Text, if both letters and numbers are permitted; or Number if only numbers are permitted). Numbers are limited to the range 1 - 999,999,999. Click Masked if you want the text to appear as dots when the respondent types within the text box (so that others cannot read what actual letters are being typed). Click Case Sensitive if you want to distinguish between upper- and lower-case letters. For example, if Case Sensitive is checked, the password "John" is not equal to "john". In general, we wouldn't recommend using case sensitive passwords. Row <#> Password Label: This text field lets you specify the words that will appear next to the text entry box, for each password specified using a row in this table. For example, highlight the first row and type "Please type your Username:". If you have multiple passwords in this table, highlight each password to access the associated Password Label field. Justify: This setting controls whether the text you supply in the Row Label field is left-justified, centered, or right-justified. You can add another password field by typing in the next blank row, or by clicking the Insert a new password row icon. To delete a row, click the Delete the selected password row(s) icon. To reorder the password fields, click the up and down arrow icons.

General Web Interviewing (CiW) Help When you click Preview, you are shown an example of how the Start question will appear to respondents:

203

Click the Password Data icon to add the actual password data into the password data file.

204

SSI Web v7

1.4.4

Merged Fields
Sometimes you already have data for respondents prior to their completing the survey and you want this information to be available during the questionnaire (for example, to display certain prior-known information or to execute skip patterns). SSI Web lets you add up to 248 data fields (variables) in addition to the passwords that will be automatically merged into each respondent's record when the respondent accesses the survey. Merged data may be used just as regular survey data for skip patterns, displaying prior answers to the screen, etc. Both the passwords and the Merged Fields are stored within the same database file. Use the table to define the Merged Fields included in your password database. As an example, suppose we already have information on our respondents (prior to interviewing them) regarding their zip code and names. In this survey, we'll be asking them to evaluate a list of grocery stores in their area. Since we already know each respondent's zip code, we can merge each respondent's zip code into each respondent's record as soon as the respondent provides his or her unique password(s) and starts the survey. We'll use the zipcode variable to build a constructed list of grocery stores available within the zipcode. Furthermore, we'll use the respondent's first name to customize the welcome screen, such as "Thank you for taking this survey, Mary."

The Firstname variable may be referenced with SSI Scripting as would any other question variable within the study. For example: Thank-you for taking this survey, [% Firstname %]. You can add another Merged Fields variable by typing in the next blank row, or by clicking the Insert a new merged row icon. To delete a row, click the Delete the selected merged row(s) icon. To reorder the merged fields, click the up and down arrow icons. Click the Password Data icon to add the actual merged fields data into the password database.

General Web Interviewing (CiW) Help

205

1.4.5

Adding/Importing Password Data

After you have specified the names for Password(s) or (optional) Merged Fields that you will be using in the study, you need to provide the password data. SSI Web stores the password data (including any Merged Fields, as shown in the example below) within a Microsoft Jet database file format named studyname.pwds. There is virtually no limit to the number of rows of information in this table (we have had success importing or autogenerating a few million password rows). The number of columns in the data file along with available memory for your PC will affect your ability to import or autogenerate huge password data files. Click the Password Data icon to access this database file. The file is opened, and the display is similar to the look of a spreadsheet. In the example directly below, we have opened the Password Data file for a study where we've already defined one password field myusername as well as two additional variables we already know prior to interviewing the respondent (Merged Fields) called, zipcode and Firstname. We will not be asking the respondent to specify a zipcode and Firstname after logging into the survey, since we'll already be able to match and merge that information from the password data file once the respondent provides a correct (and unique) myusername. Max Respondents indicates how many respondents will be allowed to access the survey using the password combination for that row. The default is 1, and this has the benefit of allowing respondents to restart an incomplete survey where they left off.

Now that we've defined the three variables to be included in the password data file (studyname.pwds), we just need to add the rows of data. There are multiple ways to add the password and merged fields information. Directly typing: Highlight the field you wish to edit with the mouse, and begin typing. Copy and Paste from a Windows application: While in a separate Windows application such as Microsoft Excel, select an area by clicking and dragging the mouse over the desired cells.

206

SSI Web v7 Press Ctrl+C (or Edit + Copy) to copy the contents into the clipboard. Switch windows back to SSI Web. Position the cursor where you want the data to begin within the Password Data table. Press Ctrl+V (or Edit + Paste) to paste the information into the Passwords Data table. Importing from a .CSV file: Click the Import... button to merge information from a text-only .CSV (comma-separated value) file. The file must include a header row, with labels matching the labels you defined within SSI Web for Passwords and Merged Fields. If you want to import a Max Respondents field, you must also include that column within your .CSV file, with a column header label Max Respondents. If you do not include a Max Respondents field in your .CSV file, SSI Web automatically inserts Max Respondents=1 for all rows. When you attempt to import data into a Passwords file that already contains rows of information, you are asked whether you would like to append to or overwrite the existing data. Note: if you are importing a data file that has lots of columns, there is a copy-and-paste trick to avoid having to first type each field name into SSI Web (see below). Auto Generate: Click Auto Generate... to generate passwords with random values (text or numeric). You are asked to supply the following: Number of Rows to Generate: Specify how many rows you wish to generate. Number of Characters: Specify how many characters (letters and/or numbers) should be used for each password. Use Letters/Numbers: Check whether you wish to generate passwords that include only letters or only numbers. Exclude: Indicates which characters should never be used. By default, the following are excluded, because they can lead to confusion when respondents are asked to re-type their passwords: I (Capital I, as in India) O (Capital O, as in Oregon) 0 Zero 1 One You can add any additional characters you'd like to exclude. Of course, if you are automatically including the password information in the URL string ( One-Click Access), then respondents will not be asked to re-type their passwords, and it is not necessary to exclude any confusing characters. Auto Generate doesn't check for duplicate records as it generates the records. Therefore, if you wish to generate a large number of passwords, you may need to generate an even larger number to yield the target number of passwords once duplicates have been purged. To purge duplicates, click the Verify Password Data icon (looks like a red check mark). If you happen to forget this step, you will be warned regarding any duplicate passwords when testing your survey under Local Test Server or when you Prepare for Web Upload prior to fielding your study. At that point, you must return to delete any duplicates here on the Password Data dialog before testing or fielding your study. Auto Generate doesn't check for words that may be considered offensive. If you are auto-generating text-only passwords, you may want to check for offensive words.

Supported Characters for Passwords: Only ASCII codes 32 to 126 may be used for passwords. The following table displays these valid characters:

General Web Interviewing (CiW) Help

207

Merged Field data are not limited to ASCII codes 32 to 126.

Trick for Importing a Large Table of Password Fields (without having to retype the field names): The general process for importing Password (and Merged Fields) information from a .CSV file requires that you first specify one-at-a-time the column labels for all variables you are importing. If you have a .CSV file with variables whose header labels already conform to SSI Web question naming conventions, there is a copy-and-paste trick for bringing those labels into SSI Web that can save you a great deal of time and hassle. 1. Create a .CSV (comma-separated value) file that contains your password data (including any additional merged variable fields and a Max Respondents field if you need anything other than Max Respondent=1). 2. Within a spreadsheet program like Excel, highlight the first row containing the header labels. Copy this selection to the clipboard by pressing Ctrl+C or Edit+Copy. In a separate worksheet, click Edit+Paste Special+Transpose. (This copies the header labels to the second Excel worksheet and transposes them into a single column with multiple rows.) 3. Within your second Excel spreadsheet where the labels are now arranged as a single column with multiple rows, highlight just the labels for the password variables. Press Ctrl+C or Edit+Copy to copy this selection to the clipboard. Within SSI Web, open the Questionnaire Access and Passwords dialog. Open the Password Fields tab. Put your cursor in the upper-left cell in that table, and press Ctrl+V or Edit+Paste to paste in just the fields to be used as passwords. 4. Repeat the steps outlined directly above to copy-and-paste the Merged Field labels into the Merged Fields tab. 5. Now that all labels have been defined within SSI Web, click the Password Data... icon to open the password file. Click Import... and follow the prompts to import the password data from your .CSV file (or cut and paste the data from your Excel spreadsheet directly into the grid).

208

SSI Web v7

1.4.6

Pass-In Fields
Some users need to pass in variable(s) to their SSI Web survey, but don't have those data ahead of time so as to include them as Merged Variables in the password data file. For example, some users choose to link respondents from a third-party interviewing software into an SSI Web study, and need to pass through and store the respondent ID so that the two datasets can later be merged. However, the respondent ID might be dynamically assigned in the third-party application, and not known ahead of time. The number of variables that can be passed into an SSI Web study is limited only by the restriction imposed by browsers. Each can contain up to 255 characters of text. Use the Pass-In Fields tab on the Questionnaire Access and Passwords dialog to specify any passin variables. To pass in a variable, it must be included on the URL link that respondents click to access the survey. For example, the following link passes in two variables called ID and brandpref: http:://www.sawtoothsoftware.com/brd3/cgibin/ciwweb.pl?studyname=brd3&username=john&password=DGHQ&ID=42836290&brandpref=14 When the respondent clicks this link, it automatically starts the SSI Web survey called brd3, provides a username and password, and saves the ID and brandpref values to the data for this respondent. The Start question (page 1) is bypassed, and the first page the respondent sees is page 2. For more information about bypassing the starting page, please see One-Click Respondent Access to Survey. (Note: be careful about specifying whether the pass-in field must be numeric only. If this restriction is in place and text is passed into the variable, an error is written to the log file.)

General Web Interviewing (CiW) Help

209

1.4.7

Cookies and User-Defined Passwords

Cookies Cookies are pieces of text sent between the server and respondent's computer. The text is stored in a file on the respondent's PC in a folder reserved by the browser for storing cookie information. By default, SSI Web does not write cookies to respondents' computers. However, you can turn cookies on using this dialog, and if respondents do not block them (through their browser configuration settings), they can be used to restart incomplete questionnaires. The cookie will also keep a respondent from completing more than one survey (unless, of course, the respondent disables cookies or deletes the cookie). If cookies are enabled within your SSI Web study, when the respondent accesses the survey, a cookie is written to the respondent's hard drive that records the respondent's internal respondent ID and starting time. The cookie never expires. If a respondent logs into a survey and SSI Web cannot uniquely identify this respondent (because of passwords with Max Respondents>1, or because the study doesn't use passwords), the cookie information is read by SSI Web to identify and restart the respondent (or to determine that this respondent has already completed the survey). If all your passwords are unique per respondent (Max Respondents=1), then there really is no sense in turning cookies on, as they would never be needed to identify a respondent. To enable cookies for your study, from the Questionnaire Access and Passwords dialog, Settings tab, check Use cookies to allow respondents to restart the survey. Remember, cookies cannot guarantee that respondents can restart an abandoned survey or that respondents cannot complete multiple surveys from the same PC. Respondents can disable and block cookies. Some respondents may have configured their computers (or anti-virus software) to warn them when there is an attempt to write a cookie to their hard drives. Such respondents may be negatively inclined or even suspicious of your survey if they receive such a warning.

User-Defined Passwords This option lets respondents create their own password(s) to access the survey. All respondents are allowed into the survey, and returning respondents will be able to restart their survey where they left off if they provide the same password. You can employ User-Defined Passwords in addition to pre-defined passwords within the same study. To enable User-Defined Passwords, from the Questionnaire Access and Passwords dialog, Settings tab, check Allow respondents to define their own passwords. Min Size (characters) lets you control the minimum number of characters that may be specified for User-Defined Passwords. If you specify a minimum number of characters, you should warn respondents in the login instruction text regarding the minimum characters required. Also, you should modify the default Invalid Login error message that currently says: "Invalid Password/User Name. Access denied." to include a tip that the passwords must have at least a certain number of characters (edit the error message from the Survey Settings + Error Messages tab). Otherwise, respondents trying to access the survey may not know why they are receiving an error message. Additional Rules: Numeric user-defined passwords may not be larger than 999,999,999.

210

SSI Web v7 Numeric user-defined passwords may not begin with a 0. Numeric user-defined passwords may include a single decimal point.

General Web Interviewing (CiW) Help

211

1.4.8

Modifying Passwords When in Field

Once you have started fielding data, you can modify the password data as well as password settings. You can add or delete passwords, and you can change the maximum number of respondents permitted for each password. You make these changes locally on your PC, through SSI Web. Then, you upload the changes to your server setup. When you modify passwords, the following files potentially change, and we recommend you re-upload updated copies: studynameqst.cgi studynamepwds.cgi studynamelogin.html If you use the Field | Upload Study Files option to automatically upload your files to the server, this will be automatically handled. Note: We strongly suggest planning ahead so that you do not need to make changes when in field. If you must make an update, you should try to pick a low-traffic time. When you upload new copies of modified project files, respondents currently taking the survey may possibly experience a brief interruption or even a server error. If they receive a server error, clicking refresh on the browser should resume the survey.

212

SSI Web v7

1.5
1.5.1

Quota Control
Introduction to Quota Control
Survey researchers often want to close the survey to new respondents after a specific number of completed records, or after a certain number of records are completed by specific types of individuals (e.g. males, females). This section discusses Quota Controlfunctionality available in SSI Web for obtaining user-specified numbers of completed records within given classifications. The desire for Quota Control can be economic: respondents completing the survey are paid a cash incentive, and the researcher does not want to go over-budget. For example, we might want to achieve 1000 total completes. Quota Control is also used to achieve a sample reflecting known characteristics of a population: representative proportions of male, female, low income, high income, etc. Using Quota Control can also help us conserve our relationship with respondents: we limit the amount of time we ask of respondents to a few screener questions if we do not need their completed record.

Quota Based on a Single Question

Consider a B-to-B study, wherein the researcher wants to interview 1000 total respondents, with a certain number of completes within different company sizes (large, small): Quota Cell Limits: N=700 N=300 N=1000

Quota Cell Definitions: Large Company Small Company Total:

A respondent starts the survey and is first asked what size company she works for. This is called the screener question. After the screener question is asked, we determine whether another respondent with this characteristic is needed. If her Quota Cell is already filled with enough completed records for her company size, then we can skip her to the end and mark her as disqualified. Otherwise, she is allowed to continue the survey. This process is represented by the diagram below:

The Quota Qualification Point is inserted in the questionnaire from the Write Questionnaire dialog, by inserting a quota "question" (in the example above, directly following Q1). The quota question appears

General Web Interviewing (CiW) Help

213

in the question list alongside other questions. The quota question really isn't a question in the traditional sense (because we don't actually show anything on the screen for this question or ask the respondent to provide an answer to a Quota question). The Quota question type represents a Quota Qualification Point where we define the quota qualification logic and determine if the respondent belongs to a quota cell that is already full (this determination is made when the page containing the quota question is submitted). Based on whether a quota is full, we decide at the Quota Qualification Point whether to allow the respondent to continue or send the respondent to another point in the survey (such as a terminate screen).

Multiple Independent Quotas

Consider a B-to-B study, wherein the researcher wants to interview 1000 total respondents, with a certain number of completes within both different company sizes (large, small) and types (manufacturing, services): Quota Cell Limits: N=700 N=300 N=500 N=500 N=1000

Quota Cell Definitions: Company Size: Large Company Small Company Company Type: Manufacturing Services Total:

In this case we have two independent quotas: company size and company type. Within company size, there are two quota cells (large and small). Within company type, there are also two quota cells (manufacturing and services). Note that we aren't concerned with (for example) how many people from large manufacturing firms are interviewed (we don't care about the joint occurrences of the two quota variables). A respondent starts the survey and is first asked what size and type company she works for. These are called screener questions. After each screener question is asked, we can determine whether this respondent is needed. If for either company size or company type her Quota Cell is already filled with enough completed records, we can skip her to the end and mark her as disqualified. Otherwise, she is allowed to continue the survey. This process is represented by the diagram below:

214

SSI Web v7 In the example above, once we determine that the respondent belongs to a quota cell that is full, we immediately terminate the respondent.

Quotas Representing the Joint Occurrence of Multiple Variables

Consider a B-to-B study, wherein the researcher wants to interview 1000 total respondents, with a certain number of completes within different company sizes (large, small) and types (manufacturing, services): Quota Quota Cell Definitions: Cell Limits: Large Company, Manufacturing N=400 Large Company, Services Small Company, Services Total: N=300 N=200 N=1000 Small Company, Manufacturing N=100

As with the previous example, we plan to obtain 700 large company interviews, 300 small company interviews, 500 manufacturing interviews, and 500 services interviews. But in this example, we are concerned with how many respondents represent each company size AND type (joint occurrence). To do this, we establish a single quota variable (at a single Quota Qualification Point) with four possible cells (by inserting a Quota question from the Write Questionnaire dialog). For each cell, we use qualification logic that references both the company size and company type variables using AND logic. The questionnaire flow is shown below:

Controlling Total Number of Respondents

The simplest application of quota control is to limit the number of total completed records. From the Write Questionnaire screen, insert a Quota question (preferably on page 1, so that respondents starting a closed survey will immediately be skipped to your termination screen that tells them that the survey is closed). Add a single quota cell, specify the cell limit (such as 1000, for 1000 total completed records), and check Always qualify. This tells SSI Web that all respondents qualify for a single quota cell, irrespective of previous answers, and the maximum number of completed records is 1000.

General Web Interviewing (CiW) Help

215

Additional Notes and Capabilities

SSI Web's quota control is flexible and powerful. Quota membership may be based on many variables, resulting in a virtually unlimited number of quota cells. Screener variables can be "passed in" through a URL link or asked at the beginning of the questionnaire. Quota control cannot guarantee that the quotas you establish for groups are not exceeded. There may be some overage. For example, you may wish to interview just 100 males. A simple rule would be to let males continue the survey if you haven't yet achieved 100 completed male records. Imagine that 99 males have already completed the survey and five males are currently in progress (have passed the Quota Qualification Point and are in the middle of the survey), but these five haven't yet finished the survey and been counted as complete. A new male respondent reaches the qualification point, and is allowed to take the survey (since only 99 complete records have been recorded). We can see how this could result in 105 completed records, if all the respondents in progress eventually complete the survey. SSI Web has more sophisticated rules for determining if the Quota Cell Limit has been or is expected to be reached. You can specify that SSI Web should assume that, say, 80% of the respondents who have passed the Qualification Point and are actively taking the survey will eventually finish (i.e. a 20% drop-out rate). You can also specify that respondents who haven't completed the survey within a certain amount of time should be marked as Inactive. If these options are in place, when a new respondent reaches the Qualification Point, SSI Web considers the number of respondents in this respondent's quota cell already completed and actively in progress (discounted by 20%) to decide if the new respondent is needed. We refer to this ability to determine more intelligently if more respondents are needed within a Quota Cell as Predictive Quota Control. Predictive quota control can certainly limit the amount of overage, but it still cannot ensure that quota cell limits are not exceeded. Restarts: When a respondent abandons a survey and returns to take the survey (restarts), if the respondent belongs to a quota that is now filled, the respondent will be immediately skipped to the Disqualify Question associated with that quota (the question specified on the Settings tab).

216

SSI Web v7

1.5.2

Quota Dialog
Quota Control lets you specify how many interviews you want to complete overall, or for certain types of respondents. Once the quota is reached, new respondents who qualify for quota cells that have already filled can be skipped to a page that tells them that the survey is full. For a review of quota control, click here. Let's begin with the simple case where the researcher wants to limit the number of total surveys to 500 completed respondents. From the Write Questionnaire dialog, add a Quota question at a point very early on in the survey (we suggest page 1 so that respondents don't have to spend time answering questions if the quota is filled). The Quota question appears in the list of questions, but it really isn't a question in the traditional sense, since nothing is displayed on the screen and no respondent answer is required. Rather, it may be considered a Quota Qualification Point, at which SSI Web evaluates the quota logic defined in the Quota question, determines if that quota cell has been filled and assigns the respondent to a quota cell. When you add a Quota question, the Quota dialog is displayed:

Click the Add button to add a quota cell to the table. In this first example, we've added just one quota cell (named "Completes"), where all respondents qualify for this cell (we've clicked Always qualify rather than giving specific qualification logic). The relevant information specified in the Cells tab dialog is as follows:

General Web Interviewing (CiW) Help Value 1 Cell Name Completes Cell Limit 500 Qualification Logic Always qualify

217

The Cell Value is a numeric value that SSI Web assigns to represent a quota cell. This is the cell's numeric code. Cell Name is a short label for describing the quota cell which can contain no spaces. As another example, the researcher may wish to limit the survey to 500 males and 550 females, based on a previous question named Gender: Value 1 2 Cell Name Males Females Cell Limit 500 550 Qualification Logic Gender=1 Gender=2

This is an example of mutually-exclusive qualification logic: respondents are either male or female. With SSI Web's Quota Control, it is possible to set up quota cells that are not mutually exclusive. Even if quota cell definitions are not mutually exclusive, respondents are assigned to one (and only one) quota cell. The order you enter the Quota Cells matters if the Qualification Logic is not mutually exclusive (by default, Qualification Logic is evaluated from top to bottom of the list). By default, respondents are assigned to the first quota cell for which they qualify, from top to bottom. Quota cells can become more complex, such as a combination of Gender and Age. Also, if you do not like the default Cell Values, you can change them: Cell Value (Code) 101 102 201 202 Cell Name YoungMales OlderMales YoungFemales OlderFemales Cell Limit 250 200 300 240 Qualification Logic Gender=1 AND Age=1 Gender=1 AND Age=2 Gender=2 AND Age=1 Gender=2 AND Age=2

Click Add to add new quota cells, or (after highlighting a quota cell) click Delete to delete a quota cell. The Qualification Logic rules and syntax are described in the section that follows. Even though you have specified specific Cell Limits, it is very possible for you to exceed those limits by some (usually small) amount. For example, respondents may be entering the survey quickly, and since those who are In Progress are not counted toward the total completed records, quota control will let extra respondents into the survey before the Cell Limit has been reached. There are more sophisticated rules (Predictive Quota Control) for accounting for respondents who have passed the Qualification Point and are In Progress, and these are described in the documentation for Predictive Quota Control (Advanced tab). Note: Changes you make to the quota control settings must be uploaded to the web server for these modifications to affect data collection. Please read the section on Changing Quota Cell Settings During Data Collection for more information.

218

SSI Web v7 Qualification Logic Syntax SSI Web uses SSI Script for its qualification logic syntax (the syntax rules are the same as for skip patterns). You type this logic into the Qualification Logic text field. Supported Logical Operators: and or not Supported Mathematical Operators: =, <, >, <=, >=, <> "not equal to" +, / "division" * "multiplication" ^ "raise to the power of" mod "modulo"

and all other SSI Web Functions.

You can check whether the instructions (syntax) you provide are valid by clicking checked when you prepare your survey for local or remote web server. Example Syntax

. Syntax is also

Below are some examples of valid Qualification Logic syntax. If these statements evaluate to true, then the respondent is qualified for that particular cell. Q1 = 23 "Qualify for this cell if Q1 is equal to 23" Q1 >= 10 " Qualify for this cell if Q1 is less than or equal to 10" Q1 <> 99 " Qualify for this cell if Q1 is not equal to 99" Age > 18 (Age > 18) and (Gender = 1) (Age > 18) or (Q1 = 23) or (Q2 > 3.534) ((ageQuestion > 18) Or (Not Shown(ageQuestion))) "Shown" is a function that returns true if the question specified in the parenthesis has been seen Q23/(Q24+Q25) >= 23.5 (Q23*Q1)^0.5 < 10 "^" means "raised to the power of" Q1OpenEnd="hello world"

General Web Interviewing (CiW) Help

219

1.5.3

Quota Control Settings

Disqualified respondent skip to question <select question> (Disqualified Question) On the Settings tab, you specify the question that respondents should be skipped to who do not qualify for an open Quota Cell. Notes: Skip patterns (post-skips) are permitted on the same page as the Quota question. Quota Cell Membership is evaluated before the post-skips. The quota disqualified skip is taken before any user-defined post skips. Quota Membership will not be assigned if respondents pre-skip off the page containing the Quota question. Quota Cell Membership is evaluated even if the Quota question is skipped due to a constructed list length of zero used on the page. Quota Cell Membership is not evaluated if the page containing the Quota question is skipped over due to skip logic. Multiple quotas on a page are evaluated in natural order, even if questions on the page have been randomized. Quota membership is not evaluated if the quota question is pre-skipped.

Distinguishing Over-Quota vs. Disqualified

Some researchers need to track whether a respondent was disqualified due to not logically qualifying for the quota versus the respondent qualified for the quota but the quota was already filled. SSI Web records a quota cell value of -1 if respondents did not qualify due to logic, and -2 if respondents qualified but the quota was already filled. You can refer to these values within Skip logic if you need to redirect people to a different terminate screen based on one condition or the other. For example, all respondents are first sent to the same disqualification question as defined on the Settings tab of the Quota dialog. However, you can create "preskips" associated with that page which skip the respondent to the appropriate terminate screen depending on whether the quota cell value is -1 or -2.

220

SSI Web v7

1.5.4

Predictive Quota Control

In the previous sections describing Quota Control, we've emphasized that it is probable that you will actually get more completed records within each Quota Cell than specified in Cell Limits. This is especially the case if you simultaneously invite a large number of individuals to complete the survey. For example, if you wait to close a Quota Cell until the last respondent qualifying for this cell has completed the survey, there may be a number of respondents In Progress who eventually complete the survey. You will exceed your Cell Limit by these respondents still In Progress. The settings on this dialog let you fine-tune the decision rules (Predictive Quota Control) for determining if the Cell Limits have been, or are predicted to be, reached. The best settings for your project depend on a number of factors, the most significant being the rate at which respondents enter the survey. Do not use Predictive Quota Control Allow respondents qualifying for a quota cell to continue If the quota limit has not been completely filled The simplest rule is to allow respondents qualifying for a Quota Cell to enter and complete the survey until the number of completed respondents within that Quota Cell reaches the Cell Limit. Of course, this leads to the greatest likelihood of overage. But, it also fills the quota cells the fastest. Use Predictive Quota Control Allow respondents qualifying for a quota cell to continue if the number of completes plus "in progress" respondents that qualify for the quota cell is less than the quota limit When you check this option, SSI Web takes into account In Progress respondents when deciding whether new respondents entering the survey are needed to fill the Cell Limits. This approach reflects Predictive Quota Control. By counting all or a percentage of In Progress respondents toward the Cell Limit and not counting Inactive respondents, you can significantly reduce the overage on Cell Limits. This leads to the distinction between a quota cell that is full versus one that is closed. A quota cell is full when the number of completed respondents reaches the Cell Limit. A quota cell is closed when the Cell Limit has been reached or when it is projected to be reached by the number of completes plus In Progress respondents. With Predictive Quota Control, the Cell Limits may take longer to fill than when letting all new respondents enter the survey until enough completes are achieved. But if overage is costly, the tradeoff is usually preferable. As an illustration, if some In Progress respondents projected to complete the survey don't actually finish, you may have turned new respondents away that you later realize were actually needed. The settings on this Advanced tab help you reduce the overage while filling Cell Limits relatively quickly. In general, we suggest you use settings that reduce the likelihood of turning new respondents away, fill quota cells quicker, and result in only modest overage of Cell Limits. Use time limit By default, SSI Web counts all respondents who have passed the Quota question but have not yet completed the survey as In Progress. But, by checking the Use time limit box, you can specify a specific number of minutes to transpire (since the respondent has qualified for a quota cell) after which the respondent will be marked as Inactive rather than In Progress. By identifying which respondents are Inactive and highly unlikely to complete the survey, you can distinguish these from the In Progress respondents and allow new fresh respondents to begin the survey. (Respondents who have quit the survey can later restart if their quota cell is not filled. Note: respondents can restart even if their quota cell is closed, but not filled.) Larger Time Limits fill quota cells slower, but result in less overage. For example, if you

General Web Interviewing (CiW) Help

221

specify a very long Time Limit of 120 minutes, it could take up to two hours (depending on the Projected Completion Rate) until a replacement is sought for an In Progress respondent who quits the survey. If you are uncertain regarding a Time Limit, we suggest trying about 30 minutes (for a survey that takes on average 10 to 15 minutes). Use projected completion rate Assigning a Projected Completion Rate to In Progress respondents allows SSI Web to make more intelligent decisions regarding whether to close or open a Quota Cell. Typically, about 60-90% of respondents that answer the first few questions eventually complete the survey. This also means that around 10-40% drop out. By only counting a fraction of the respondents In Progress toward the Cell Limits, you are less likely to turn away new respondents that you actually need. Larger Projected Completion Rates fill quota cells slower, but result in less overage. Imagine that you project that 90% of In Progress respondents will complete the survey and only 60% actually do. Until the Time Limit expires for respondents who quit so they are marked as Inactive, new respondents will have been turned away that could have been used to fill the quota cell. If you are uncertain regarding a Projected Completion Rate, we suggest trying about 60%. Check for cell membership If your Quota Cells aren't mutually exclusive (if respondents can qualify for more than one Quota Cell), you'll want to consider the order that SSI Web evaluates the Quota Cells and Qualification Logic. By default, SSI Web checks Sequentially for Quota Membership. SSI Web starts at the top of the Quota Cell table (on the Cells tab) and works down the list. The respondent is assigned to the first Quota Cell for which she qualifies. Sometimes, (if respondents can qualify for multiple cells) you may want to fill quota cells randomly. Sequentially Evaluate and assign quota cell membership sequentially, from top to bottom of the table specified in the Cells tab. Randomize all Evaluate and assign quota cell membership randomly, with probability of assignment proportional to quota cell limits. Randomize subset Evaluate and assign quota cell membership randomly (with probability of assignment proportional to quota cell limits) within a subset of the range of quota cells from the table specified in the Cells tab. Those quota cells outside the range are evaluated sequentially.

222

SSI Web v7

1.5.5

Prequalification in Quota Control

This section describes a technique called Pre-Qualification for checking if all the Quota Cells for which a respondent could potentially qualify are full. If all potential quotas are full, we wouldn't want to waste the respondents time by asking additional questions. The simplest approach is to check at the very start of the survey (on page 1) if all quotas are full. To do this, add a post-skip (a skip that is executed after the respondent submits the question) to the Start question (or another question on page 1) with the following logic: Skip if the following logic evaluates to true: AreAllQuotasClosed() Click Here to view other SSI Script functions that involve Quota Control.

General Web Interviewing (CiW) Help

223

1.5.6

Quota Changes Mid-Study

SSI Web allows you to change some of your quota control settings even after you have begun data collection. Common changes include: Increasing or decreasing Quota Cell Limits Modifying the Quota Cell Name Adjusting Predictive Quota Control settings, including Time Limit, and Projected Completion Rate Adding additional quota cells You make these modifications within the SSI Web interface on your local computer (make sure to upload the changes, stored in the STUDYNAMEqst.cgi file, to the web server so that the settings take effect within the server installation.) If you use the Field | Upload Study Files option to automatically upload your files to the server, this will be automatically handled. There are certain settings you should not change after beginning data collection: Quota Cell Values Quota Cell Qualification Logic (you can change the logic to correct errors, but respondents already assigned to cells will not be reassigned) Quota question names

224

SSI Web v7

1.6
1.6.1

Testing/Running the Survey Locally

Testing/Running the Survey Locally
Once you have composed some questions and specified passwords, you can test/run your questionnaire locally on your PC using the Local Test Server that is installed along with the SSI Web system. Click Test | Test Survey Locally.... First, your project is checked for errors. Note: When testing your study, the first "test" respondent will always receive internal respondent number "1". Randomizations are seeded by respondent number. To see different randomizations and conjoint designs, you must take a few surveys. Caution: We do not recommend using Local Test Server for collecting real data. It raises the possibility of accidental data loss. Rather, we recommend you use the SSI Web CAPI interviewing procedure if collecting data locally on your PC. If you are using the demo version of the software, you are limited to a total of 5 completed records (either qualified or disqualified). The demo software is not to be used for professional research studies. Test Survey Locally Click this to test/run your survey using the Local Test Server. Test Survey Locally with Options... Click this to test/run your survey, but with the ability to Skip to Question, Show Question Names, Show Variable Names, Use a Specific Internal Respondent Number, Remove all Skip Logic, or Remove Randomization. Admin Module... Click this to run your Admin Module, to view, accumulate or tabulate results for your test study running on your Local Test Server. Download Data Click this to download data for your test study running on your Local Test Server. Reset Click this to Reset your test study. Reset means deleting all data and resetting all passwords and quotas for your test survey! Settings... Accesses the Local Test Server Settings dialog, that permits you to define whether the Apache server should start up (and run in the background) when SSI Web is opened and closed when it is closed, or whether the user can control manually when the Local Test Server is started and stopped.

Acknowledgment and Warning

The Local Test Server included in SSI Web includes software developed by the Apache Group for use in the Apache HTTP server project ( http://www.apache.org/). The Apache Group gives the following disclaimer:

General Web Interviewing (CiW) Help

225

"The Win32 release of Apache should still be considered beta quality code. It does not meet the normal standards of stability and security that Unix releases do. There are numerous known bugs and inconsistencies. There is also a much greater chance of security holes being present in the Win32 version of Apache." We at Sawtooth Software have spent many hours testing Apache Server code with SSI Web and have seen good stability and performance. Using the Local Test Server within SSI Web is very convenient for testing your survey prior to uploading to a Web server. But, because of this disclaimer, if you plan to use the SSI Web CAPI mode for actual data collection, you must test your study thoroughly to make sure it is executed as you expect and that the data are being stored properly.

Troubleshooting
If the survey is not working (Login page cannot be found) you may need to adjust the browser settings. In Explorer, select Tools | Internet Options | Connections | LAN Settings. Make sure if you are using a proxy server that you bypass proxy server for local addresses. Some corporate LANs have network configurations that do not allow a Local Test Server to work. Contact your IT department for help.

226

SSI Web v7

1.7
1.7.1

Survey Installation on Web Server

Uploading Survey to Web Server
SSI Web can upload your project to a web server (Field | Upload Study Files) to publish your questionnaire to the Web. This web server can be your company's server, a rented server, or Sawtooth Software's servers (we offer a hosting service for SSI Web questionnaires). When you click Field | Upload Study Files, the Upload Study Files dialog is shown:

You may have multiple servers that you use, and each server's FTP (File Transfer Protocol) configuration can be stored within this dialog. When you open a different project in SSI Web, any server configuration information you have entered for other projects (on your same computer) is shared and made available. Click Add... to add a new configuration, Edit... to edit an existing configuration, or Delete to delete a configuration. Each project must be uploaded to its own folder on the server. If you try to upload a project to a folder already containing an SSI Web project (at the root level), SSI Web will warn you of this and not proceed. Use the Remote Study Folder (Optional) to create a new folder for the current SSI Web study on your web server. For example, specify study1 to place your study within the study1 folder on your web server. If you are using a common cgi-bin for your studies, click Don't upload cgi-bin. Use Field | Prepare for Manual Upload if you want to save a copy of all the files to be placed on the server so you can manually upload the project using your own FTP software. When you click Add... or Edit... to add or edit a server configuration, the Add Account dialog is displayed:

General Web Interviewing (CiW) Help

227

Your ISP, Sawtooth Software hosting service, or your IT department will provide you with the Host Name, User Name, and Password needed to connect with and save files to your server. The Host Name provides the location of your server, and the User Name and Password log you into that server. We encourage you to use strong passwords that include a combination of alphabetic and numeric characters, and are at least eight characters long.

Advanced Settings: When you click Advanced, the Advanced Server Configuration dialog is displayed:

FTP Connection determines how auto upload will copy your files to the server. Normal is the standard network protocol for uploading files with no security. Over SSL is an extension to FTP that will securely upload your files using Transport Layer Security (TLS) or Secure Sockets Layer (SSL) if your server supports FTPS or FTP-SSL or FTP-Secure. Over SSH uses the Secure Shell protocol 2.0 (SSH-2) to securely upload your files if your server supports SFTP or Secure FTP or tunneling FTP through an SSH Connection.

228

SSI Web v7 Transfer Mode determines how the connection is established. Passive Mode is the default settings on most servers, and should also be used if your computer is behind a firewall. Sometimes, you will be asked to use a different Port number from the default. If you have been given a Port number, specify that here. Active Mode tells the server to initiate the connection to your computer using the specified range of ports.

How Respondents and Survey Administrators Access the Survey

Once you've uploaded your project to the Web server, you'll need to know how respondents can access the survey and how you access the survey to manage and download the data. We'll assume that your webserver has associated URL: www.yourwebserver.com and that the Remote Path you used (to create a folder for your study) was study1. We'll also assume that your study is named mystudy1. Respondents can open the Start page by browsing to: http://www.yourwebserver.com/study1/mystudylogin.html If you want respondents to be able to click a link that automatically logs them into the questionnaire and bypasses the start screen, see instructions on one-click access to the questionnaire. You can download respondent data by clicking Field | Download Data You can access the Admin Module to manage and view your data by browsing to: http://www.yourwebserver.com/study1/mystudyadmin.html

General Web Interviewing (CiW) Help

229

1.7.2

Manual Upload to Server

Overview
SSI Web can automatically upload your project to a server using Field | Upload Survey Files. If you want to manage to process yourself, using FTP software, this section gives you the needed information. Surveys are created using the SSI Web authoring interface on a local PC. The SSI Web authoring program saves various files that store the study setup, passwords, and questionnaire settings (these files are listed below). These study-related files, together with SSI Web's Perl script files (programs that run the survey), are uploaded to a remote web server. The data stored on the web server can be managed and retrieved using SSI Web's Online Administrative Module, which is made available by installing the files listed below. The documentation in this section assumes Linux/Unix based servers. Our website includes full web server documentation including instructions for Microsoft IIS servers at http://www.sawtoothsoftware.com/support/issues/ssiweb/server_setup.

Create Files to Upload to Server

After you have finished composing an SSI Web survey, click Field | Prepare for Manual Upload (from the SSI Web interface). Prepare for Manual Upload assembles the files required to run the web survey, placing them in a Web Upload folder inside the study folder on your hard drive. These are the files you should upload to the web server. These files are shown below, in the Setting Permissions section.

Uploading Files and Folders

The files and folders created in the Web Upload folder must be copied to and configured on your web server. By default, we expect you can set up a cgi-bin folder for each study. Some ISPs do not allow this. For configurations other than the default, see the Alternate Configuration section below. The contents of the Web Upload folder are usually uploaded to the web server via FTP (File Transfer Protocol). FTP is a method for transferring (uploading) files from your local PC to the web server on the Internet. To FTP your files, you need an FTP program. One popular program is WS_FTP (http://www.ipswitch.com) When uploading files to the web server, it is essential to upload in binary mode. Binary mode ensures that the file is not changed in any way during the upload process. The size (number of bytes) of all files should be the same on the web server as they are on your local PC.

Setting Permissions
Certain permissions must be set for SSI Web to work properly on your web server. Perl needs to be installed on the server and the Perl scripts (ciwweb.pl and admin.pl) need permission to execute. Other permissions also need to be set so that these scripts are able to read, write, and delete files in the admin and data directories. Unix file permissions can be set through an FTP program. In WS_FTP, right-click the file, select FTP Commands, and then select CHMOD.

230

SSI Web v7

Security
If you choose to host your own SSI Web surveys, you are responsible for securing your web server so that unauthorized people cannot access your data. Here are a few items to consider: Make sure that *.cgi files (including password and data files) are not visible to the world. The server needs to be configured so that it will not permit a browser to display the contents of *.cgi files. One way to do this is to configure your server such that *.cgi files are treated as cgiscripts. Make sure that you cannot use a browser to open the file containing your Admin Module passwords (STUDYNAMEpacc.cgi). For example, you should not be able to view the contents of the STUDYNAMEpacc.cgi file if you paste an address similar to the one below (using the URL specific to your study) into your browser's address bar: http://www.yoursite.com/yourstudy/admin/STUDYNAMEpacc.cgi If attempting to access the *.cgi file causes an error to be posted to the screen, then this lets you know that the *.cgi files are configured correctly. The server needs to be configured so that server directory listings are not visible to the world. Make sure that if you paste the URL to your "admin" folder into your browser's address bar, you are not able to see any file names. For example (using the URL specific to your study): http://www.yoursite.com/yourstudy/admin/

Files and Folders for Upload

The table below shows all files and folders necessary for SSI Web to run (SSI Web automatically assembles these for you in the Web Upload folder on your hard drive when you click Field | Prepare for Manual Upload). The required Unix permissions are also shown. "STUDYNAME" is used to represent the name of your study.

General Web Interviewing (CiW) Help

231

File / Directory
STUDYNAMElogin.html

Description
Login page. Entry point for respondents. Admin Module login page. Entry point for study administrators. Folder or directory. Main web questionnaire file. Contains server path information. Contains Admin Module passwords. Contains passwords and merged data info. File doesn't exist if no passwords implemented. Design file for CBC studies. Design file for CVA studies. Design file for MaxDiff studies. Folder or directory that contains all of the Perl files.

Unix Permissions
(* See notation help)
644 r _ _ 644 r _ _ 703 _ w x 644 r _ _ 644 r _ _ 644 r _ _ 644 r _ _ r w _ | r _ _ |

STUDYNAMEadmin.html /admin STUDYNAMEqst.cgi STUDYNAMEpath.cgi STUDYNAMEpacc.cgi STUDYNAMEpwds.cgi

r w _ | r _ _ | r w x | _ _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ |

**STUDYNAME_CBC [exerciseID]_ [designID].cgi ***STUDYNAME_CVA [exerciseID]_ [designID].cgi ****STUDYNAME_MXD [exerciseID]_ [designID].cgi /cgi-bin

644 r _ _ 644 r _ _ 644 r _ _ 101 _ _ x 755 r _ x 755 r _ x 644 r _ _ 644 r _ _ 644 r _ _ 644 r _ _ 644 r _ _ 644 r _ _ 644 r _ _ 644 r _ _ 644 r _ _ 644 r _ _ 707 r w x 707 r w x 644 r _ _ 644 r _ _ 644 r _ _

r w _ | r _ _ |

r w _ | r _ _ |

r w _ | r _ _ |

_ _ x | _ _ _ |

ciwweb.pl admin.pl ciwlibX_X_X.pl authlibX_X_X.pl pverlibX_X_X.pl grdlibX_X_X.pl enterlibX_X_X.pl acalibX_X_X.pl cbclibX_X_X.pl cvalibX_X_X.pl acbclibX_X_X.pl maxdifflibX_X_X.pl /graphics /system ssi_javascriptX_X_X.js ssi_styleX_X_X.css ssi_admin_styleX_X_X.css

r w x | r _ x | r w x | r _ x | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ | r w x | _ _ _ | r w x | _ _ _ | r w _ | r _ _ | r w _ | r _ _ | r w _ | r _ _ |

Folder or directory that contains all the graphics in your study. Folder containing supporting files. SSI Web Javascript library Base style sheet Style sheet for Admin Module

232

SSI Web v7 * Unix Permission Notation - The notation consists of three sets of the letters r, w, and x. The letter "r" stands for "read," "w" for "Write," and "x" for "Execute." The first set of letters is for the Unix group "Owner," the second for "Group," and the third for "Other." Full permissions r w x | r w x | r w x or 777 give "read," "write," and "execute" permissions to "Owner," "Group," and "Other." ** Only for CBC studies *** Only for CVA studies **** Only for MaxDiff studies After uploading these files and folders, and setting correct permissions, your survey should be working. Go to STUDYNAMElogin.html on your web site and test your survey, making sure the study behaves as expected and that it is saving data properly. Remember to log into the Administrative Module through STUDYNAMEadmin.html to see your results and download your data. If you are receiving errors, see Troubleshooting - Common Mistakes

Alternate Configuration - Common cgi-bin for all Studies

The default configuration of SSI Web has a separate cgi-bin directory inside each study root directory. Some web server configurations and ISPs do not allow this. They have a specific location where all CGI scripts must reside in order to execute. The following steps outline how to make SSI Web work with this alternate configuration: Upload all of the SSI Web Perl files (*.pl) to the location where they have permission to execute (the "common cgi-bin"). In the authoring interface of SSI Web, choose Compose | Survey Settings + Server Settings tab. Modify the default paths to account for the new location of the Perl scripts. The following list describes the Server Directory Paths that need to be modified: Administrative Directory: Path from the Perl scripts to the admin directory. CGI Scripts Directory: Path from STUDYNAMElogin.html to the Perl scripts. Data Directory: Path from the Perl scripts to the data directory. Relative Path from CGI Scripts to Graphics: Path from the Perl scripts to the graphics directory. Relative Path from login.html to Graphics: Path from STUDYNAMElogin.html to the graphics directory.

These changes are stored in the STUDYNAMEpath.cgi, STUDYNAMElogin.html, and STUDYNAMEadmin.html files. The notation "../" means go up one directory. For example, given the directory structure:
\cgi-bin ciwweb.pl admin.pl (etc.) \htdocs \study1 \admin \data \graphics STUDYNAMElogin.html STUDYNAMEadmin.html

the alternate paths are: Administrative Directory: ../htdocs/study1/admin/

General Web Interviewing (CiW) Help CGI Scripts Directory: ../../cgi-bin/ Data Directory: ../htdocs/study1/data/ Relative Path from CGI Scripts to Graphics: ../study1/graphics/ Relative Path from login.html to Graphics: graphics/

233

Remove STUDYNAMEpath.cgi from the admin directory. Upload the STUDYNAMEpath.cgi with the modified path information to the "common cgi-bin" directory next to the Perl scripts.

234

SSI Web v7

1.7.3

Uploading Survey to Microsoft IIS Servers

SSI Web surveys may be hosted on Microsoft IIS Servers. The full SSI Web system does not need to be installed on the server. You only need to copy the folder to your server that includes the files needed to run the survey and collect the data. This folder is created using SSI Web when you click Field | Upload Study Files | Create Web Upload Folder for Manual Upload. SSI Web uses scripts written in the Perl language to run web-based surveys. A Perl interpreter must be installed on your web server in order for surveys to work correctly. Active State provides a free version of their ActivePerl interpreter at http://www.activestate.com/Products/Download/Download.plex?id=ActivePerl. See detailed explanation regarding server setup for hosting SSI Web surveys, by clicking the following links: IIS 7.0 (Windows Server 2008) IIS 6.0 (Windows Server 2003) IIS 5.0 (Windows 2000 Servers) IIS 4.0 (Windows NT 4.0 Server) Or visit http://www.sawtoothsoftware.com/support/issues/ssiweb/server_setup to view the more detailed documentation. Note: Many of the files uploaded to your server use a .cgi extension (such as your data file and the file containing administrative passwords), which generally provides an additional level of security. You should verify that .cgi files are configured to be cgi scripts on the server. This prevents others from opening these .cgi files and viewing their contents. You can verify that this is set correctly by pasting the URL to a .cgi file (such as your STUDYNAMEpacc.cgi) directly into your browser. If you can view or download the contents of the .cgi file, this indicates that others may do so as well, and sensitive information (e.g. your data and admin passwords) is being exposed. If attempting to access the .cgi file causes an error to be posted to the screen, then this lets you know that the cgi files are configured correctly.

General Web Interviewing (CiW) Help

235

1.7.4

Deleting Practice Data Prior to Taking the Study "Live"

You should collect some practice data and check that everything is turning out as you expect prior to taking a study "live." If you use Test Survey Mode, you do not need to worry about deleting practice data, as test data are kept separate from actual study data. However, if you still want to test the actual study (in real data collection mode), then we provide the following information. After collecting the practice data, you will probably want to delete these records prior to starting actual data collection. There are two methods for deleting the practice respondent data and resetting the survey: 1) Using the Admin Module Access the Admin Module by connecting to the Web and browsing to you STUDYNAMEadmin.html page. Use the Administrative Read-Modify username and password specified under Compose | Survey Settings + Server Settings tab. Once at the main menu, choose Reset Web Survey and click Continue. (Warning: this option deletes all survey data. While that may be fine for practice data, you must be very careful with this option not to delete real data in the future.) 2) Using the View Data function in the Administrative Module. To delete selected practice data, go to View Data in the Administrative Module, and delete the desired records. You might, for example, assign a username of "test" for test records. You could then search for only those records that had "test" as a username and delete them.

236

SSI Web v7

1.7.5

Troubleshooting Server-Side Problems

Below is a list of suggestions you might find helpful in troubleshooting web problems: 1. Make sure Perl is working. Use our Test1.pl. If it doesn't work, contact your system administrator. 2. Make sure that the path on the first line of the script (ciwweb.pl and admin.pl) is the correct path to your Perl executable. We have set the path at the most common directory for the Perl interpreter [sometimes called the Perl executable] which is: /usr/bin/perl. Another common location is /usr/local/bin/perl. Consult the Web server administrator if you don't know the correct path.) 3. When uploading your files, make sure you upload them in "binary" mode so that the number of bytes in the files do not differ between your machine and the server. 4. Unix servers are case sensitive when it comes to filenames. Make sure that the case on your filenames matches. We'd suggest using all lower case names. It is a good idea to make your study names all lower case. 5. Make sure you set the permissions correctly on the files (see our documentation for details). The admin and data directories need "read," "write," and "delete" permissions so we can store data, etc.. When troubleshooting, you can enable full permissions. Once it is working, you can tighten your permissions. 6. Don't use Microsoft's Front Page to upload. It often inserts extra HTML instructions that can conflict with the HTML instructions SSI Web automatically creates. 7. Some ISPs or web servers require a special subdirectory where Perl scripts can run. Find out from your ISP what their rules are for running Perl scripts. Also, some servers provide added protection for files with an extension of .cgi. This is why many of our files have this extension even though not all of them are CGI scripts. If you use a directory other than cgi-bin, make sure you update this information in your server directory paths under the Server Settings tab of the Survey Settings dialog. 8. Some servers, including many NT servers, require alternate setups that are different from our default procedure. In these cases, you might need to change the paths in Server Settings. Under the Compose | Survey Settings, Server Settings tab, you can modify the system paths. When changing the paths under Server Settings, the five paths are as follows: Administrative Directory: CGI Scripts Directory: Data Directory: Relative Path from CGI Scripts to Graphics: Relative Path from login.html to Graphics: Path from where the Perl files are running to the /admin directory Path from STUDYNAMElogin.html and STUDYNAMEadmin.html to the directory that contains ciwweb.pl and admin.pl Path from where the Perl files are running to the /data directory Path from where Perl scripts are running to the /graphics directory Path from STUDYNAMElogin.html to the /graphics directory

9. If you have changed the default directory structure, you may need to move the file STUDYNAMEpath.cgi to the same directory as the Perl files so the scripts can find it. Here are the search paths for STUDYNAMEpath.cgi from the directory from which the Perl scripts are running: ../admin/

General Web Interviewing (CiW) Help If not found in ../admin/, then it looks for the STUDYNAMEpath.cgi in: current directory (where Perl scripts are running) If not found there, it looks for: current directory/studyname/admin If not found there, it looks for the STUDYNAMEpath.cgi in: ../studyname/admin/

237

10. If you want to delete all the respondent data up on the web that is associated with a study, go to the admin.html and choose Reset Survey. SSI Web Perl test scripts If you are experiencing difficulty in getting your questionnaire to run on the web, it may be helpful to troubleshoot the problem by using very simple Perl scripts. You can find these scripts in your \Resources directory in your SSI Web installation. Once these 2 test scripts run successfully, your SSI Web survey will also have a very good chance of running successfully. Upload the test script to your cgi-bin and test it by opening up a browser and going to http://www.yourfile.com/cgi-bin/test1.pl. TEST1.PL - simply tests Perl linkage. A debug test message is written ("Debug Test #1 Successful") if Perl is installed correctly and permissions are set correctly. Failure here would mean that the Perl interpreter is not being called and executed. You can check for the correct path to the interpreter on the first line of the script and that Perl has been installed on the machine. You can also check the permissions on the script to make sure it has "executable" permission. If you are still having problems, call your ISP and explain that you need help running a simple "hello world" Perl script. Test2.pl - Helps you determine where your current working directory for Perl is.

238

SSI Web v7

1.8
1.8.1

How Respondents Access the Web Survey

How Respondents Access the Web Survey
Once you've uploaded your survey to your Internet server and verified that it is working properly, you need to decide how to invite respondents to complete it. The most common way to invite respondents is via email, with an embedded hyperlink. Other methods include linking from another survey or website, or directly typing an address within the browser. Respondents can open the Start page by browsing to (or clicking a link to): http://www.yourwebserver.com/study1/mystudylogin.html where www.yourwebserver.com is the URL for your server study1 is the folder containing your project (your "Remote Path") mystudy is the name of your study Getting to your survey: 1. Email URL links. Most every Email package is "Web-enabled," meaning that it recognizes URL addresses such as http://www.sawtoothsoftware.com as Internet sites. Whenever the Email package encounters an Internet address in the course of a message, it changes the text string to a hyperlink (underlines the text, changes its color, and makes it a clickable area). When Email recipients click hyperlinks, their computer automatically connects to the Web and their browser points them to your survey site referenced in the hyperlink. You can also include the password information (or other "pass through information") in the hyperlink for one-click access to the survey with password verification. 2. Link from Website or Other Survey. Respondents browsing a site on the Internet or taking another survey click a submit button or hyperlink (either text-based or clickable graphic) that connects them to your survey. You can also include the password information or other "pass through information" in the hyperlink. This provides one-click access to the survey with password verification. 3. Typed URL. Respondents connect to the Internet and type the URL into their browser (or cut and paste it from an Email message) to access your survey (the STUDYNAMElogin.html page).

General Web Interviewing (CiW) Help

239

1.8.2

"One-Click" Respondent Access to Survey

Your survey is just "one click" away for respondents if you email them a hyperlink. The hyperlink includes the address for your survey, and optionally "passes along" information about passwords or other identifying (passed-in) information. Most email programs automatically interpret http:// as a hyperlink. We suggest you use one of the many bulk* email packages available on the market. These permit you to automatically process hundreds or thousands of email messages at a time. When the respondent receives the email, they do not have access to the email addresses of other respondents; they only have the return path email address to your account that sent the message. An automatic bulk email package can also import customized text strings from a database into the email message. Those text strings might include the respondent's user name and/or password generated by SSI Web's Password Program/Generator. (*Check applicable laws and regulations dealing with bulk email.)

Simple URL Hyperlink

If you type the following into an email message, it becomes a clickable "hyperlink" (underlined clickable text string, that sends respondents automatically to a web page): http://www.sawtoothsoftware.com/gonzo/STUDYNAMElogin.html Note that the "study root directory" on the server is called "gonzo" and the link sends the respondent to the opening login (password) page, called "STUDYNAMElogin.html". However, the respondent is required to see the login page (page 1) and type a password to continue the survey.

URL Hyperlink with Passwords

If you want respondents to simply click a link that automatically logs them into a survey and supplies their passwords and/or additional "passed-in" data (that you've defined in the passwords table), you should create a URL that links respondents directly to the Perl script located in the cgi-bin directory. When respondents click such a link to start the survey, they skip the Start page (page 1). The following text included in an email message starts a survey with a studyname "tep1" located in the "gonzo" study root directory, with password fields named myusername and mypassword, with valid values of "michael" and "A6Z99": http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&myusername=michael&mypassword=A6Z99 You can also pass additional variables into the data set, as long as those are defined as Pass-in Fields within the Questionnaire Access and Passwords dialog. For example, if we define a Pass-in Field called panelid, we can specify this directly following any password information, as below: http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&myusername=michael&mypassword=A6Z99&panelid=7362836 When respondents start the survey by clicking the hyperlink, if the passwords are valid (and the quota is not complete), the respondent is sent to the page following the password screen. If the respondent is restarting an interview, the page where he/she left off is displayed. Any passed in information is stored in the data file, as if it were any other question in the survey.

240

SSI Web v7

sys_skipto
If you want to restart an SSI Web questionnaire and skip to ("link to") a specific question name, you can do so by including a "sys_skipto" variable within the URL hyperlink (note: only works for restarts). For example: http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&panelid=7362836&sys_skipto=Q11 The hyperlink above restarts the tep1 survey and skips respondents directly to Q11. The most common use for this functionality is for linking multiple surveys (such as multiple conjoint modules) while still maintaining quota control. For example, consider the following study design: Study1 (CiW-type survey) Study2 (CBC survey #1) Study3 (CBC survey #2) All respondents start in Study1 and complete Q1-Q10. After Q10, respondents receive either Study2 or Study3, depending on answers within Study1. After completing the CBC section, respondents return to Study1 to complete the survey (Q11-Q20) and be counted toward the quotas that are maintained in Study1. To do this, a skip pattern is inserted within Study1 that skips respondents from Q10 to one of two Terminate/Link questions placed in Study1. The Terminate/Link questions mark respondents as incomplete, and link them to either Study2 or Study3. Then, at the end of Study2 and Study3, Terminate/Link questions link respondents back to Study1, with sys_skipto=Q11 placed within the URL. Respondents finish Q11-Q20 and reach a Terminate/Link question in Study1 that marks respondents as complete. Note: if restarting a survey and skipping forward within the questionnaire, make sure not to set passthrough variables to new values. If a passthrough variable has already been stored and is later resubmitted as a different value, the original value is maintained.

Important Note:
If the text string for the URL and passwords becomes very long, your email package may automatically "wrap" the text in the middle of the hyperlink by inserting a hard return. This disables the hyperlink and it will no longer work. Some email packages by default wrap after 80 characters. Some email packages let you control how many characters are displayed before automatic wrapping occurs. For example, if using Microsoft's Outlook Express, select Tools | Options | Send and under the Mail Sending Format click the Settings... button. There is a field to specify "Automatically wrap text at ___? characters, when sending". Make sure to test that the hyperlink works properly by sending test messages to yourself and other individuals (preferably outside your company's own internal email system) prior to emailing invitations to respondents. Sending email in HTML format will decrease the chance of links wrapping incorrectly and becoming broken.

General Web Interviewing (CiW) Help

241

1.8.3

Linking to Sawtooth Software's Web Survey from Another Survey Program

Some users may find the need to link to Sawtooth Software's web survey software from another HTML form or Web-based survey. One of the most critical aspects is that you pass through an identifier that can be saved as the user name or password with the Sawtooth Software data set. That way, respondents from the two separate data sets can be matched during data processing. Note: Linking multiple SSI Web surveys that result in more CiW questions than your license conveys is a violation of your license agreement.

URL Hyperlink to Pass Respondents Directly in SSI Web Survey

If you want to automatically pass respondents from another Web survey page to an SSI Web survey and automatically "pass through" the user name and/or password, you should link respondents directly to the SSI Web Perl script located in the cgi-bin directory. The following text included in an email message starts a survey with a studyname "tep1" located in the "gonzo" study root directory, with a valid username of "michael" and a password of "A6Z99": http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&myusername=michael&mypassword=A6Z99 In this next example, only a username is used: http://www.sawtoothsoftware.com/gonzo/cgi-bin/ciwweb.pl?studyname=tep1&myusername=michael You can also pass additional variables into the data set, as long as those are defined as Pass-in Fields in the Questionnaire Access and Passwords dialog: http://www.sawtoothsoftware.com/gonzo/cgibin/ciwweb.pl?studyname=tep1&mypassword=A6Z99&Q1=1&Q2=5 In this example tep1 is the study name, and there is just one password named mypassword, with A6Z99 as a valid value for mypassword. Q1 and Q2 are additional variables to be passed into the data set, and these variables were previously defined in the Pass-in Fields tab of the Questionnaire Access and Passwords dialog. When respondents start the survey by clicking the hyperlink, if the passwords are valid (and the quota is not complete), the respondent is sent to the page following the Start page (page 1). If the respondent is restarting an interview, the page where he/she left off is displayed. Warning: If you are "passing in" a variable to an ACA, CVA, or CBC study that you plan to use as a case identifier for later merging the conjoint data with other survey variables, you should only use numeric values. SSI Web will not permit the creation of conjoint data files for summarizing or computing utilities that use non-numeric case identifiers.

Form Submit Button

You can create a "submit button" that sends respondents to the address of a Sawtooth Software survey and "passes through" a variable to be saved as the user name and/or password. Note that your survey program must be able to customize the HTML text on the fly with respect to the values of the variables passed through.

242

SSI Web v7 Here's some sample HTML: <HTML> <BODY> Click the button below to continue survey. <FORM METHOD="GET" ACTION = "http://www.website.com/survey/cgi-bin/ciwweb.pl"> <INPUT TYPE="HIDDEN" NAME="myusername" value="michael"> <INPUT TYPE="HIDDEN" NAME="mypassword" value="A6Z99"> <INPUT TYPE="HIDDEN" NAME="studyname" value="tep1"> <INPUT TYPE="SUBMIT" VALUE="Continue"> </FORM> </BODY> </HTML>

Linking from SSI Web to Another System

You can link from SSI Web to another system by specifying the Terminate/Link question type. You can dynamically insert respondent identifiers (such as user name and password) or even responses to questions within the SSI Web survey within the HTML of that hyperlink. For more information about dynamically inserting user name, password, or answers to previous questions within the hyperlink, please see the section on Displaying Responses to Previous Questions.

General Web Interviewing (CiW) Help

243

1.9
1.9.1

Making Changes after Fielding Begins

Making Changes after Fielding Begins
Ideally, you will not have to make changes to your survey once you have begun interviewing (see warnings below). However, the reality is that you will often be asked to make changes. Most every change to your project involves modifying the project locally on your hard drive, and uploading updated files to the Web server. If you use the automatic upload capability (Field | Upload Study Files) SSI Web will only upload the files that have been affected by your local changes. Even though SSI Web makes it easy to upload changes, don't get lulled into thinking that any change you make to your project after fielding has begun is OK. The warnings listed below still apply, since it is possible for respondents currently taking the survey to be temporarily disrupted by your accessing and modifying files on the server. Also, more importantly, some changes you can make to your survey pose no problems for later analysis; but, some changes you might make can cause a great deal of trouble later. Please see the section below entitled Common Survey Changes. If you are using the manual upload method with your own FTP software, then you first prepare the project files for web upload (Field | Upload Study Files | Create Web Upload Folder for Manual Upload). In that case, we recommend uploading just a few selected files that have modified (but not typically the entire project).

Warnings
Making any change to the project files on your internet server will potentially disrupt respondents as they are taking the survey. If, for example, you upload a new questionnaire file (STUDYNAMEqst.cgi) to replace the existing questionnaire file on your server, the file will be locked for a short period of time (usually less than a second). If a respondent hits the Next button to submit responses on a survey page during the time when the STUDYNAMEqst.cgi file is locked, an error message will be displayed when the browser requests the next page (the data submitted from the previous page will not be lost). If, after seeing the server error page, the respondent clicks the refresh button or the back button on the browser, the survey will resume. Nonetheless, it is concerning that respondents can be potentially receive a server error. To reduce this likelihood, we suggest you make changes during a low traffic time. Before making any change, it is a good idea to download any data that have been collected to this point through the Admin Module. That way, if you accidentally do something to delete your data, you have a backup. Make sure not to delete accidentally the existing /Data directory on your server, or you will lose all your data. Also, do not delete any files or folders that have been created automatically by the software on the server.

Common Survey Changes

1. Changing Text Only: Sometimes, you've made a typographical error, or you need to modify the text in your survey. You can edit the survey locally on your hard drive to make the change. Click Field | Upload Study Files to make the change on the server. If using the manual method to upload files to the server, click Field | Upload Study Files | Create Web Upload Folder for Manual Upload to prepare new files for uploading to the server. These files are saved to a folder on your hard drive within your study folder called /Web Upload. For this change in the questionnaire, the *only file that needs to be uploaded to the server is the STUDYNAMEqst.cgi file (within the /Admin directory). Copy the STUDYNAMEqst.cgi file from the /Admin folder on your hard drive to the /Admin directory within

244

SSI Web v7 your study's server installation (overwriting the existing file). Test the survey (following the instructions below) to ensure that your survey has been updated properly. * If the text change involves the "Start" question (or other questions on page 1), then you will also need to upload a new STUDYNAMElogin.html file to the root study directory on the server. 2. Changing Skip Patterns (follow same instructions as Changing Text Only). 3. Adding a Question: When you add a question, any respondents who did not receive this question will be marked as "missing" for the new variable. Follow the same instructions as Changing Text Only. 4. Changing a Question Name: This is problematic, as the new survey layout will not contain the variable name for the early respondents who completed the questionnaire prior to the change. If you must make this change, you can put the original question on a new page with an unconditional skip. That way, you will be able to export the data from the original question name and the new question name separately. To upload the changes, follow the same instructions as Changing Text Only. 5. Changing a Question Type: Sometimes, users may need to change the question type from Text to Numeric. This poses a problem for data export, and you may not be able to export the data prior to making the change if you do this. Therefore, we suggest the same remedy as Changing a Question Name above. 6. Adding New Passwords: See the earlier section entitled: Modifying Passwords When in Field. 7. Changing Administrative Passwords (the passwords you use to access the Online Administrative Module): Open your project using SSI Web, locally on your hard drive. Click Compose | Survey Settings + Server Settings tab to access and modify your Administrative Access passwords. Click Field | Upload Study Files to make the change on the server. If using the manual method to upload files to the server, click Field | Upload Study Files | Create Web Upload Folder for Manual Upload to prepare new files for uploading to the server. These files are saved to a folder on your hard drive within your study folder called /Web Upload. For this change, the only file that needs to be uploaded to the server is the STUDYNAMEpacc.cgi file (within the /Admin directory). Copy the STUDYNAMEpacc.cgi file from the /Admin folder on your hard drive to the /Admin directory within your study's server installation (overwriting the existing file).

Testing Your Survey after the Change

The Admin Module includes a Test Survey capability that lets you enter a mirror image of your survey as a respondent (but your test data are not mixed with the real data). You should do this to ensure that the change you made has taken properly on the server installation, and that you haven't somehow disabled your study.

General Web Interviewing (CiW) Help

245

1.10
1.10.1

CAPI Interviewing with "SSI Web CAPI"

CAPI Interviewing with "SSI Web CAPI"
SSI Web generally is used for interviewing over the Internet. But, SSI Web studies can also be used for interviewing in "CAPI" mode (Computer Aided Personal Interview), which generally refers to data collection using a laptop or a PC not connected to the internet. We recommend Windows 2000 or later for the PC or laptop used for collecting data. You do not need to buy an additional module or software package other than the general SSI Web system to conduct CAPI interviewing (the CAPI module can be downloaded at no charge from our web site). The main steps are as follows: 1. Develop your study as you would for Web interviewing, including setting up passwords, etc. 2. Test the study to ensure it works properly, including checking skip patterns, examining the resulting data, conducting a pre-test, etc. 3. Use Field | Create CAPI Packages to create a "CAPI package," which is a single file (.CAPI extension) that contains all the information necessary to run the study. If multiple PCs (not networked) are used in interviewing, you may want to create multiple .CAPI files, one for each interviewing PC, for better field data management. 4. Install the "SSI Web CAPI" software on laptops or PCs used for interviewing. This is freely downloaded software from our website. Or, you can burn this installation program onto a CDROM to deliver to the data collection facility. (The CAPI installation program is also included in the "CAPI" subdirectory of the SSI Web program.) 5. Email or otherwise deliver the .CAPI file(s) to the data collection facility. They load the study into SSI Web CAPI and begin interviewing. They can give you periodic reports regarding the number of complete, incomplete and disqualified records. 6. The data collection facility sends you a data file for each .CAPI file that was loaded and used for interviewing, which represents all data records collected to that point. 7. Place each data file within a folder on your computer. Select Field | Accumulate CAPI Data File(s) and select the folder that contains the data files. That folder is searched for .dat files with the format STUDYNAME_#.dat, where # is a number.

Create CAPI Package Files Dialog

Each .CAPI file you create contains all the files and information needed to run the study (the questionnaire, passwords, graphics, Perl scripts, etc.) By including the Perl scripts in each .CAPI file, this ensures that each project is fully self-contained, and the data collection site can manage multiple studies without regard to which version of SSI Web was used to create the studies. Beginning CAPI Package Index Number Creating multiple .CAPI packages and indexing them is a useful aid for field data management, however, there isn't any requirement that you create multiple packages. In reality, multiple .CAPI files from the same study are identical in every way (assuming you didn't change anything else about your study between .CAPI file creation sessions, such as to add new passwords or modify the questionnaire). Select a beginning number for the .CAPI package(s) to be created in the current session. If you select "1" as the starting point (with multiple packages to create), the .CAPI packages are named starting with STUDYNAME_1.CAPI. The next .CAPI packages in the series follow as

246

SSI Web v7 STUDYNAME_2.CAPI, STUDYNAME_3.CAPI, etc. You can create .CAPI packages in multiple sessions. For example, you may discover after a few days of interviewing that you want to create a few additional .CAPI packages. If you earlier had created .CAPI packages 1-10, you might now create .CAPI packages 11-20. (If for some reason you wanted to start naming the files in the second session starting with #83, it wouldn't matter. Again, the numbers attached to files are merely to help you organize and manage files coming from potentially multiple sources in the field.) Some SSI Web users that are familiar with our other interviewing systems (such as Ci3 or SMRT) may wonder how the beginning respondent numbers are specified for each .CAPI file, or how many respondents are allowed per .CAPI installation. Internally, we use a random seed based on the system clock to assign each record an internal respondent number (this is used as a seed for question/item randomizations and assignment to CBC designs). If you interview many respondents, it is possible to get duplicates (there are 475,254 possible unique internal respondent IDs). However, we generally recommend that you use a different variable (such as User Name or Passwords) as respondent IDs, or that you simply export the data with the cases numbered sequentially from 1 through n. We don't suggest you use SSI Web's internal respondent number in analysis. Each .CAPI installation can collect up to 475,254 possible respondents. Number of CAPI Package Files to Create Select the number of .CAPI files to create within the current session. If the PCs or laptops used for interviewing operate independently, then you may want to create one file per interviewing PC, for ease of field data management. (If you and your field agency would not be bothered by different batches of respondents having the same file name, then you are welcome to create a single .CAPI package and distribute this same file to all data collection sites and PCs.) The name of the .CAPI files follows directly from your study name. If the study is named ABCD and you create 10 CAPI packages, starting with an index number of 1, the files are named ABCD_1.CAPI, ABCD_2.CAPI . . . ABCD_10.CAPI. Warning: Only one .CAPI file with the same study name should be used by each computer. If (using the SSI Web CAPI program) the interviewer loads a second .CAPI file (say XXXX_2.CAPI) onto a PC that has already been interviewing using a XXXX_1.CAPI installation, all the data files for the XXXX_1.CAPI installation will be overwritten (though we always store vital data in a sub-directory of the /Archive folder within a SSI Web CAPI installation prior to deleting any data). We provide a warning dialog within the CAPI module prior to any step that would delete the data. Directory to Create CAPI Package Files: This indicates the folder in which to save the .CAPI files, and is located inside your study folder. You cannot change this folder. Include Administrative Module If you want the individual(s) conducting data collection to have access to SSI Web's Online Administrative Module, check this. We generally suggest not including the module, unless the individual(s) performing data collection have a good understanding of the functionality of the Administrative Module. Even without including the Administrative Module, the SSI Web CAPI program lets the data collection facility accumulate data for each .CAPI file into a single data file, and gives them a tally of the number of completes, incompletes, and disqualified records in each data file.

General Web Interviewing (CiW) Help

247

Interview "Recycle"
When running in CAPI mode, you may want the questionnaire to automatically recycle from the last page of the survey back to the first, so that it doesn't drop to the main SSI Web CAPI menu each time. The easiest way to do this is to add a Terminate/Link question at the end of your survey (on its own page), that (on the Settings tab) specifies a Web Address (URL) of "../XXXXlogin.html" where "XXXX" is the name of your study.

248

SSI Web v7

1.10.2

Accumulating CAPI Files

This dialog lets you select the folder that contains the .dat files that were delivered to you (for example, from the data collection facility). Any and all .dat files are accumulated into your project from that folder and all its sub-folders. For example, let's assume your main SSI Web study (studyname: 0893) is located in a folder named Auto0893. Further let's assume you have created three CAPI packages for interviewing: 0893_1.CAPI, 0893_2.CAPI, and 0893_3.CAPI. The data collection site has interviewed using three PCs and delivered three separate .dat files to you via email: 0893_1.dat, 0893_2.dat, and 0893_3.dat. You place these three files in a folder (perhaps a sub-folder from your main Auto0893 folder) called Auto0893 CAPI Data: Auto0893 (Contains SSI Web study files, including your final 0893dat.dat file, which is the one used in all export and data analysis functions in SSI Web.) Auto0893 CAPI Data 0893_1.dat 0893_2.dat 0893_3.dat During the Accumulate CAPI Data Files process, you'll select the Auto0893 CAPI Data folder, and all data files in that folder named 0893_?.dat will be accumulated and either append to or overwrite (based on your selection when prompted) the 0893dat.dat file in your Auto0893 folder. Please note that the data files delivered to you from Web CAPI interviewing are cumulative data files, including all interviews conducted to date. Therefore, we suggest when you accumulate CAPI data files, that you accumulate all data over all CAPI files, overwriting any data in the current project file.

General Web Interviewing (CiW) Help

249

1.10.3

Deleting Sensitive Data from CAPI Interviewing PCs

We have designed SSI Web CAPI to make it very difficult to lose important data. Prior to any step in which data might be deleted, we warn the interviewer that important data are about to be deleted. Even if the interviewer chooses to perform an operation within the SSI Web CAPI interface that deletes data, we still save a copy of vital data files within a sub-directory of the /Archive folder within the SSI Web CAPI installation. This keeps interviewers from accidentally deleting critical data. However, this also makes it more difficult to clean sensitive data completely from a PC at the end of a project (e.g. prior to returning rental PCs). Even if you uninstall the SSI Web CAPI program, data files created during project administration are still maintained on the hard drive. To completely delete study data from an interviewing PC (running under SSI Web CAPI): 1. From the SSI Web CAPI menu, highlight the study to be deleted on the list of Available Studies. Click Delete Study button. This deletes the data for the project from the project folder. However, for safety precautions, we place critical data files in a sub-folder of the /Archive directory as a result of this deletion step. 2. Browse to the ..\Documents and Settings\YourUserName\Application Data\Sawtooth Software, Inc\SSI Web CAPI Module\Archive directory. You may find sub-folders within that directory with the study name, plus a time-stamp reflecting the exact time and date the data were archived. Delete any of these folders containing sensitive data. 3. Empty the Recycle Bin. (Even though you deleted the folders in Step 2, Windows retains these in the Recycle Bin until you empty it.)

250

SSI Web v7

1.11
1.11.1

Managing Survey Data

How Data Are Stored
When respondents generate records (either complete or incomplete), each is saved as a separate file in the /data folder on your server. Each data file starts with the four-character Study Name followed by an alpha-numeric code. For example, the first three respondent records for a study with a study name of soda might be named: soda1.cgi soda2.cgi soda3.cgi The data files are in comma-delimited text-only format. The data files include all information provided by the respondent. When you use the Admin Module to Download the data, any completed records are moved from the /data directory to the master data file (STUDYNAMEdat.cgi) in the /admin folder on your server (again comma-delimited text-only format). Incomplete records always remain in the /data directory.

Backing Up/Skipping Backward and "Limbo" Data

If respondents back up in a survey (or are skipped backward) to re-answer a previous question, answers to the later questions (which are now only conditionally valid) are thrown into "limbo" status and are not saved in the final data file (though they are available in the "limbo" folder). Respondents must re-answer those questions. Respondents can back up in a survey by either clicking the browser's back button or by being skipping back to a previous question. Example: As respondents answer questions to an SSI Web survey, the answers are recorded in the data file like this: Q1, 5 Q2, 3 Q3, 2 Q4, 13 Here is an example of backing up and re-answering a previous question: Q1, 5 Q2, 3 Q3, 2 Q4, 13 Q2, 7

(The respondent backs up and re-answers Q2)

The data file is then cleaned and old data go into the limbo folder. Now the data record looks like this: Q1, 5 Q2, 7 and the respondent needs to answer Q3 and Q4 again.

General Web Interviewing (CiW) Help

251

1.11.2

Admin Module
SSI Web includes an Admin Module that makes it easy to manage your survey in the field, including: Testing your survey prior to fielding Tracking completes and incompletes Viewing/editing individual records Downloading lists of passwords that are incomplete/complete Downloading data at any point during fielding Viewing real-time tabulations of the data Shutting down the survey

You can access the Admin Module from any computer connected to the Internet, and it is protected by passwords that you set when you develop your project in SSI Web.

Accessing Admin Module from Any Web Connection Survey authors or clients should be pointed to the STUDYNAMEadmin.html file, which is the login page for the Admin Module. User name and passwords for the online administrative module are set in the Compose | Survey Settings, Server Settings tab menu.

Launching Admin Module from SSI Web You can avoid having to manually browse to the Admin Module and log in by directly launching the Admin Module using Field | Admin Module. SSI Web automatically logs you in using your administrative passwords that are stored in your project settings.

When you log into the Admin Module using either two methods above, the main dialog is shown:

Note: Using the Admin Module requires a browser with Javascript enabled.

252 1.11.2.1

SSI Web v7

Setting Passwords for Administrative Access

You should assign both a User Name and Password to limit access to the Admin Module. Two levels of access are available within the Admin Module: Read Only and Read-Modify. You may want to give only Read Only access to your client or other individuals who need to monitor the progress of the study, while retaining Read-Modify access for yourself.

To Specify Administrative User Name and Passwords

1. Click Compose | Survey Settings + Server Settings tab The Server Settings dialog is displayed. 2. Type the User Name and Passwords into the spaces provided. Default random user names and passwords are provided for your convenience (new random names and passwords are created for each new study), but you can change these if you like. Up to 12 letters or numbers may be used for each field. The user name and passwords are case sensitive. For example, if the User Name is AdminJudy, the administrator cannot type adminjudy or ADMINJUDY to access the data online.

Read Only vs. Read/Write Administrative Rights

Read Only administrative rights let the user... View the Study Summary View Survey Data Run Marginals View the Passwords Reports View the Incompletes Report Test Survey

Read/Write administrative rights permit all the above operations, plus... Download Data Edit Survey Data Delete Selected Records Close a Survey Reset a Survey

(There is an advanced capability for further customizing the menu items that are displayed for Read Only or Read/Write rights, documented later in this chapter.)

General Web Interviewing (CiW) Help 1.11.2.2

253

Logging On
You or your client can access the Admin Module by directing your browser to the STUDYNAMEadmin.html file located in your study's root directory (you can rename the STUDYNAMEadmin.html file if you'd like before uploading it to the server). If your study has many concurrent respondents, the more computationally intensive aspects of the Admin Module (such as Marginals) can place significant load on the server, which could delay performance for respondents currently taking surveys. These operations are more safely done when you expect relatively "light" traffic. When you access the STUDYNAMEadmin.html page with your browser, you are asked to supply a Name and Password. When you initially created your study, random six-character passwords were created for administrative access. You may have kept those randomly-generated passwords or changed those passwords for your needs under Compose | Survey Settings + Server Settings tab. If you cannot remember your password, you can go to Compose | Survey Settings + Server Settings tab to review the passwords. You must type the correct name and password to access the administration module. The User Name and Password are case-sensitive. (If the User Name is AdminJudy, then adminjudy and ADMINJUDY are not valid.) Type the Name and Password, and click the Login button.

Main Menu (Read/Modify Access)

After the system verifies a correct Name and Password (for Read/Modify access), the main menu is displayed, with the following options:

By default, the number of complete, incomplete, and disqualified records are shown. The options listed on the left are clickable links leading to additional menus.

254

SSI Web v7

Main Menu (Read Only Access)

After the system verifies a correct Name and Password (for Read Only access), the main menu is displayed, with the following options:

The options listed on the left are clickable links leading to additional menus.

General Web Interviewing (CiW) Help 1.11.2.3

255

Downloading Survey Data

We recommend you download your data often while a survey is fielding (at low traffic times, as these operations take up computing power and respondents taking the survey could experience a decrease in performance). Downloading the data moves any completed records from the /data directory into the /admin directory and accumulates it into a single data file. Also, it is good practice to download data and back it up on your local hard drive periodically. You can also download data using Field | Download Data from within the SSI Web interface. Specify the URL for your studynameadmin.html file, and provide settings regarding which data to download (All Data, Finished Surveys Only). Each time you download data, all data are downloaded from the server to your study directory on your hard drive. By default, SSI Web backs up your previous data stored locally, but you can change this behavior by de-selecting Backup Local Data. Downloading data makes a copy of the data file in the destination directory you specify; it doesn't move/delete the data from your server. In other words, whenever you download data, you are downloading a copy of all records every time (a cumulative process). For example, if on day one, 100 respondents complete the survey and you download the data, then on day two 75 more respondents complete the survey and you again download the data, 175 records will be downloaded on day two. Click the Download Data link from the main menu and the following is displayed:

When you choose Download Finished Surveys Only, your browser asks you to supply a destination drive and directory. In most cases, you will probably specify the study directory on your hard drive. The accumulated data file, named STUDYNAMEdat.dat, is copied to your hard drive.

If you Download All Data, both completed and incomplete records are copied into a file that is downloaded (downloading data never removes/deletes data from the server). Downloading incompletes is useful if you want to analyze why they are incomplete.

256 1.11.2.4

SSI Web v7

Viewing Survey Data

When you click View Data from the main menu, you are next given the opportunity to View All Data Records or to Search for Data Records. View All Data Records If you choose to View All Data Records, the View Data dialog is displayed. Select the record(s) on the left to view, and click View Selected Records. To select multiple records, hold down the Shift or Ctrl keys while clicking on record numbers.

You can also delete selected records (if you have logged on with Read/Modify access). If you delete a record, you can indicate whether to allow the deleted respondent to restart or not. Search for Data Records If you choose to Search for Data Records, the View Data - Search dialog is displayed:

General Web Interviewing (CiW) Help

257

This dialog allows you to search for specific records by different variables, or combinations of variables. For example, you could search for all records where Q1=5 (Variable Name "Q1", Operator "=", Value "5"). After the search is complete, the records that qualify on the search criteria are displayed in the View Data dialog described above.

258 1.11.2.5

SSI Web v7

Marginals: Tabulating Data On-Line

The marginals program lets you run simple frequency tabulations of the data in real-time by browsing to and logging into the Online Administrative Module. You can analyze complete and/or incomplete records.

Select the Question Respondent Status

This combo box lets you select which question to tabulate. You can only tabulate results for one question at a time. Here you specify whether you want to tabulate results for All Respondents, Qualified/Completes Only, Incomplete, or Disqualified.

Add additional filters You can specify filters, so results are tabulated only for a subset of respondents. Click Show Report to run the marginal report. You can print the results of any marginals run by clicking the Print icon in your browser, or choosing File | Print.

Marginals for Numeric/Single Select Questions Marginals reports for Numeric and Single select questions are quite similar, since each question contains a single response. Here is an example for the question gender, for just four respondents:

General Web Interviewing (CiW) Help

259

The Value column shows the values that were recorded in the data. The Label column displays the label used for each response category (if applicable). Count shows how many respondents used each unique value. Percent shows a bar graph with percentage of respondents (who answered the question) that used each value. Finally, the N (total number of respondents that answered the question), Min (minimum value used), Max (maximum value used), Mean (average of values used), and Missing (number of respondents who saw the question and submitted the corresponding page, but did not provide a response to this question.) Note that missing data do not factor into the N, Min, Max, and Mean computations.

Marginals for Multiple Select Questions Consider the following question:

260

SSI Web v7 The marginals report for this question may look like:

With select-type questions, a Label column appears to show the text associated with each value for this question. The Count column shows that a total of 3 + 5 + 4 + 5 = 17 responses were provided by the 10 respondents who received this question. The percents are percents of respondents (30% of respondents said they use their laptops at home) rather than percents based on total responses given. As a result, the percents will not usually sum to 100%. Please note that all respondents who saw this question and submitted the corresponding page are counted in the base for multiple response questions, even if they did not check any of the boxes. Therefore, no Missing category is reported.

Notes: Grid Questions: Multiple-response check-box questions within grids are available for tabulation. The variable for a row-based question is listed using the convention grid_r1_c* where the asterisk means that check boxes across all columns for the question in row1 will be tabulated. The convention for column-based questions is grid_r*_c1 where the asterisk means that check boxes across all rows for the question in column1 will be tabulated.

General Web Interviewing (CiW) Help 1.11.2.6

261

Downloading Password Report

You can download a list of passwords for respondents with different states of completion for the survey. This can be helpful if you want to send reminder notices to respondents who have not yet completed the survey, or if you want to identify those who have already completed the survey, etc. While in the Admin Module, from the main menu, click Password Reports. The following is displayed:

When you click any of these links, you are given a download link that saves a text-only file to your hard drive named STUDYNAMEPRT.txt containing the report.

262 1.11.2.7

SSI Web v7

Incompletes Report
This is sometimes referred to as a Drop-outs Report, though for a live survey we don't know for sure if all individuals reported have truly dropped out or are just incomplete and still in the process of completing the survey. For that reason, we call it the Incompletes Report. In the example below, 9 total respondents have started the survey (we learn this from the Study Summary report). 5 respondents completed the survey, and 4 are incompleted records. These incomplete respondents could have dropped out, or perhaps they are still working on the survey.

When we click the Incompletes Report link, the following report is shown:

This shows us the last question that the 4 incompleted respondents saw. 2 respondents last saw the Q1 question, 1 respondent last saw the sel1 question, and 1 respondent last saw the Q6 question.

General Web Interviewing (CiW) Help 1.11.2.8

263

Test Survey Mode

SSI Web includes the ability to test the survey using test survey mode. When in test survey mode, any test data are kept separate from real respondent data. You can run test surveys even during data collection, and it will not interfere with actual respondents. When in test survey mode, a bright orange banner indicating "TEST Mode" appears across the top of the survey to confirm that you are not collecting real data:

When you start a survey in Test Mode, a helpful dialog appears:

Check any functions that you want to enable during the test survey, and click the Next button. Notes: Quotas are not enforced in test mode. Also, you cannot restart incomplete test surveys. The error log file is shared between regular and test modes.

Admin Module (Test Mode)

When you have collected data in test survey mode, you are also given the full capabilities of the Admin Module again in test mode (with bright orange coloration in the dialogs so that you can distinguish working with test data from working with real survey data). When you click Test Survey from the main menu, a link is provided to Enter Test Mode. The following is displayed:

264

SSI Web v7

When in test survey mode, any operations you do within the test mode Admin Module are confined to the test data. The real data are unaffected. This includes all steps that delete data or reset the survey. Note: resetting real data will not reset your test data.

General Web Interviewing (CiW) Help 1.11.2.9

265

Reset Web Survey

This option resets your survey data. It deletes all respondent data and resets password files to reflect no completed surveys. Resetting your survey doesn't delete the questionnaire, just the respondent data. This is a convenient way to delete practice data prior to taking a study live. (Note: Test Data generated in Test Survey Mode is not deleted. You must select Reset Web Survey within Test Survey Mode.) The following information is displayed when you click Reset Web Survey:

WARNING: Clicking "Reset Survey" below will cause this web survey ranq to be reset. Resetting a survey includes: 1) Deleting ALL respondent data. This includes incomplete and complete data records from both the data and admin directories. The internal respondent counter will also be reset. 2) Resetting all the password information. All passwords will be reset and ready to use again. Changes made to Maximum Respondents in the On-line Data Management Module will remain. Only reset a survey if you have been testing and want to clean out test data and restart or if you are finished with your survey and no longer have use for the data. Clicking Reset Survey will DELETE ALL OF YOUR DATA for this study!

Click the Reset Survey link to proceed.

266 1.11.2.10

SSI Web v7

Customizing the Administrative Module

(This is an advanced area.) The Admin Module uses a default format and appearance, but you can modify the settings. You can also change the default behavior regarding which menu items are available based on the level of password (read-write vs. read only).

Customizing the Menu Items

Under the Compose | Survey Settings + Server Settings tab dialog, you specify Administrative Module Access passwords: Read-Only and Read-Modify administrative rights. These two permission levels control which menu items are available within the Admin Module. Read Only administrative rights let the user... View the Study Summary View Survey Data Run Marginals View the Passwords Reports Test Survey

Read/Write administrative rights permit all the above operations, plus... Download Data Edit Data Delete Selected Records Close a Survey Reset a Survey

You can change these default settings by creating a file named STUDYNAMEconfig.cgi and by placing it in the admin folder in your server setup. If this file is present, SSI Web ignores the user name and password for the Admin Module as specified in the Survey Settings dialog. Using STUDYNAMEconfig.cgi allows you to specify multiple passwords with customized levels of access to the Admin Module. Here is an example demonstrating the STUDYNAMEconfig.cgi file format: STUDYNAMEconfig.cgi { "admin_access" => { "Bob|3829" => {"read_access" => 1}, "Joe|8743" => {"study_summary" => 1, "marginals" => 1}, }, }

In the example above, the user "Bob" with password "3829" is able to access all of the settings that come with Read Access. "Joe" on the other hand with password "8743" will only see the Study Summary and Marginals menu options.

General Web Interviewing (CiW) Help The available settings are: full_access read_access study_summary download_data view_data marginals password_reports test_survey close_survey reset_survey

267

Customizing the Look of the Admin Module

You can customize the look of the Admin Module by creating your own CSS file. In the Survey Settings dialog, under the Server Settings tab, enter the name of your custom CSS file. You can place this CSS file in the graphics folder or you can refer to a remote file by listing the full URL (i.e. http://...). If the CSS file is placed in the graphics folder then you will need to include the graphics path. For example: [%GraphicsPath()%]your_style.css By including a custom CSS file, you can remove all "Sawtooth Software" branding from the Admin Module. To remove our title and include your company's logo, include the following CSS: #title_text { display: none; } #title { background:url('your_logo.gif') no-repeat center center; height: 50px; }

268

SSI Web v7

1.12
1.12.1

Exporting/Analyzing Data
Exporting Data
(Before you can export data, you must have accumulated the data and downloaded the STUDYNAMEdat.dat file to the study directory on your hard drive using the Admin Module.) Click File | Export Data | All Data and the Export All Data dialog is displayed. This dialog offers four tabs: Export File Type (described here) Respondent Filter Data to Export Data Formatting You can export the data from your Internet questionnaire to one of the following files: Comma Delimited (.csv) (Supported by Microsoft Excel) This file can be directly opened with Microsoft Excel(TM). It is a comma-delimited text file, where the first line includes variable labels. Export Labels File. If you are using Sawtooth Software's online analysis tool, you should export a labels file. Note that this will slow down the export process. Microsoft Excel 2002- 2003 (.xls) We suggest this format when using Excel 2002 or 2003 (limited to no more than 256 columns), especially if you are exporting open-end responses that have been typed using double-byte character sets. The file is saved with a .xls extension. Microsoft Excel 2007 (.xml) This file format can support up to 16,384 columns. We suggest this format when using Excel 2007 or later, especially if you are exporting open-end responses that have been typed using double-byte character sets. The file is saved with an .xml extension. SPSS (.sav) This file can be directly opened with SPSS. It is a binary file. Fixed-Width Text (.tab and .lay) The .tab file is a fixed column text file. This file can be read by most any statistical processing or cross-tab package. The layout for the .tab file is included in the .lay file, which is also a text file. The .lay file specifies the variable names and in which column in the .tab file each variable begins and ends. A default name and path is provided in the Output Filename: field. You can directly edit the name. By clicking the browse button, you can specify the folder in which you wish to save the file.

General Web Interviewing (CiW) Help

269

1.12.2

Selecting Data to Export

This dialog lets you specify the data you wish to export to the output file. All items in the Export List on the right are exported when you click the Export button. You can move any items from the left window (list of available data) to the Export list using the arrow buttons. The > button moves just the highlighted items. The >> button moves all items. You can highlight a range of items by holding down the Shift key and clicking with the mouse. You can highlight multiple items selectively by holding down the Ctrl key and clicking each one with the mouse. Items marked with an asterisk (*) are considered "system variables." These include respondent number, passwords, interview timing information, and respondent status. User Name is one of the fields from the passwords module. Password is also one of the fields from the passwords module Sequential Respondent # are integers 1 through n assigned to each respondent record that is exported. Internal Interview Numbers are those that the system assigns when respondents complete the survey over the Web. There can be gaps in the sequence if some respondents started a survey but didn't finish. The internal respondent numbers are alphanumeric, and are converted to numeric. Start Date/Time is the time (as recorded on the server, often GMT-Greenwich Mean Time) that respondents entered the survey. End Date/Time is the time (as recorded on the server, often GMT-Greenwich Mean Time) that respondents last submitted data. Respondent Status (2 = incomplete; 5 = complete) Pagetime is the time that respondents took to answer each page.

Notes Regarding Exporting Multiple-Response Data: With multiple-response select questions, more than one variable is stored for the question. For example: Q1. Which of the following colors do you like? (Select all that apply) Red Blue Yellow Green Violet None of the above

The data for this question are saved as six separate variables: Q1_1, Q1_2, Q1_3, Q1_4, Q1_5 and Q1_6. There are two ways that you can export the responses: Checked/Unchecked: If respondents chose the item, the value is 1, if not, it is set to 0. Item Numbers: Values corresponding to the response options are listed, for as many items as were selected.

270

SSI Web v7 Grid questions can also save multiple responses. Grid data use the convention gridname_r#_c#, where the r# refers to the row number and c# refers to the column number. For example, q11_r3_c5 stores information for the grid named q11 and the response contained in row 3 column 5.

Some Notes Regarding Missing and Blank Data

There are different ways to achieve blanks in data positions, and it is important to understand the distinction when analyzing the data. 1) Respondent never saw the question. The data are blank for questions on the pages the respondent never saw. Or, the respondent failed to click the submit button on a page with questions. Skip patterns can create blank data. 2) Respondent saw the question, but did not respond. For each CiW question-type, you can specify whether the question can be skipped by respondents. If you need to distinguish between whether none of the categories of a multiple-response question truly apply for a respondent versus whether the respondent simply skipped the question, you should probably include a "None Apply" response category and then require a response for that question.

General Web Interviewing (CiW) Help

271

1.12.3

Additional Data Formatting Settings

This dialog lets you specify how multi-select responses should be formatted. It also allows you to specify if you would like to have the text from other/specify responses exported as a data field. There are additional controls for SPSS value and variable labels.

Multiple-Response Questions
Multi-select questions are ones such as the following: Q3. Which of these flavors do you often eat? (Select all that apply) Chocolate Rocky-Road Strawberry Peach Vanilla Neapolitan When the data are exported, this question becomes six fields: Q3_1 through Q3_6. Consider a respondent that checks Vanilla (item #3) and Peach (item #5). SSI Web can export the data formatted in two ways (you should select which method makes most sense given your tabulation software or data processing needs): Item Numbers: Q3_1 Q3_2 Q3_3 Q3_4 Q3_5 Q3_6 3 5 Checked / Unchecked: Q3_1 Q3_2 Q3_3 Q3_4 Q3_5 Q3_6 0 0 1 0 1 0

Exporting Open-Ends
You can export openend ends to a text file by clicking File | Export Data | Open-Ends Only.... Or, you can export open-end responses as a data field within a standard data export under File | Export Data | All Data.... From the Data Formatting tab of the Export All Data dialog, check Export text from "Other / Specify" Responses. When you export an other/specify field, it is named questionname_item#_other. For example, if in Q1, the seventh item was an other specify response, the variable name is Q1_7_other.

A Note on Column Widths

When you export data to SPSS or Fixed-Width Text using the .tab/.lay format, numerics and open-end responses can take on different widths, depending on how long of an answer the respondents in a data set have provided. By default, SSI Web selects a minimum width required to export the data for the current respondents. But, if you later add respondents, your variable widths might change in a subsequent export. If you want to fix the export widths and the resulting export table, so that new data will not alter the variable widths, you should select Export columns using fixed widths. The drawback is that the resulting data table is not as compact.

272

SSI Web v7

Exporting to SPSS
When you export data to SPSS, you can specify which portion(s) of your question should be used as variable labels. You can also specify whether to strip HTML from value and variable labels.

The final option allows you to include Additional Question-Specific text in variable labels (grid header, list labels, etc.). SPSS Variable Labels cannot be more than 120 characters in length, so anything beyond this will be truncated.

General Web Interviewing (CiW) Help

273

1.12.4

Exporting Open-End Data

The Export All Data option creates a data file (STUDYNAME.tab) intended for reading by a statistical or cross-tabulation package. If you want to export the open-ends in their entirety in a format that is intended for use with word processors or coding programs, you should choose File | Export Data | Open-ends Only. The Export Open-end Data dialog is displayed. Respondent Numbering You can choose from five options for assigning respondent numbering. This lets you specify how the respondent numbers (case IDs) are to be generated. User Name is one of the passwords from the passwords module. You should only choose this if these values were purely numeric (no letters). Password is also one of the passwords from the passwords module. You should also only use this if the passwords were purely numeric (no letters). Internal Interview Numbers are those that the system assigns when respondents complete the survey over the Web. There can be gaps in the sequence if some respondents started a survey but didn't finish. Sequentially Assigned Numbers (default) are integers 1 through n. Other lets you choose another numeric variable to use for respondent numbering. This last option is useful if you are linking surveys from another source and need to merge the data sets by a "passed-in" variable. By default the open-ends are saved in the STUDYNAME.txt file, but you can change this name if you want. The data are grouped by question. Within each question, responses are displayed by respondent number. Here is a sample open-end file for two respondents and three open-end responses:
Why_like 196 I have tried other soft drinks, but I like Sawtooth Spritz best. 234 Don't know, I just like its zing. First_Name 196 John 234 Mary Last_Name 196 Jones 234 Burns

Any carriage returns respondents type are converted to spaces in the STUDYNAME.txt file.

274

SSI Web v7

1.12.5

Analyzing Data with Online Simulator

Sawtooth Software provides online market simulators for analyzing conjoint data and also MaxDiff. After you compute your part-worth utilities (in the case of conjoint methods) or scores (in the case of MaxDiff) you import the .HBU, .UTL, or .CSV files directly into the online simulator for analysis. The online simulator is a service that you purchase on a monthly or quarterly basis, depending on the needs of your project. Many researchers find the online simulator to provide a more attractive and simple interface for clients (both internal and external). Sawtooth Software's SMRT system provides a more powerful, but less attractive interface for conducting market simulations.

General Web Interviewing (CiW) Help

275

1.12.6

Moving Data from SSI Web to SMRT for Analysis

If conducting conjoint analysis studies (CBC, ACBC, ACA and CVA) with SSI Web, you'll often want to move the data from SSI Web into the SMRT software package for analysis (market simulations, counting analysis, logit, etc.). Sometimes data are moved directly into SMRT, and other times data are moved from SSI Web into a preliminary analysis program such as hierarchical Bayes, and then afterward to SMRT.

CBC (Choice-Based Conjoint) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. Export the CBC data (just the data directly involving the choice tasks) to a studyname.CHO (text-only format) file. From the SSI Web menu, click File | Export Data | Prepare CBC Data files (*.cho/*.chs and *.att). 2. Start the SMRT software (a different program from SSI Web, also downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the .CHO file into SMRT, by clicking (from the SMRT software menu) File | Import, browsing to your .CHO file, and clicking Import. The raw choice data (respondent answers and experimental design) are now within the SMRT study. You can proceed with counting analysis or logit within SMRT. Note: If you use CBC/HB or Latent Class software to estimate part-worth utilities, use the .CHO (or .CHS, in the case of constant sum responses to be analyzed with CBC/HB) and .ATT files with those systems (see their respective manuals for details) to produce a data file containing respondent partworths (a .HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .HBU or Latent Class probability file (.Pxx). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

ACBC (Adaptive Choice-Based Conjoint) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. Estimate part-worth utilities within SSI Web by clicking Analysis | Calculate ACBC Utilities Using HB.... This process creates a text-only format file called STUDYNAME.hbu. 2. Start the SMRT software (a different program from SSI Web, also downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new

276

SSI Web v7 study functions independently of your original SSI Web study. 4. Import the STUDYNAME.hbu file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CBC utilities (selecting .HBU file type). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use CBC/HB or Latent Class software to estimate part-worth utilities, use the .CHO (or .CHS, in the case of constant sum responses to be analyzed with CBC/HB) and .ATT files with those systems (see their respective manuals for details) to produce a data file containing respondent partworths (a .HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .HBU or Latent Class probability file (.Pxx). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

ACA (Adaptive Conjoint Analysis) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. If using the default OLS (Ordinary Least Squares) part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate ACA Utilities using Ordinary Least Squares. This process creates a text-only format file called STUDYNAME.utl. 2. Start the SMRT software (a different program from SSI Web, downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing ACA utilities (selecting ACA v4.x Utilities *.utl as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use ACA/HB software to estimate part-worth utilities, you can either run ACA/HB as a separate program, or if you own ACA/HB v3 or later, you can run the program as an integrated component from within the SSI Web interface. If running the estimation outside of SSI Web, from the SSI Web menu, select File | Export Data | Prepare ACA Data Files (*.acd and *.aca)... to export the ACA information to a STUDYNAME.acd format into the same folder containing your ACA/HB system. See the ACA/HB manual for details of producing a data file containing respondent part-worths (a .HBU file). After estimating part-worth utilities using either ACA/HB approach (within SSI Web menu or outside program), import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .hbu file. Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

General Web Interviewing (CiW) Help

277

CVA (Conjoint Value Analysis) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. If using the OLS (Ordinary Least Squares) part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate CVA Utilities using Ordinary Least Squares. If using the Monotone Regression part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate CVA Utilities using Monotone Regression. Either process creates a text-only format file called STUDYNAME.utl. 2. Start the SMRT software (a different program from SSI Web, downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CVA utilities (selecting CVA Traditional Conjoint Utilities *.utl as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use CVA/HB software to estimate part-worth utilities, run the program as an integrated component from within the SSI Web interface. After estimating part-worth utilities using the CVA/HB approach, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .hbu file. Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

Moving Segmentation (Generic Survey) Data from SSI Web to SMRT

We assume you have first followed the procedures above for estimating part-worth utilities and establishing those as a utility run in a new project within SMRT. 1. From the SSI Web system, export the data you wish to merge into SMRT. From the SSI Web menu, select File | Export Data | All Data, select the respondent numbering method (the numbering method must match the method you chose when exporting the conjoint data or estimating utilities), and export the data you wish to merge into SMRT (only numeric data may be merged). Select either a .CSV file (comma-delimited text-only file) or a fixed text-only file (.TAB). 2. Within the SMRT software, open the project previously established that contains the conjoint data/utility run. Use File | Merge Variables | Add to merge the data into your SMRT project (see the SMRT manual or online help within SMRT for details). After merging the data, if using SMRT v4.11 or earlier, you'll additionally need to specify Custom Segmentation Variables (Under Analysis | Custom Segments) if you wish to use the new variables as banner points or respondent filters during simulations.

278

SSI Web v7

1.13
1.13.1

Files created by SSI Web

Files created by SSI Web
Files created when creating surveys on the local PC Name STUDYNAME.ssi STUDYNAME.ssi.bak STUDYNAME.pwds Description Main authoring file. Contains all questionnaire information. Backup of .ssi file. Passwords database, contains merged data. Contains design information Contains design information Design file for a particular MaxDiff exercise.

Files created by CBC studies only STUDYNAME_CBC1_<DesignID>.cgi Files created by CVA studies only STUDYNAME_CVA1_<DesignID>.cgi

Files created by studies with MaxDiff exercises STUDYNAME_MXD<ExerciseID>_<DesignID>.cgi

Files created in the Web Upload folder to upload to the web server Name STUDYNAMEqst.cgi STUDYNAMEpath.cgi STUDYNAMEpacc.cgi STUDYNAMEpwds.cgi STUDYNAMElogin.html STUDYNAMEadmin.html Description Main web questionnaire file. Contains server path information. Contains Admin Module passwords. Contains password information Login page. Entry point for respondents. Admin Module login page. Entry point for study administrators. Contains design information Contains design information Design file for a particular MaxDiff exercise.

Files created by CBC studies only STUDYNAME_CBC1_<DesignID>.cgi Files created by CVA studies only STUDYNAME_CVA1_<DesignID>.cgi

Files created by studies with MaxDiff exercises STUDYNAME_MXD<ExerciseID>_<DesignID>.cgi Perl Files ciwweb.pl admin.pl ciwlibX_X_X.pl authlibX_X_X.pl pverlibX_X_X.pl grdlibX_X_X.pl enterlibX_X_X.pl acalibX_X_X.pl cbclibX_X_X.pl acbclibX_X_X.pl

(where X_X_X is the version number of the software, e.g. 7_0_0) Files created during data collection on the Web server (You should not delete these files manually during fielding. If you click the Reset Survey option in the Online Administrative Module, these files and folders are deleted.)

General Web Interviewing (CiW) Help Name STUDYNAMEcnt.cgi STUDYNAME????.cgi STUDYNAMEdat.dat STUDYNAMEadminchangeslog.cgi STUDYNAMElog.cgi /STUDYNAME_quotas STUDYNAME_close_survey_off.cgi /STUDYNAME_pass admin_limits.cgi STUDYNAMEavgimp.cgi STUDYNAME_ACBC_log.cgi

279

Description Counter file (in /admin directory) for assigning internal respondent numbers. Individual respondent data record (in /data directory). Accumulated data file when downloaded to local drive. Admin module edit changes log file. Records any changes made to respondent data through the Admin Module. Error log file reporting errors in the survey. Please review contents and contact Sawtooth Software if this file exists in your /admin directory. Created on server to manage quota control. This file contains the message that will be shown when "Close Survey" is turned on. Directory created within the /admin directory to keep track of password files, for restarting surveys and ensuring passwords are not used multiple times, unless permitted. Inside STUDYNAMEquota folder. Stores cell limits specified in admin module. Contains average derived ACA importance scores for complete respondents. Is only present if omitting the Importance section and using population means for priors. ACBC log file created by ACBC studies. Contains warnings and error messages.

Files located in the graphics/system and their corresponding image files mootools_v1_2.js Located in graphics/system. JavaScript library. ssi_admin_styleX_X_X.css Web Stylesheet ssi_styleX_X_X.css Web Stylesheet ssi_javascriptX_X_X.js Javascript library (where X_X_X is the version number of the software, e.g. 7_0_0) Files created during analysis of data on the local PC Name STUDYNAME.csv STUDYNAME.lay STUDYNAME.xml STUDYNAME.sav STUDYNAME.tab STUDYNAME.txt STUDYNAME.xls CBC Files CBCEXERCISENAME.att CBCEXERCISENAME.cho CBCEXERCISENAME.chs ACBC Files ACBCEXERCISENAME.acbccal ACBCEXERCISENAME.alp ACBCEXERCISENAME_Counts.csv ACBCEXERCISENAME.cov ACBCEXERCISENAME.csv Description Exported data file (Comma-Separated Values) Data layout file for .tab file Used in the online simulator Exported SPSS(TM) data file Exported text-only data file Text file of open-end responses (created by export) Exported Excel data file CBC Attribute label file (created by export) CBC choice data file, discrete choice only (created by export) CBC choice data file, allocation data only (created by export) Calibration concept design and responses, by respondent Estimated population means for part-worths from HB, by iteration Counts results by individual (created by Analysis+ACBC Counts) Covariance matrix from HB Contains part-worth utilities computed via Analysis+HB (or monotone)

280

SSI Web v7 ACBCEXERCISENAME.gamma ACBCEXERCISENAME.hbu ACBCEXERCISENAME.log ACBCEXERCISENAME.prc ACBCEXERCISENAME.restart ACBCEXERCISENAME.std ACBCEXERCISENAME.sum ACBCEXERCISENAME.val MaxDiff Files MAXDIFFEXERCISENAME.att MAXDIFFEXERCISENAME.cho MAXDIFFEXERCISENAME.eff MAXDIFFEXERCISENAME.mtrx MAXDIFFEXERCISENAME.val ACA Files ACAEXERCISENAME.aca ACAEXERCISENAME.acd ACAEXERCISENAME.hbu ACAEXERCISENAME.utl CVA Files CVAEXERCISENAME.att CVAEXERCISENAME.hbu CVAEXERCISENAME.utl Contains history of estimates of task-specific scale factors from HB (Otter's method) Contains part-worth utilities computed via Analysis+HB (or monotone) Contains summary report of fit statistics by iterations for HB runs Prior covariance matrix used for HB runs Restart file for HB runs Standard deviations by part-worth parameter by respondent from HB Contains final estimate of population means and covariances for HB runs Values file describing any linear coding of utilities for use in SMRT MaxDiff attribute label file (created by export) MaxDiff choice data file (created by export) Effects file defining coding method and interaction terms (created by export) Prior covariance matrix (created by export) Values file specifying values for linear or user defined variables (created by export) Contains attribute level text to accompany .utl for ACA ACA audit trail file (export to HB) ACA/HB conjoint utility file (created by computing utilities via HB) ACA conjoint utility file (created by computing utilities) Contains attribute level text to accompany .utl for CVA CVA/HB conjoint utility file (created by computing utilities via HB) CVA conjoint utility file (created by computing utilities)

CBC Help

281

2
2.1
2.1.1

CBC Help
Getting Started with CBC
Overview of CBC Software
CBC is a component within the SSI Web suite of Internet interviewing products. CBC is used for conducting Choice-Based Conjoint (CBC) studies over the web, in non-web "CAPI" interviewing mode, or via paper-and-pencil questionnaires. CBC questions are usually integrated within a longer SSI Web questionnaire that includes other standard survey questions. The number of additional questions that can be added depends on your license. A CBC study can include additional standard survey questions that collect up to 50 data fields. An unlimited number of additional questions can be used if the largest CiW interviewing license is acquired. CBC studies are used for learning about respondents' preferences for the combinations of features that make up products or services. CBC analysis can help with (among other things) product design, line extensions, pricing research, and market segmentation. Your license entitles you to free technical support via phone or email. We encourage you to contact us at support@sawtoothsoftware.com or 360/681-2300 if you cannot find the answer to your problem or question in this documentation. We are available Mon-Fri, 7:00 am to 5:00 pm Pacific time. (Note: we provide full technical support for composing and testing your Internet survey on your local PC, and provide written instructions on how to install your questionnaire on your server. We cannot take responsibility for or be expected to provide technical support to debug any customized HTML or JavaScript you may add to your survey. You are also responsible to make sure that Perl is running and that you can run Perl (CGI) scripts on your web server. We can provide some assistance related to SSI Web's use with your server once the fundamental Perl functionality is in place.) A sample CBC questionnaire (tut3.ssi) and supporting files are available in the \Samples folder within your SSI Web installation. You can easily access this example study by clicking Help | Sample Studies. This questionnaire lets you see an example of a CBC survey and investigate how it is set up. If you know a little HTML (which is not necessary to use CBC), you can employ some "tricks" to help you customize the look of your survey beyond the standard options provided by the software. By opening the sample study and looking at the settings and text we've used, you can become familiar with some of the possibilities. Using CBC requires the following skills and knowledge base: The analyst managing the survey should know the basics of conjoint analysis, including defining attributes and levels properly, correctly phrasing the questionnaire text and response scales, interpreting the part-worth utilities, and running simulations. It is not necessary to be a statistician to use our software effectively, but knowledge of basic statistics for market research is highly recommended. The questionnaire is first composed locally on a PC running under Windows XP, Vista, or Windows 7. The individual inputting/formatting the survey should have working knowledge of standard Windows applications such as word processing, Web browsers and/or spreadsheets. Though it is not necessary, some knowledge of HTML is valuable. For web-based data collection, one must have access to an Internet/intranet server running Perl 5.003 or later. This can be on your company's own Web server, or your ISP's (Internet Service Provider's) Web server. The user must know how to use FTP software to create directories, upload files to those directories, and set permissions on the files and directories. Click here to receive detailed instructions on the directories, files and permissions. (Sawtooth Software also provides web hosting services for SSI Web studies.)

282

SSI Web v7 Relationship between CBC and SMRT CBC is used for designing, creating, and fielding CBC studies (either web-based, non-web "CAPI" interviewing, or paper-and-pencil modes). But you analyze the CBC data using the SMRT software (an online simulator option is also available from Sawtooth Software). These two software systems are installed and run independently--but they share a common file format (*.CHO), enabling the SMRT software to easily read the data resulting from a CBC study. If you don't already own SMRT software, you receive a special copy (along with a manual) to analyze your CBC data.

Steps in Conducting a CBC Study

1) Formulate attributes and levels and type them into the CBC interface. 2) Make decisions regarding your study design: decide how many product concepts to display per choice question (task), whether to include a "None" option, and how many choice tasks to field in your study. Using the point-and-click CBC interface, add those CBC questions to your SSI Web study. 3) Test your questionnaire in preview mode, by using the Local Test Server included in SSI Web, or by uploading the files to your Web server. Make modifications and re-test. 4) Decide how many unique versions of the CBC questionnaire to use in your overall design (default is 300). If using paper-and-pencil data collection, it is common to use a few to a dozen design versions. Generate the design. Using the Test Design module, test the CBC design to make sure the design is efficient to estimate the part-worth utilities (main effects). 5) If using web-based data collection, upload your study to an Internet server, and continue testing--by yourself, with colleagues, and later preferably with a small number of pilot respondents. Receive feedback from colleagues and pilot respondents. Examine the data to make sure all data are being stored properly using the Admin Module, and by downloading the data to your hard drive. Export the data to a text-only *.CHO file. Estimate part-worth utilities using HB (hierarchical Bayes) or Latent Class. Import the results into the SMRT software package or the online market simulator for analysis. 6) After ensuring that your CBC questionnaire works properly, that the data are being stored correctly, and that your design is not deficient, invite respondents to take the survey. 7) Monitor the data collection using the Admin Module. Download the data, and then Prepare files for analysis within the SMRT system (or the online market simulator). See the SMRT manual, or online help within the SMRT software for documentation regarding CBC analysis.

CBC Help

283

2.1.2

What is Choice-Based Conjoint?

CBC is used for conducting "Choice-Based Conjoint Analysis" studies over the web, in non-web "CAPI" (Computer-Aided Personal Interview) interviewing mode, or via paper-and-pencil questionnaires. CBC studies are used for learning about respondents' preferences for the combinations of features that make up products or services. CBC analysis can help with (among other things) product design, line extensions, pricing research, and market segmentation. The market simulators that result from CBC analysis enable managers to test numerous product formulations and competitive scenarios. Market acceptance/competitiveness, price sensitivity, and cannibalization are just some of the issues that market simulators based on CBC data can probe. The main characteristic distinguishing choice-based conjoint analysis from earlier types of conjoint analysis is that the respondent expresses preferences by choosing concepts (products) from sets of concepts, rather than by rating or ranking them. Over the last decade, choice-based conjoint has become the most widely used conjoint-related technique.

The Choice Question

A CBC question is often referred to as a "task." A set of products (concepts) is displayed on the screen, and the respondent chooses among the concepts. For example:

This is a basic display for illustration. The user has a great deal of control over the fonts, colors, and layout. Graphics may also be used. Realistic-looking "store shelf displays" can be developed with scores of product graphics displayed if using the Advanced Design Module for CBC. The attributes that make up each concept are carefully chosen so that the independent effect of each attribute level upon a product concepts' likelihood of choice can be estimated. CBC software automatically designs the concept combinations based on a few user-defined inputs. Two important decisions to make when constructing choice tasks are 1) how many concepts to present per task, and 2) how many total tasks to ask. These are sometimes difficult issues to answer, and ongoing research is helping to define guidelines and understand the consequences. A typical study might include about a dozen or more choice tasks, with about four to six product concepts per task. The composition of the choice tasks usually will vary between respondents. Each respondent receives one version of a larger pool of efficient designs. Because respondents are randomly selected to receive a different version of the overall design, we refer to these designs as "random" designs.

284

SSI Web v7 However, the term random can be misleading, as the designs themselves are carefully selected using a computer algorithm that ensures that each separate version has level balance and near orthogonality. Often some "holdout" choice tasks are included in the CBC questionnaire. These are not initially used for estimating the preferences (part-worth utilities) for the respondents, but are used to check the internal validity of the estimated utilities. We refer to these as "holdout" tasks or "fixed" choice tasks. They are "fixed" in the sense that the product combinations within these tasks are shown in exactly the same way to all respondents. (After holdout tasks have been used for validation purposes or selection of alternate models, some researchers choose to employ them in a final model for part-worth estimation.)

The Role of CBC

Choice-based conjoint analysis has attracted much interest over the last two decades in the marketing research field. There are many reasons for its rise to dominance among the conjoint-related methods: Researchers tend to favor it because the task of choosing a preferred concept is similar to what buyers actually do in the marketplace. Choosing a preferred product from a group of products is a simple and natural task that anyone can understand. Choice-based conjoint analysis lets the researcher include a "None" option for respondents, such as "I wouldn't choose any of these." By selecting that option, respondents who do not like any of the options can express their lack of interest. Comparing "None" usage across groups of respondents can reveal segments that are relatively more or less likely to purchase product concepts. Most conjoint analysis studies use "main effects only" assumptions. Choice-based conjoint analysis typically involves leveraging the data across respondents, making it feasible to quantify interactions. This capability is enhanced by the random designs used by the CBC System, which, given a large enough sample, permit study of all interactions, rather than just those expected to be of interest when the study was designed. It should be noted, however, that using HB estimation for estimating individual-level part-worths often reduces the need to model additional interaction terms (in other words, the interactions observed in aggregate models are often due to unrecognized heterogeneity). It is possible in choice-based conjoint analysis to have "product-specific" (alternative-specific) attributes. For example, in studying transportation we might consider walking shoes and bicycles. The attributes describing shoes are different from those describing bicycles, and yet one might want to learn how much improvement in walking shoes would be required to switch a respondent from cycling to walking. CBC's Advanced Design Module permits alternative-specific designs. Choice-based conjoint analysis does have a disadvantage, however: it is an inefficient way to elicit preferences. Each concept is described using many attributes and each choice set contains several concepts. Therefore, the respondent has to do a lot of reading and process a lot of information before giving each answer. Moreover, the response data are sparse. Each choice does not indicate how much more preferred that option is relative to the remaining options, or the relative preferences for the not-chosen alternatives. For those reasons, in the early days of CBC research, choice-based conjoint studies were usually not used to estimate the values that individual respondents attached to attribute levels, as had been done with traditional ratings-based conjoint methods. Instead, data from groups of respondents were aggregated for analysis. This has been done either by combining all respondents or by studying subsets defined by specific market segments. "Part-worth utility values" could be produced for each

CBC Help

285

group of respondents that summarize the choices made by those individuals. And, as in other conjoint methods, the part-worth values could be used to simulate and predict respondent reactions to product concepts that may not have actually appeared in the choice tasks (questions). Aggregating respondents in CBC analysis assumed respondent homogeneity, which was not always appropriate or desirable. Developments since the mid-1990s have recognized segment-based or even respondent-by-respondent differences for CBC analysis. Latent Class analysis (as offered in the CBC Latent Class Segmentation Module) can simultaneously delineate relatively homogeneous segments and estimate their unique part-worth functions. Computationally intensive Bayesian estimation (available in Sawtooth Software's CBC/HB System) has permitted estimating individual-level part-worths from choice data. HB generally leads to more accurate predictions and greater flexibility during analysis. It is generally considered a gold-standard today, though other sophisticated part-worth estimation methods often produce very similar quality results. If you are relatively new to CBC research, we highly recommend that you visit our Technical Papers library at www.sawtoothsoftware.com/techpap.shtml. We recommend you download and read: The CBC Technical Paper An Overview and Comparison of Design Strategies for Choice-Based Conjoint Analysis The Benefits of Accounting for Respondent Heterogeneity in Choice Modeling Special Features of CBC Software for Packaged Goods and Beverage Research

286

SSI Web v7

2.1.3

CBC Tutorial and Example

Introduction
This tutorial leads you step-by-step through a simple CBC study, from thinking about a marketing problem, planning a questionnaire and creating a new study in SSI Web, to entering a list of the attributes and levels, generating the questionnaire and analyzing the results. SSI Web is a powerful program, and this tutorial only touches on the basics. You'll become aware of different or more sophisticated approaches by reading other sections in this manual, by viewing on-line help and by experimenting with SSI Web. We also suggest the following two articles, available within our Technical Papers Library at www.sawtoothsoftware.com/techpap.shtml: "CBC Technical Paper" and "Getting the Most from CBC." (Note: this example and the data presented are purely fictional. The suggested steps represent one approach to this type of marketing problem, but may not necessarily reflect the single "best" way to conduct such a research study using the SSI Web system.)

The Marketing Problem

You have been hired as a consultant to a little-known startup company called Performance Plus. The brilliant design engineers at Performance Plus have developed a golf ball that flies farther than average balls. The proposed name for the ball is "Long Shot." Because Performance Plus is not well known among golfers, your client has considered approaching a well-known maker of golf clubs and balls (Golfers, Inc.) to market the ball using their existing brand name for balls: "Eclipse." Sales for Eclipse have been declining, and they welcome the opportunity to benefit from the new technology. A line extension brand called "Eclipse+" has been proposed. However, the royalty Golfers, Inc. is proposing seems high. Two other manufacturers already market "high performance" balls: High-Flyer Pro (by Smith and Forester), and Magnum Force (by Durango). High performance balls command a price premium of between 20% to 50% over traditional balls, and the market is growing rapidly. One day between rounds of golf with the president of Performance Plus (you gracefully lost), you jointly draft up the following questions on the back of your score card. 1) How well could Long Shot by Performance Plus hope to compete with existing competitors Smith and Forester and Durango in the high performance ball market? 2) Should Performance Plus form an alliance with Golfers, Inc.? How much incremental value does the Golfers, Inc. brand name "Eclipse" add? 3) How do golfers trade off performance vs. price for high performance balls? 4) How should "Long Shot" be manufactured and positioned in terms of performance and price to maximize profitability? 5) Do novice or experienced golfers differ in their preferences for high performance golf balls? Could this lead to a target market strategy?

Defining a List of Attributes and Levels

One of the first steps in using any Conjoint technique is to define the problem in terms of attributes and

CBC Help

287

levels. Back at the office, you review the points written on the back of the score card, and you write down the following: Brand High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc. Long Shot, by Performance Plus Performance Drives 5 yards farther than the average ball Drives 10 yards farther than the average ball Drives 15 yards farther than the average ball Price $4.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls Each of these levels is mutually exclusive; golf balls are described using a single level from each attribute. The range of the attributes is based on technical specifications that Performance Plus has given you. As for prices, your research suggests that the two competitors are currently selling packages of three balls for between $7.99 to $10.99. You select a wider range to cover the price levels you are interested in, but don't consider any prices below $4.99, having determined that such a low price cannot cover the manufacturing and marketing costs and result in a profit.

A Study Plan and Sample Scripted Questionnaire

Now that you have formulated some attributes and levels, you begin to think about the rest of the questionnaire, and plan a design and field strategy. You decide to invite golfing enthusiasts (by email) to visit a site on the web and take a CBC survey. You've purchased an opt-in list of 2000 email addresses from a widely-read golfing magazine. Each of the participants who complete the survey will be entered into a drawing for a free set of golf clubs from a leading manufacturer. You expect a completion rate of between 15% to 25%, resulting in a final sample size of between 300 to 500. You have scripted the following introductory text screens and questions in your word processing package: Start: Thank you for your willingness to participate in this study. You are part of a special group of golfers we've selected to ask for feedback regarding golf equipment. By completing this survey, you'll be eligible for a drawing to receive a free set of PrimoDriver clubs. (Click the Next button to continue)

288

SSI Web v7 Frequency: First, we'd like to ask you about how often you golf. Would you say that you golf... m m m m 5 times or fewer per year from 6 to 25 times per year from 26 to 50 times per year more than 50 times per year

Skill: We'd like to get a feel for your skill level. Over the last year, what was your best score on a par-72, 18-hole course? m m m m m 120 strokes or more Between 100 and 120 strokes Between 85 to 100 strokes Above par, but lower than 85 At or below par (72 or lower)

Venue: Do you usually golf at... m Public courses m Private courses IntroCBC: In the next section, we'll be asking you specifically about high performance golf balls. These balls are premium balls specially engineered to travel farther than the typical ball with good accuracy. We'd like you to imagine that you are considering purchasing golf balls for your next golf outing. We'll show you some different high performance golf balls, and ask which one you would purchase. Some of the golf balls you are going to see are not currently available on the market, but we'd like you to imagine that they were available today. It is important that you answer in the way you would if you were actually buying golf balls. If you wouldn't purchase any of the balls we'll show you, you can indicate that by choosing "None". By choosing none, you indicate that you'd buy another brand, or would continue using existing balls in your golf bag.

At this point in the questionnaire, you plan to ask the Choice-Based Conjoint questions (tasks). You want to measure preference for the different levels of the attributes and their impact on choice for high performance golf balls. To accomplish this, you decide to specify 15 CBC tasks (below is an example of one such task). The features will combine freely to form up to 48 (4 x 3 x 4) different product offerings. Even though there are only 48 total possible product concepts in this relatively small design, there are thousands of different ways to display multiple product concepts at a time in choice sets.

CBC Help

289

Choice Task (one of 15 constructed tasks using a controlled randomization design strategy): If you were considering buying golf balls for your next outing and these were the only alternatives, which would you choose? High-Flyer Pro, by Smith Magnum Force, by and Forester Durango Drives 15 yards farther than the average ball Drives 15 yards farther than the average ball Eclipse+, by Golfers, Inc. Drives 5 yards farther than the average ball $6.99 for package of 3 balls None: I Wouldn't Purchase Any of These

$10.99 for package of 3 $6.99 for package of 3 balls balls

Even though these CBC tasks are called "random tasks," that term is easily misunderstood. Some people take it to mean that the composition of the questions is randomly (haphazardly) selected. However, the questions are carefully chosen according to design principles of balance and independence (orthogonality). We call the tasks "random" because respondents are randomly selected to receive one of many carefully constructed, unique versions of the CBC questionnaire. In addition to the random choice tasks, you also plan to ask a few fixed "holdout" tasks that will be constant across all respondents. You plan to do this for three main reasons: 1) By doing some research on the Internet and at local golf stores, you have learned the performance specifications and average prices for the two existing balls on the market (HighFlyer Pro and Magnum Force). Your client has also given you a good feel for the product specifications they think could match up well with these two brands. Therefore, you and your client can imagine a specific potential product scenario you'd like to directly assess. 2) Your client has no experience with conjoint methods and seems a bit uncertain regarding the reliability and accuracy of the market simulator. Demonstrating that the market simulator can accurately predict responses to some holdout fixed scenarios should boost his confidence in the method. 3) You'll may want to test how well different part-worth utility analysis methods work for the study. You'd like to have a couple of "holdout observations" by which to compare the predictive accuracy of alternative solutions you might develop. These fixed tasks will not be constructed randomly, but are to be displayed exactly with the specific levels you assign. These two tasks will be asked as the 7th and 9th tasks. In total, you'll ask 15 random + 2 fixed = 17 tasks. In the first fixed holdout, you plan to show the relevant competition versus your client's offering with your client's brand name. In the second fixed task, you plan to show that same relevant competition versus the Eclipse+, by Golfers, Inc. Here are those two fixed holdout concepts: Fixed Holdout Task #1 (7th overall choice task): If you were considering buying golf balls for your next outing and these were the only alternatives, which would you choose?

290

SSI Web v7 High-Flyer Pro, by Smith Long Shot, by and Forester Performance Plus Drives 15 yards farther than the average ball Drives 10 yards farther than the average ball Magnum Force, by Durango Drives 5 yards farther than the average ball $8.99 for package of 3 balls None: I Wouldn't Purchase Any of These

$10.99 for package of 3 $6.99 for package of 3 balls balls Fixed Holdout Task #2 (9th overall choice task):

If you were considering buying golf balls for your next outing and these were the only alternatives, which would you choose? Magnum Force, by Durango Drives 5 yards farther than the average ball $8.99 for package of 3 balls Eclipse+, by Golfers, Inc. Drives 10 yards farther than the average ball $6.99 for package of 3 balls High-Flyer Pro, by Smith None: I Wouldn't and Forester Purchase Any of These Drives 15 yards farther than the average ball $10.99 for package of 3 balls

Notice that your client's offering is in the center position for both tasks, but the position of its competitors is rotated. The two fixed holdout tasks are separated by a random choice task so that respondents will generally not recognize that it is a repeated task with just one small difference (the brand name attached to your client's ball). Lastly, you decide to ask about gender and income. These might prove useful as a respondent filter in analysis, for bench-marking versus future waves of research, or for future target/database marketing efforts. Gender: Are you.. m Male m Female m Refused

Income: This is the last question in the survey. What is your total household income for the last calendar year, before taxes? m m m m m Less than $30,000 Between $30,000 and $60,000 Between $60,000 and $100,000 More than $100,000 Refused

Signoff: That completes our survey. Thank you very much for your input.

CBC Help

291

Now that you have developed a list of attributes and levels, and scripted a rough-draft of your questionnaire and the design of the choice tasks, you are ready to create the study and compose the questionnaire within the SSI Web system.

Using SSI Web to Write the Questionnaire

Start by opening the Sawtooth Software SSI Web program. If you have a standard installation, you start it by clicking Start | Programs | Sawtooth Software | Sawtooth Software SSI Web. "SSI" stands for "Sawtooth Software Inc." and "Web" conveys the idea that this software is for developing Web-based surveys (although SSI Web can also be used for standalone computer interviewing with its CAPI interviewing module.) CBC is one component within that software program. The next step is to open a new study, and define a study name. After starting SSI Web, choose File | New Study. The New SSI Web Study dialog appears, with your cursor active in the Name field.

The Location field lists the folder (directory) that SSI Web currently is using to store studies. You can use any folder you like for this tutorial project. You can browse to and create new folders by clicking the Browse... button to the right of the Location field, and then (after browsing to the folder in which you want to create a new sub-folder to contain this study) by clicking the "new folder" icon. Each studyname in SSI Web has a maximum number of twenty-four characters (either letters or numbers), and SSI Web requires that the study be located within its own folder. The folder that SSI Web automatically creates for your study also carries the same name as the study. For this tutorial, you might create a name such as golfexample. From the New SSI Web Study dialog, specify golfexample as the studyname. Click the OK button. You are returned to the main menu, and a new "Study Navigator" window is displayed along the left-hand side of the screen. As you work with this study, items appear in the Study Navigator window, listing the functional areas you have been using or that are now available to you. This list of items provides a Shortcut link to access those parts of SSI Web. Alternatively, you can access those same areas by clicking icon buttons on the toolbar or by using the pull-down menus. Now that we have created an SSI Web study called golfexample, we can add our CBC exercise to that study. Click Compose | Write Questionnaire... to access the Write Questionnaire dialog. Place your cursor at the place in the questionnaire when you'd like to add the CBC exercise. Right now, there are no questions in your survey, so you can just click the last page break in the questionnaire list, and click the Add... button. Specify to add a new CBC exercise named CBCgolfexercise to the study (you can have multiple CBC exercises within the same study).

292

SSI Web v7

Click OK, and the CBC Exercise Settings dialog is shown:

Default text is supplied within the Header 1 field. Paste the text we'll be using for this task into the Header 1 field:

CBC Help

293

If you were considering buying golf balls for your next outing and these were the only alternatives, which would you choose?

Entering the List of Attributes and Levels

When composing conjoint analysis studies, one typically begins by specifying the list of attributes and levels in the software. To enter the list of attributes and levels you developed, go to the Attributes tab on the CBC Exercise Settings dialog.

To add the first attribute, click the Add button (at the bottom left of the Attributes panel).

294

SSI Web v7

Type in the first attribute name: Brand. The attribute name is a label that the respondent may see in the interview (if you select the option to show the attribute labels at the left of each choice task). If you want to use a shorter label to display to you as the questionnaire author for program settings and data export, specify that label in the Internal Label field. If you do not specify an internal label, the label in the Attribute Display Text is used. Click OK to accept this information and close the dialog. Now that at least one attribute name is in the list, the Add button under the Levels panel becomes active. Also note that the Brand attribute is highlighted in the Attributes panel. With the Brand attribute highlighted in the Attributes panel, click the Add button under the Levels panel to add levels within the Brand attribute. The Level Text dialog is displayed. Type High-Flyer Pro, by Smith and Forester in the Level Display Text field. To add the next level of Brand, press the ENTER key twice. Type in the next level: Magnum Force, by Durango. Repeat the process for the remaining two levels of brand. When you are ready to add the next attribute (Performance), click the Add button under the left Attributes panel, type the attribute label, and click OK to place that new attribute on the attribute list. With that new attribute highlighted on the attributes list, click the Add button under the Levels panel to add the three levels of that attribute. Follow the same pattern for the last attribute, Price. For your convenience, we repeat the full list of attributes below. Note that you can copy-and-paste attribute level text from this document (and other text documents) into the text fields within SSI Web. After highlighting the words to copy with your mouse, use the shortcuts Ctrl-C to copy, and Ctrl-V to paste into the desired field. (Hint: you can select a list of attributes or levels from Word and paste into SSI Web using the processor or spreadsheet" icon. This can save a great deal of time.) Brand High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc. Long Shot, by Performance Plus Performance Drives 5 yards farther than the average ball Drives 10 yards farther than the average ball Drives 15 yards farther than the average ball Price $4.99 for package of 3 balls "paste from word

CBC Help $6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls

295

Now that you have finished entering the list of attributes and levels, we'll use the other tabs to continue developing our CBC exercise.

Additional Study Parameters

After you have specified your list of attributes and levels, you can specify other study parameters to govern your CBC exercise. From the CBC Exercise Settings dialog, click the Design tab. The upper left-hand corner of that dialog shows the following:

Here, you can specify how many Random (experimentally designed) choice tasks and how many fixed (user specified) tasks you want. For this golf ball study, we want 15 random and 2 fixed tasks. Change the Number of Random Choice Tasks field to 15. As planned, the questionnaire shows three products on the screen, plus a None. Change the Number of Concepts per Choice Task to be 3 (excluding the None option). Recent research has suggested that the Balanced Overlap design approach is a robust approach for studies like this, so select Balanced Overlap as the Random Task Generation Method. It is useful to have many versions of the questionnaire (each showing a different combination of golf balls). This leads to efficient designs for estimating part-worth utilities, and also controls for order bias. The default in the software is to generate 300 unique versions of the "Randomized" CBC tasks. Such a quantity may be overkill, but the software can handle this quite easily, so it seems prudent to go with a large number such as 300. Recall that we want to include a "None" option in this CBC questionnaire. This gives respondents the option to indicate that they would choose none of the products displayed on the screen. Click the Format tab. Toward the bottom of that dialog, you'll see:

296

SSI Web v7

The Traditional None Option is selected default, and you can specify the text to use for your None option by clicking the Settings... button. Specify the following text within the None Option dialog: None: I Wouldn't Purchase Any of These Click OK to return to the Format tab. The upper part of that tab shows the following settings:

The Format Options control how wide the choice tasks might appear on the respondent's monitor, how much white space is included between concepts and attribute levels, and how many columns to use to display the product concepts. Number of Columns describes how many columns will be used to arrange the product concepts on the screen. Our questionnaire calls for three products plus a none, for a total of 4 columns. Change the Number of Columns to 4. The Alternating Concept Colors are used in alternating concepts in the choice task to help make the choice task more readable, and provide a visual indicator from one choice task to another that the question has changed (sometimes choice tasks can look so similar that respondents might not notice from, say, the first task to the next that anything is different.) It appears that no alternative colors have been selected, but indeed alternating colors have already been selected as part of the Style that is currently being used for your SSI Web study. If you would like to override the default colors used in your style, you can select new colors here. You can override the default font color and size specified in the Style that is currently being applied to

CBC Help

297

your SSI Web study (Styles are selected by going to Compose | Survey Settings, Style tab) by clicking the Paintbrush icon.

To preview how the CBC task will look when it runs on the respondent's computer using Windows Explorer, click the Preview button. The question is displayed in "Preview" mode (no data are saved, and no special "error checking" or "validation" is performed for the question).

This may or may not be the look you want for the questionnaire. You can modify the font and sizes of the font using the Paintbrush icon. Or, you can select a different Style (selecting styles was described in the 45-minute tutorial for SSI Web). Go ahead and close the browser window. At this point, you might try modifying some of the settings on the Format tab. After each change you make, you can click Preview to see the effect of the changes.

Generating the Experimental Design

In CBC studies, we use a carefully chosen experimental design plan. The "experiment" involves observing how respondents react to different golf ball designs. The "design" reflects the attribute combinations that make up the golf balls, and how these combinations are placed into choice sets. Ideally, each respondent would receive a unique questionnaire version, with different combinations of golf balls arranged within sets in unique ways. There are only a finite number of possible combinations, so there potentially can be some identical choice tasks across respondents, but the idea is to improve measurement of the effects of the attribute levels (including reducing order and learning effects) by ensuring a high degree of variability in the choice tasks across individuals. CBC lets you generate up to 999 unique versions of the questionnaire to be uploaded to the web server. By pre-specifying the design plans in a file residing on the server, we can simply assign each new respondent to the next design version within the file, which places minimal demands on the web server. Even if you have more than 999 respondents, once a reasonably large number of designs are allocated across respondents, the statistical gains of using even more questionnaire versions are very minimal. The default in the software is to use 300 design versions. Even 300 designs is probably overkill, but the software manages hundreds of designs for typical studies quite easily, and it seems beneficial to reduce order bias and improve design efficiency by using numerous questionnaire

298

SSI Web v7 versions. If you aren't already at the CBC Exercise Settings dialog, go there by navigating to the CBC exercise from the Study Navigator panel, or by clicking Compose | Write Questionnaire and editing a question within the exercise in the question list. Click the Design tab, and the following is displayed:

Many of the features of this dialog are beyond the scope of this tutorial. You can read additional details by pressing F1. We'll cover some of the basics here. Under the Design Settings area, we specify the Random Task Generation Method. This indicates the strategy that CBC uses to generate the "random" choice tasks in our study. We'd recommend using the Balanced Overlap method for this study. Number of Questionnaire Versions refers to how many unique versions of the CBC questions that SSI Web will save into a file. When respondents enter the questionnaire, they are assigned the next questionnaire version. Once a respondent is assigned the 300th version, the next respondent begins again with version #1, and so forth. Randomize Attribute Position within Concepts lets you randomize the order in which the attributes appear in the choice task. We specified the attribute list in the order Brand, Performance and Price. If we use the default (which is not to randomize the order), Brand will always appear in the top and Price always at the bottom. This seems like a natural presentation order for this study, so we'll retain the Do not Randomize Attribute Order setting. Concept Sorting within Tasks controls how the concepts are arranged within the choice task. By

CBC Help

299

default, the order of presentation is randomized. But, if we wanted the first brand always to appear in the first concept position, etc., we could specify to sort concepts based on the Natural Order for Brand. You can investigate details regarding Concept Sorting by pressing F1. For this tutorial, we'll retain the defaults. Once we are comfortable with our settings, we click Generate Design. You are given two warnings, notifying you that you have yet to specify the composition of the two fixed tasks in your study. We'll do that later, so you can ignore the warnings. The following report is displayed:
CBC Design Efficiency Test Copyright Sawtooth Software 12/9/2009 3:43:42 PM Task generation method is 'Balanced Overlap' using a seed of 1. Based on 300 version(s). Includes 4500 total choice tasks (15 per version). Each choice task includes 3 concepts and 3 attributes. A Priori Estimates of Standard Errors for Attribute Levels ------------------------------------------------------------Att/Lev Freq. Actual Ideal Effic. 1 1 3375 (this level has been deleted) High-Flyer Pro, by Smith and Forester 1 2 3376 0.0275 0.0275 0.9970 Magnum Force, by Durango 1 3 3375 0.0275 0.0275 1.0023 Eclipse+, by Golfers, Inc. 1 4 3374 0.0275 0.0275 0.9999 Long Shot, by Performance Plus 2 1 4500 (this level has been deleted) Drives 5 yards farther than the average ball 2 2 4500 0.0231 0.0231 0.9964 Drives 10 yards farther than the average ball 2 3 4500 0.0230 0.0231 1.0052 Drives 15 yards farther than the average ball 3 3 3 3 1 2 3 4 3375 (this level has been deleted) 3374 0.0276 0.0275 0.9954 3376 0.0276 0.0275 0.9994 3375 0.0276 0.0275 0.9934 $4.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls

Note: The efficiencies reported above for this design assume an equal number of respondents complete each version.

The report indicates that 300 versions of the CBC questionnaire were created. We know that each version of the questionnaire had 15 "random" choice tasks, so a total of 300 x 15 = 4500 choice tasks were generated. Some of the details of the report are beyond the scope of the unit, but you can obtain more information within the Help text. The "Freq." column indicates how many times each level is represented in the design. Within each attribute, you can see that CBC has nearly perfectly balanced the presentation of levels. The "Effic." Column indicates the relative efficiency of the design. Design efficiency runs from the worst (0.0) to the best (1.0), and we can see that this particular design is quite efficient with respect to main effects. If you include prohibitions in your design, sometimes designs can be quite inefficient. If the Effic. values are relatively low, this is a warning that your estimates may lack precision, and you should reconsider your setup. In some cases, you will receive a warning stating that the design is deficient or will notice asterisks rather than data in the test design report. You should re-evaluate your design should this occur. Failure to correct the problem can result in unusable data. Click Close to close the test design report, and click OK to close the CBC Exercise Settings dialog.

300

SSI Web v7 Let's now examine the questionnaire we've built to this point:

You'll note that SSI Web has added 15 Random (CBCgolfexercise_Random1 through CBCgolfexercise_Random15) and 2 Fixed CBC tasks (CBCgolfexercise_Fixed1, CBCgolfexercise_Fixed2) to the survey. These questions are based on your settings and the default th template within SSI Web. The Fixed tasks aren't yet in the proper positions (we wanted them in the 7 th and 9 task positions in the survey). You also haven't yet specified the attribute combinations used for your user-defined Fixed tasks, so SSI Web has inserted the questions with default levels that we'll modify shortly. Before we fine tune those CBC questions, let's turn our attention to formatting other aspects of the survey.

CBC Help

301

Passwords and the Start Screen

Placing a survey on the web makes it convenient for a geographically dispersed population to take surveys. However, the danger is that the survey may become available to people that have not been invited to take the survey. Also, some respondents might try to take the survey multiple times. Assigning respondent passwords is a way to deal with both of these issues. Password assignment is beyond the scope of this tutorial, so to make things simple we'll assume that no passwords are to be used. The Start screen is placed as the first page in your questionnaire, whether you are using passwords or not. If you are using passwords, it is the page in which respondents type passwords to access the survey. If not using passwords, you'll use this screen to specify any introductory/welcome text. Let's enter the introductory text into the Start screen using the Write Questionnaire dialog. You can access the Write Questionnaire dialog by selecting Compose | Write Questionnaire or by clicking the "pencil" icon on the toolbar.

The introductory text and other survey questions would most likely be initially developed within a word processing document. Assuming you really had such a document, you might use the Ctrl-C to copy, and the Ctrl-V shortcuts to paste the information into SSI Web. We suggest you simply copy-andpaste the text within this document into your SSI Web questions rather than re-type the text (if viewing this document with Acrobat Reader, you can use the "text select" icon from the Acrobat Reader toolbar.) The introductory screen is as follows: Start: Thank you for your willingness to participate in this study. You are part of a special group of golfers we've selected to ask for feedback regarding golf equipment. By completing this survey, you'll be eligible for a drawing to receive a free set of PrimoDriver clubs. (Click the Next button to continue)

We'll insert this introductory screen in the Start question that is automatically the first question of any SSI Web survey. From the Write Questionnaire dialog open the Start question by highlighting it in the list of questions and clicking Edit.... Alternatively, you can double-click the Start question, and it automatically opens. The Questionnaire Access and Passwords dialog is displayed:

302

SSI Web v7

Most question types in SSI Web have "Header 1", "Header 2," "Body" and "Footer" sections. (The Start question has all but the "Body" section.) These are "text" areas in which you can insert any text (including HTML). When the question is viewed with a web browser, the sections are organized roughly as follows:

For this introductory screen (that includes three paragraphs of information), it seems to make sense to place the first paragraph in the "Header 1" area, the second paragraph in the "Header 2" area, and the

CBC Help third paragraph in the "Footer" area.

303

Type (or cut and paste) the following text for the Start question into the text areas in the Start question. Put the first paragraph in Header 1, the second paragraph in Header 2, and the third paragraph in Footer. Thank you for your willingness to participate in this study. You are part of a special group of golfers we've selected to ask for feedback regarding golf equipment. By completing this survey, you'll be eligible for a drawing to receive a free set of PrimoDriver clubs. (Click the Next button to continue) To preview how the question will look when it runs on the respondent's computer using Windows Explorer, click the Preview button. The question is displayed in "Preview" mode (no data are saved, and no special "error checking" or "validation" is performed for the question).

SSI Web automatically places paragraph breaks (extra blank lines) between the Header 1, Header 2, and Footer sections. If you put all the text in a single section, you may see that when the web browser interprets the text, it all runs together without any blank lines between the paragraphs (unless you insert some HTML instructions to force blank lines between paragraphs). We'll talk about using HTML within your text to take greater control over the layout, font, and style later in this tutorial. Click the "X" icon in the upper-right hand portion of the Preview window to close that window and return to the previous SSI Web dialog. After viewing the text in preview mode, you might decide that it is really too small, or that you want to make the text bold. You can change the size and styles (bold, italic, underline) or the text justification for the three major text sections by clicking the Paintbrush icon After changing any setting, click Preview again. on the Start Question Text tab.

After you are happy with the layout of the Start screen, click OK to return to the Write Questionnaire dialog. If you need to move a question once it has been added to the List of Questions, simply highlight the question to be moved and click the or buttons to move the questions within the list (you can also highlight a question and click Ctrl-X or Ctrl-C to cut or copy and Ctrl-V to paste questions within the current questionnaire, or even to another SSI Web study.) Now you are ready to specify the first Select-Type question:

304

SSI Web v7 Frequency: First, we'd like to ask you about how often you golf. Would you say that you golf... m m m m 5 times or fewer per year from 6 to 25 times per year from 26 to 50 times per year more than 50 times per year

First, make sure that you highlight the page break directly beneath the Start question on the List of Questions (when you add a new question, it is placed directly below the highlighted question/page break on the list). At the Write Questionnaire dialog, click Add..., choose Select as the question type, and type Frequency for the question name. Click OK and the Select Question dialog is shown. Place the heading text for the question in the Header 1 field. To specify the response options, click the Response Options tab. From the Response Options tab, use the Add... button to add the four response options for this question. When you are finished, the question should look something like:

Now that you have seen how to specify Single Select question types in SSI Web, you have the tools you need to specify the remaining four select-type questions for the golf ball questionnaire (Skill, Venue, Gender and Incomeplease refer to the questionnaire text for these questions as presented near the front of this tutorial). Remember that when you add a new question to the List of Questions, it is inserted directly below the question that was highlighted when you clicked the Add... button.

Formatting Text in SSI Web Questions

In addition to the Select questions we added to the questionnaire previously, there are two text-only instructional screens to add. Referring to the questionnaire we outlined earlier, these "questions" are named IntroCBC and Signoff. Even though the respondents aren't asked to provide specific inputs to these, we refer to these as "questions" in the sense that we add them to the questionnaire as if they were standard questions, and they are listed separately by their "question" name in the List of Questions.

CBC Help After the Venue question follows the text-only instructions:

305

IntroCBC: In the next section, we'll be asking you specifically about high performance golf balls. These balls are premium balls specially engineered to travel farther than the typical ball with good accuracy. We'd like you to imagine that you are considering purchasing golf balls for your next golf outing. We'll show you some different high performance golf balls, and ask which one you would purchase. Some of the golf balls you are going to see are not currently available on the market, but we'd like you to imagine that they were available today. It is important that you answer in the way you would if you were actually buying golf balls. If you wouldn't purchase any of the balls we'll show you, you can indicate that by choosing "None". By choosing none, you indicate that you'd buy another brand, or would continue using existing balls in your golf bag.

This text layout is a bit more challenging than we dealt with before (when we put three paragraphs of introductory text into the Start question). There are more than three separate paragraphs here, and we'll need to deal with the additional element of bolding selected text. Add a Text/HTML Filler question directly following the Venue question in the questionnaire. To do so, highlight the Venue question, click Add, and specify the Question Name as IntroCBC and the question type as Text/HTML Filler. The Text/HTML Filler dialog is displayed. Earlier in this tutorial, when we used the Start question to format the opening page of the survey, we placed each paragraph of text in a separate Header 1, Header 2, or Footer section. We saw that SSI Web automatically places blank lines between text in these sections. However, with the text in the IntroCBC question, there are many more paragraph breaks. We'll take the opportunity here to introduce the concept of using a few simple HTML instructions within our survey text. HTML stands for "HyperText Markup Language" and provides simple ways for you to enhance the look of your surveys, such as by bolding or underlining words, or making paragraph breaks within text. Browsers know how to interpret HTML instructions when displaying the page on the screen. If you know HTML, you can use it whenever you wish within SSI Web surveys to accomplish your aims. If you don't know HTML, it isn't very difficult to learn a few HTML tricks, or you can use the toolbar available within the editor that appears when you click "Pencil" icons in SSI Web:

Either type or cut-and-paste the text for the IntroCBC question into the Header 1 text field (Any of the other three fields could be used also, though the Body field is indented). Click the "Pencil" icon to display the larger text editing window. Initially, the text appears something like this:

306

SSI Web v7

Even though it appears that there are extra blank lines between the paragraphs, web browsers ignore these hard returns (as well as more than one consecutive space characters), so you'll need to provide HTML instructions to insert these paragraph breaks (hard return plus blank line). HTML instructions are placed within <> brackets, called "tags." For example, the HTML instruction to create a paragraph break begins with an "open" paragraph tag written as <p> and optionally ends with a "close" paragraph tag written as </p>. The text to be formatted as a separate paragraph is enclosed within these tags. You can either directly type HTML within your document, or you can highlight text to be modified and click the icons on the HTML toolbar in the editor. If you highlight the first paragraph with your mouse: In the next section, we'll be asking you specifically about high performance golf balls. These balls are premium balls specially engineered to travel farther than the typical ball with good accuracy. and then (with the text in the first paragraph highlighted) click the "Paragraph" icon this inserts a <p> prior to the text and a </p> after the text: on the toolbar,

<p> In the next section, we'll be asking you specifically about high performance golf balls. These balls are premium balls specially engineered to travel farther than the typical ball with good accuracy.</p> When the browser interprets this text, it doesn't display the "tags" but instead separates the text enclosed within the tags as a separate paragraph. Repeat the same for each of the paragraphs in the IntroCBC question. Next, we need to bold certain words in the text. The HTML tags for bolding text are <b></b> (with the text to be bolded placed between the open and close bold tags). You can either directly type these tags within the document, or highlight the text to be bolded with the mouse and click the "Bold" icon on the toolbar. After you finish separating the paragraphs with <p></p> tags and bolding the appropriate text with <b></b> tags, it should look something like:

CBC Help

307

Click OK to return to the Text/HTML Filler dialog and then Preview to see how the web browser displays this question. It should look like:

When you add the Signoff at the end of the survey, make sure to add it at the end of the survey as a Terminate/Link question type. On the Settings tab of the Terminate/Link question, click Terminate Respondent to indicate that respondents that reach this question are finished. SSI Web may warn you at this point that a terminating question must stand alone on its own "page" in the survey. This leads us into our next discussion on page layout.

308

SSI Web v7

Page Layout and Study Settings

It may be helpful at this point to review how SSI Web breaks the survey into separate pages, and some basic global settings that affect the look and functionality of your SSI Web questionnaire. The Write Questionnaire dialog also displays the page breaks that may be currently set for your questionnaire.

This dialog shows how the various questions we've specified (or that SSI Web has automatically added to the list) are arranged across different pages. Page breaks are indicated by "---------------<Page>". Notice that the preliminary questions (Frequency, Skill and Venue and IntroCBC) we've added to the questionnaire are all currently arranged on the same page. Let's assume we wanted to break these

CBC Help

309

up, one question per page. You add a page break after the Frequency question by highlighting Frequency on the list and clicking the Add Page Break button. Frequency is now placed on its own page. Repeat this operation to place Skill, Venue, IntroCBC and CBCgolfexercise_Random1 on separate pages. Make sure to place the final Terminate/Link screen Signoff on its own (last) page. While we had been adding page breaks for our questionnaire, you may have noted that the two fixed th th CBC tasks (CBCgolfexercise_Fixed1 and CBCgolfexercise_Fixed2) are not yet in the 7 and 9 CBC task positions. You can move a question on the list by highlighting the question in the List of Questions and clicking the or buttons to move the question to another point in the questionnaire. Move CBCgolfexercise_Fixed1 directly after CBCgolfexercise_Random6 and move CBCgolfexercise_Fixed1 directly after CBCgolfexercise_Random7. You may need to re-arrange the page breaks so that each CBC question remains on its own page.

Specifying Fixed Tasks

A fixed choice task looks just like typical "Random" (experimentally designed) tasks. However, rather than let the design algorithm determine the combination of attribute levels to be shown for each respondent, you specify the codes for the levels to be displayed in each fixed product concept. Recall that the purpose of specifying fixed holdout tasks for this example is to achieve a controlled and direct measure of the existing competitors' offerings versus our client's proposed offerings. You'll also be able to use the results to check the ability of the market simulator you develop using the 15 random choice tasks to predict the responses to the 2 fixed holdout tasks. Recall that our list of questions already includes two fixed tasks named CBCgolfexercise_Fixed1 and CBCgolfexercise_Fixed2. These tasks will look exactly like the experimentally designed (random) tasks within the survey, except that you'll specify precisely which levels to show in those two tasks. Edit any of your CBC questions (or use the Study Navigator panel to directly go to the CBC exercise). Go to the Design tab on the the CBC Exercise Settings dialog. To modify the levels displayed in the fixed choice task, click the Fixed Task Designs button.

From this one dialog, you can specify the attribute combinations to be used in each of your fixed tasks. To change from one fixed task to another, use the Fixed Choice Task drop-down box at the upper-left hand corner of the dialog. We'll first modify CBCgolfexercise_Fixed1. Recall that the first fixed holdout was to display the following three products:

310

SSI Web v7 High-Flyer Pro, by Smith Long Shot, by and Forester Performance Plus Drives 15 yards farther than the average ball Drives 10 yards farther than the average ball Magnum Force, by Durango Drives 5 yards farther than the average ball $8.99 for package of 3 balls None: I Wouldn't Purchase Any of These

$10.99 for package of 3 $6.99 for package of 3 balls balls

Use the drop-down controls provided to specify the three product concepts as shown. Click the OK button when finished. Preview the fixed holdout task to make sure it looks as you expect. Make any necessary changes. Next, you need to modify the second fixed holdout task. From the CBC Fixed Choice Task Settings, used the Fixed Choice Task drop-down control to select CBCgolfexercise_Fixed2. Modify it to have the appropriate specifications for our second fixed choice task: Magnum Force, by Durango Drives 5 yards farther than the average ball $8.99 for package of 3 balls Eclipse+, by Golfers, Inc. Drives 10 yards farther than the average ball $6.99 for package of 3 balls High-Flyer Pro, by Smith None: I Wouldn't and Forester Purchase Any of These Drives 15 yards farther than the average ball $10.99 for package of 3 balls

Again, preview this task to make sure it looks as expected.

Adding Polish and Style

You have probably noted that the survey we've created is pretty bland looking. We can add some polish and style by clicking the Survey Settings icon from the Write Questionnaire dialog (or by clicking Compose | Survey Settings from the main menu). First, let's select a style from the Styles tab. Select a style you wish to use, and click Use Selected at the bottom of the dialog to implement the style. We'll select Panama as the style, but you might want to experiment with other styles. On the General Settings tab, you can select to use graphical Next, Previous, and Select/Checkbox buttons. A library of buttons is provided, available in the C:\Program Files\Sawtooth Software\SSI Web\graphics folder. On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Golf Ball Questionnaire as the header text. The Progress Bar tab lets you add a progress bar to your survey. We'll click the check box to add a progress bar to the page footer. When you go back and test run the survey, your survey should have a bit more polish. Under Panama style (and by selecting graphical Previous and Next buttons), our survey now looks like:

CBC Help

311

Test Running Your Survey Using "Local Test Server"

Although the Preview Survey function is nice, you generally will want to test your survey in runtime mode, with any skip patterns, response verification, and randomizations in place. You could upload your files and the Perl scripts to the Web Server, but SSI Web provides a more convenient way to test run your survey locally on your PC or laptop. When SSI Web was installed on your computer, web server software called "Apache" was also installed. Apache lets your computer run the questionnaire just like a remote Web Server would.

Local Test Server

From the Write Questionnaire dialog, Click the Test Survey button Locally.

and select to Test Survey

Your browser opens up the first survey page. This is your survey, as it would appear to a respondent over the Internet (there are slight visual differences from browser to browser). To close the survey, simply close the browser window (using the X in the upper-right hand corner). To run the survey again, click the Test Survey button .

After you have edited the survey to the point that you are pleased with its look, content, and functionality, you should examine the test data you've collected using Local Test Server (this process is described in a separate tutorial called " Getting Started with SSI Web: A 45-minute Hands-On Tour" available on the Help menu), then at least examine the CBC data using Counting analysis (described later). The results should generally reflect your preferences (assuming you answered in a rational manner). If they don't, this suggests something may be wrong with your study setup. After you are convinced that the survey is functioning properly, you should also pretest your survey

312

SSI Web v7 among your colleagues. They can give you feedback regarding the usability of the survey, and you can examine the resulting Counts data or perhaps even the part-worths (assuming you have enough data) to make sure the data at least have face validity.

Pretesting and Fielding

Let's now return to the marketing problem and story we began at the beginning of this document. After you have tested the survey using Local Test Server, you post the survey to the Web on your web server, on a server provided by an ISP (Internet Service Provider), or through Sawtooth Software's web hosting service. (Setting up your survey on the web is beyond the scope of this unit, but is described in detail in the SSI Web help documentation. Fielding options also include hosting on your own company's server, or using Sawtooth Software's hosting services). The next day, you send an email to your client, with a link to take the survey. The president of Performance Plus first takes a survey and suggests some minor wording changes. After you make those changes, you invite six other individuals at Performance Plus take the survey. You download the results and analyze the resulting data. Due to the small sample size, the results are a bit noisy (and there is an obvious bias toward your client's balls), but the data seem to feel right. Next, you recruit six golf enthusiasts within your city to come to a central site to take the survey. You watch silently as they take the survey. At the end of each session, you ask each respondent follow-up questions to ensure that there weren't any parts that were difficult to understand or just didn't make sense. You ask them specifically about the choice tasks, making sure that there wasn't too much information on the screen at once, and that they didn't feel overwhelmed with the task. After debriefing the test respondents and analyzing their data to ensure that the results looked reasonable, you make a few small adjustments and proceed to field. Using a bulk email program, you send an email to the list of 2000 golfing enthusiasts from the opt-in list mentioned at the beginning of this unit. After a week, and a reminder email sent to those who had not yet completed the survey, you achieve 300 total completes. The completed data reside on the server where you uploaded the SSI Web survey. The details for viewing and downloading the data from the server are provided in other areas of the SSI Web documentation and are beyond the scope of this CBC unit. Even so, it is useful to at least describe the steps you would perform to access and download the data from the Web, estimate the part-worth utilities, and begin analysis using the market simulator.

Data Management, Utility Estimation, and Moving the Data into SMRT
SSI Web includes an Admin Module so that you can monitor or access your project from any computer connected to the Web. The Admin Module is password protected, and your passwords for access are specified on the Server Settings tab of the Survey Settings dialog. SSI Web generates random administrative access passwords whenever you create a new study, but you can modify them to suit your needs. To download the data from the 300 respondents to the golf ball study, you can browse to the administrative module for your study on the web site (again, we are speaking hypothetically, as for this tutorial study there is no such site set up). Once at the administrative module, you would download your data (by clicking Download Data from the main menu), making sure to save your data (named STUDYNAMEdat.dat, where STUDYNAME is your study name) to the same folder on your hard drive in which you developed the SSI Web project. Alternatively, from the SSI Web interface, you can simply click Field | Download Data... to download your data without logging into the Admin Module.

CBC Help

313

Once you have downloaded the data, you are ready to export the CBC data to a .CHO format file (with accompanying attribute labels in a file with extension .ATT), and estimate part-worths in preparation for running market simulations. Assuming you had downloaded the data as described above, you would click File | Export Data | Prepare CBC Data Files (*.cho/*.chs and *.att). If you try this with this tutorial study, you will receive an error stating that you have no data within your project folder to export. However, hypothetical data for this project are stored in a tutorial folder within the accompanying SMRT software that you received together with your SSI Web system. SMRT stands for "Sawtooth Software Market Research Tools" and contains the market simulation tool used to analyze data resulting from any of Sawtooth Software's conjoint analysis systems. SMRT is the platform that (given the proper license with its user identification codes) runs CBC for Windows studies and includes the tutorial data for this golf ball study. We'll discuss how you would move data from your CBC project into SMRT (hypothetically, since the data aren't actually in your CBC project) but then open the SMRT software system and continue using the golf ball tutorial data that were installed there. CBC researchers employ a variety of techniques for analyzing their data. One common approach for initially understanding the basic, summary preferences for the market is called "Counting" analysis. This produces proportions from 0.0 to 1.0 for each level in your study, reflecting how often this level was chosen, when available in a choice set. The higher the proportion, the higher preference for the level. While this method of analysis has good intuitive appeal, there are more powerful ways to analyze the data based on estimation of part-worth utilities and subsequent market simulations. There are three part-worth estimation routines that Sawtooth Software provides. The first technique is called multinomial logit, which "pools" respondent data in a single aggregate model. This technique was the first part-worth estimation technique that Sawtooth Software used for analyzing CBC data in the early 1990s. It is the standard base estimation module provided with the SMRT software that accompanies your CBC software license. It is a good technique for quickly summarizing the results for the sample, but it is quite susceptible to the IIA (red-bus/blue-bus) problem, which is discussed in many technical papers and in the CBC help documentation. Two other estimation techniques are commonly used for developing the final simulator models delivered to clients: Latent Class and Hierarchical Bayes (HB). These techniques model respondent heterogeneity (recognize differences between respondents in terms of preferences) and in most cases provide more useful part-worths (than aggregate logit) for the purpose of market simulations. It is beyond the scope of this tutorial to discuss these part-worth estimation methods. These part-worth estimation programs all work seamlessly with the .cho and .att files exported by SSI Web. If you use the logit estimation routine built into the SMRT software system, you are immediately ready to perform additional market simulations and analysis within SMRT. If you use either Latent Class or HB estimation, these routines produce a text-only format files containing case IDs and part-worth estimates that can easily be imported by SMRT. For simplicity, we'll assume that you plan to use logit estimation provided by SMRT to produce quick topline results. To move the STUDYNAME.cho file into SMRT for analysis by logit, you would (again, this is for discussion purposes onlyyou should not actually perform these steps for this golf ball tutorial): 1. Start the SMRT software by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 2. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 3. Import the STUDYNAME.CHO file into your SMRT study by clicking (from the SMRT software menu) File | Import, selecting the file type as Choice Data (*.cho) and browsing to your .CHO file. Once you import the CBC data from the .CHO file, you can analyze the data.

314

SSI Web v7

Analyzing the CBC Data Using Counts

SMRT is a companion software system to SSI Web that can be used for analyzing the results of CBC studies. To start SMRT, click Start | Programs | Sawtooth Software | Sawtooth Software SMRT. Open the golf ball study by clicking File | Open and then browsing to find the Tutor2.smt file located in the ...Program Files\Sawtooth Software\SMRT\Tutorial folder. Hypothetical data are provided to be used for the remainder of this tutorial. To access the data, please select the Tutor2 study (select File | Open, and double-click the Tutor2.smt file located in your Samples directory). If you used a randomized design (which you did for this project), usually the first step in analyzing the choice results is to conduct a "Counting" analysis. CBC's Counts program reports the percent of times each attribute level was chosen when it was available on the screen. Counts provides an intuitive measure of the impact of each attribute level on overall choice for golf balls. Counts are proportions ranging from 0 to 1. For example, a Count of 0.31 for an attribute level would mean that when a golf ball was displayed including that particular level, respondents chose it 31% of the time. To access the Counts program, return to the CBC main menu, and select Analysis | Counts. By default, the Counts program analyzes all one-way and two-way Count proportions. Notice also by default that the Counts program uses only responses to the 15 randomized choice tasks you fielded (under Choice Tasks to Include, All Random is checked). The two fixed holdout tasks are not included. That is because results from Counts assume random designs where each attribute level appears an equal number of times with each level of the other attributes. Randomized designs (with no prohibited level combinations) make it possible to analyze the effect of each attribute level independent of all other levels. That characteristic does not usually hold if fixed holdout tasks are included in the analysis. Click Compute! and the following report is displayed in the report window:
Brand Total Respondents High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc. Long Shot, by Performance Plus Within Att. Chi-Square D.F. Significance Performance Total Respondents Drives 5 yards farther than the average ball Drives 10 yards farther than the average ball Drives 15 yards farther than the average ball Within Att. Chi-Square D.F. Significance Price Total Respondents $4.99 for package of 3 balls Total 300 0.397 Total 300 0.168 0.282 0.339 259.984 2 p < .01 Total 300 0.367 0.336 0.189 0.159 416.495 3 p < .01

CBC Help
$6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls Within Att. Chi-Square D.F. Significance Brand x Performance Total Respondents High-Flyer Pro... High-Flyer Pro... High-Flyer Pro... Magnum Force, ... Magnum Force, ... Magnum Force, ... Eclipse+, by G... Eclipse+, by G... Eclipse+, by G... Long Shot, by ... Long Shot, by ... Long Shot, by ... Drives Drives Drives Drives Drives Drives Drives Drives Drives Drives Drives Drives 5 yards farther... 10 yards farthe... 15 yards farthe... 5 yards farther... 10 yards farthe... 15 yards farthe... 5 yards farther... 10 yards farthe... 15 yards farthe... 5 yards farther... 10 yards farthe... 15 yards farthe... Total 300 0.263 0.401 0.436 0.234 0.340 0.431 0.091 0.206 0.266 0.083 0.180 0.216 25.644 6 p < .01 0.292 0.231 0.132 478.331 3 p < .01

315

Interaction Chi-Square D.F. Significance Brand x Price Total Respondents High-Flyer Pro... High-Flyer Pro... High-Flyer Pro... High-Flyer Pro... Magnum Force, ... Magnum Force, ... Magnum Force, ... Magnum Force, ... Eclipse+, by G... Eclipse+, by G... Eclipse+, by G... Eclipse+, by G... Long Shot, by ... Long Shot, by ... Long Shot, by ... Long Shot, by ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ...

Total 300 0.549 0.394 0.332 0.204 0.477 0.323 0.323 0.216 0.292 0.232 0.144 0.080 0.284 0.202 0.124 0.039 58.408 9 p < .01

Interaction Chi-Square D.F. Significance Performance x Price Total Respondents Drives 5 yards... Drives 5 yards... Drives 5 yards... Drives 5 yards... Drives 10 yard... Drives 10 yard... Drives 10 yard... Drives 10 yard... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ...

Total 300 0.281 0.193 0.114 0.079 0.400 0.326 0.264 0.130

316

SSI Web v7
Drives Drives Drives Drives 15 15 15 15 yard... yard... yard... yard... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... 0.520 0.353 0.312 0.185 20.754 6 p < .01

Interaction Chi-Square D.F. Significance None Total Respondents None chosen:

Total 300 0.211

At first, this report may seem overwhelming, so we'll break it up and discuss it in pieces. First, Counts reports that 300 respondents were used. Then, the count proportions for brand name are displayed:
High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc. Long Shot, by Performance Plus 0.367 0.336 0.189 0.159

High-Flyer Pro was the most preferred ball name and brand on average, being chosen 36.7% of the times that is was presented and available for choice. Your client's ball name and brand, Long Shot, by Performance Plus, was the least preferred at 15.9%. If your client markets their ball under the Eclipse+ name with the Golfers, Inc. brand, choice probability improves from 15.9% to 18.9%. These are ratio quality data, so one might infer from the counts that using the Golfers, Inc. name and brand increases the probability of choice by 19% (18.9/15.9 - 1). Next comes the counts for Performance:
Drives 5 yards farther than the average ball Drives 10 yards farther than the average ball Drives 15 yards farther than the average ball 0.168 0.282 0.339

Respondents on average chose balls that flew 15 yards farther more than twice as often as those that flew 5 yards farther. There appears to be a non-linear effect of performance on choice probability. Likelihood of choice nearly doubles as performance increases from +5 yards farther to +10 yards farther than the average ball. Much less is gained in terms of choice probability by increasing flight from +10 yards to +15 yards. A preliminary conclusion might be that your client should make sure their ball is rated to travel at least 10 yards farther than the average ball. But improving the performance to travel 15 yards farther than the average ball might not be worth the extra manufacturing cost, if that cost is significant. Last comes price:
$4.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls 0.397 0.292 0.231 0.132

As expected, respondents prefer lower prices over higher ones. Probability of choice decreases monotonically for each step increase in price. To this point, you've only analyzed one-way (main effect) effects of attribute levels on choice. You can further consider the probabilities of choice when a combination of two attribute levels are available for choice. There are three tables of two-way probabilities in our study. Rather than look at all three, let's examine the table (brand x price) that appears (by the Chi-Square statistic) to show the most promise of being interesting (and reflect a potentially significant interaction effect) for our study:

CBC Help
High-Flyer Pro... High-Flyer Pro... High-Flyer Pro... High-Flyer Pro... Magnum Force, ... Magnum Force, ... Magnum Force, ... Magnum Force, ... Eclipse+, by G... Eclipse+, by G... Eclipse+, by G... Eclipse+, by G... Long Shot, by ... Long Shot, by ... Long Shot, by ... Long Shot, by ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... $4.99 for package of 3... $6.99 for package of 3... $8.99 for package of 3... $10.99 for package of ... 0.549 0.394 0.332 0.204 0.477 0.323 0.323 0.216 0.292 0.232 0.144 0.080 0.284 0.202 0.124 0.039

317

This table shows the probability of choice for each brand when it was shown at each price. This table is more involved than the simpler one-way tables you examined. Therefore, it may be helpful to plot the results with a graphics or spreadsheet package. If you plot price on the x-axis and probability of choice on the y-axis, it would appear like the familiar demand curves one learns about in economics. We should note, however, that counting analysis has some drawbacks and inaccuracies associated with it. There are more accurate ways to generate demand curves with CBC using the market simulator. Also, demand curves from CBC assume perfect information, equal distribution, and other assumptions mentioned in the online Help. The "pseudo demand curves" seem to suggest that the Eclipse+ name is preferred over all levels of price to the Long Shot name. Also, the gap appears to widen slightly as prices increasethough we cannot tell from this chart whether that is a significant or a chance occurrence. After you have spent some time looking at the choice results for the randomized tasks, you might consider also looking at the results for the fixed holdout tasks. Recall that we asked these fixed scenarios to gauge preference for what may play out as actual future market scenarios. We also asked these fixed questions so that we could see how well the market simulator predicts the preferences for those two questions. Recall that the first holdout task was asked as the seventh task, and the second holdout as the ninth. To analyze responses for the first holdout, you need to use the Counts program again, this time isolating only the seventh choice task. To do that, while in Counts, in the Choice Tasks to Include Area, un-check the All Random option, then click the Filter... button. Un-check all of the random choice tasks (by clicking the boxes in front of the task labels), and select just the first fixed holdout choice task, called Fixed_T1. Click OK to close the dialog. You will only be interested in one-way (Main Effect) counts, so in the Level of Analysis area, un-check the 2-Way Interactions selection (leaving only Main Effect checked). Click Compute!. The Counts for brand are displayed:
High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc. Long Shot, by Performance Plus 0.307 0.213 * 0.237

Recall that we did not display Eclipse+ in the first holdout task, so it is marked with an asterisk. For review, here is the configuration for the first fixed holdout task, with the choice probabilities listed:

318

SSI Web v7 30.7% 23.7% 21.3% Magnum Force, by Durango Drives 5 yards farther than the average ball $8.99 for package of 3 balls 24.3% None: I Wouldn't Purchase Any of These

High-Flyer Pro, by Smith Long Shot, by and Forester Performance Plus Drives 15 yards farther than the average ball Drives 10 yards farther than the average ball

$10.99 for package of 3 $6.99 for package of 3 balls balls

If you isolate only the second fixed holdout task (by selecting only Fixed_T2 in the Choice Tasks to Include dialog) and click Compute!, you'll get the following probabilities for the second holdout task: 18.0% Magnum Force, by Durango Drives 5 yards farther than the average ball $8.99 for package of 3 balls 28.7% Eclipse+, by Golfers, Inc. Drives 10 yards farther than the average ball $6.99 for package of 3 balls 29.6% 23.7%

High-Flyer Pro, by Smith None: I Wouldn't and Forester Purchase Any of These Drives 15 yards farther than the average ball $10.99 for package of 3 balls

Each of these choice probabilities are based only on 1 task x 300 respondents = 300 total choice tasks, whereas the previous counting data was based on a total of 15 tasks x 300 respondents = 4,500 total tasks. In accordance with the previous conclusion we made based on the random choice tasks, the holdout data suggest that there is marginal benefit from using the Eclipse+, by Golfers, Inc. name. As a final note on counting analysis, one should not put too much emphasis in the None percentage. We shouldn't conclude that since the None percentage is running at about 25% that roughly threequarters of these respondents would be expected to purchase a premium ball for their next golf outing. It is our experience that respondents tend to exaggerate their likelihood to purchase or choose products in survey research. The actual proportion of buyers that purchase premium balls would probably be significantly less. Now that you have spent some time learning about the Choice results using the Counts program, you are ready to run logit analysis and construct a market simulation.

Analyzing the Choice Data Using Logit

Aggregate logit is the method provided in the base CBC license for analyzing choice results. There are great advantages to Latent Class and Hierarchical Bayes analysis programs, but for simplicity we'll focus on logit for this tutorial. Logit analysis estimates an effect, or logit "utility" for each level of each attribute. It also can be used to estimate interaction effects. A utility refers to a degree of worth or preference for a product feature. As with any complex analytical tool, we suggest you learn as much about the technique as you can, to ensure that you interpret the results wisely. Much more information about logit analysis and logit utilities can be found in the online help.

CBC Help

319

To compute logit, select Analysis | Compute Utilities. First click the Settings button. You should provide the following settings:
Respondents to Include Respondent Weights Choice Tasks to Include Effects Coding Output Precision (All) (Equal) (All Random) (Main) (5 decimal places)

After you have checked these settings and closed the Logit Settings dialog, click Compute!. The following report is displayed:
Main Effects Choice Tasks Included: All Random Total number 1 2 3 NONE of choices in each response category: 1135 25.22% 1377 30.60% 1039 23.09% 949 21.09%

Files built for 300 respondents. There are data for 4500 choice tasks. Iter Iter Iter Iter Iter 1 2 3 4 5 log-likelihood log-likelihood log-likelihood log-likelihood log-likelihood = = = = = -5517.00384 -5498.70622 -5498.66372 -5498.66372 -5498.66372 rlh rlh rlh rlh rlh = = = = = 0.29335 0.29466 0.29466 0.29466 0.29466

Converged. Log-likelihood for this model = Log-likelihood for null model = -5498.66372 -6238.32463 -----------Difference = 739.66091 Std Err 0.03219 0.03213 0.03728 0.03878 0.03006 0.02652 0.02589 0.03219 0.03350 0.03527 0.04092 0.03825 t Ratio 16.42166 11.56654 -10.12212 -13.48669 -16.41310 4.87399 14.06139 20.75429 4.92893 -2.76111 -17.98437 -0.73693

Chi Square =

1479.322

1 2 3 4 5 6 7 8 9 10 11 12

Effect 0.52865 0.37167 -0.37734 -0.52298 -0.49336 0.12924 0.36411 0.66815 0.16510 -0.09740 -0.73585 -0.02819

Attribute Level 1 1 High-Flyer Pro... 1 2 Magnum Force, ... 1 3 Eclipse+, by G... 1 4 Long Shot, by ... 2 1 Drives 5 yards... 2 2 Drives 10 yard... 2 3 Drives 15 yard... 3 3 3 3 1 2 3 4 $4.99 for pack... $6.99 for pack... $8.99 for pack... $10.99 for pac...

NONE

The column labeled "Effect" contains the utilities for each level of each attribute. The larger the utility, the more preferred the level. The utilities sum to 0 within each attribute (they are zero-centered). These utilities are used within CBC's market simulator to compute interest (share of choice) among products in competitive scenarios. The next column displays the standard errors for each logit effect. Earlier under Counting analysis, we saw that Eclipse+ was preferred to Long Shot, but we could not tell at that point if the result was statistically significant. Here, we note that the utility for Eclipse+ is 0.14564 utility points higher than Long Shot (-0.37734 - -0.52298). We also note the standard errors for each of these utilities. The pooled standard error for the difference between these two utilities is equal to the square root of the

320

SSI Web v7 sum of the squared standard errors, or sqrt (0.03728 + 0.03878 ) = 0.0537912. To get the t-value, we divide the difference in the two utilities by the pooled standard error 0.14564 / 0.0537912 = 2.708. If we look this up on a standard t-table, we see that this difference is significant at roughly the 99% confidence interval. Therefore, we conclude that the Golfers, Inc. name is more preferred than the Performance Plus name. The logit run above reflects main effects only (no interactions). The results from Counts suggested that interaction effects might be significant. Adding interaction terms to the logit model (and the resulting simulator) might significantly improve our ability to predict respondent choices. We will not go into detail in this tutorial regarding how to test for significance of interaction terms in logit. To briefly summarize the process, since we believe that changes in price may act differently on different brands, we decide to rerun the logit model specifying the brand x price interaction. We compare the overall fit (log likelihood) of the model before and after the inclusion of the interaction terms. With this data set, we find that the interaction term improves the fit. To include the interaction between brand and price in your logit run, click Analysis | Compute Utilities. Click the Settings... button. Make sure your settings are the same as previously outlined in this section. However, instead of checking Main Effects, un-check it and then click Effects... and a grid is displayed. Checks are displayed along the diagonal of the grid, representing main effects only (no interactions). To specify the interaction between brand and price, additionally check the box in the lowest cell of the left-most column. Click OK. You are returned to the Logit Settings dialog. Click OK again to close the dialog. Then, click Compute!. Specify a name by which to reference that run, such as "Main Effects + A1xA3". Click OK. A new solution is computed including main effects and interactions (it may take a few minutes).
2 2

Running Market Simulations

Once you have computed logit utilities, you can use those within a market simulator to predict choices for any combination of products defined using the attributes and levels you measured. You can test what-if scenarios, conduct sensitivity analysis, and predict interest in different product concepts in competitive scenarios. To open the Market Simulator within SMRT, click Analysis | Market Simulator or double-click Market Simulator from the study Navigator window. The logit runs that you have saved are displayed in the Utility Runs list. If you saved a run (or multiple runs) while working through the last section of the tutorial, these should appear on the list. The first time you enter the Market Simulator for a study, the list of Simulation Scenarios is empty. Create a simulation scenario by clicking Add.... When you click the Add... button, the Scenario Specification dialog is displayed. Recall that one of the reasons for including fixed holdout tasks in your survey was to check the accuracy of the market simulator. We use the simulator to predict the outcome of a choice task that wasn't included when estimating the attribute utilities. To check this, you decide to specify the first fixed product scenario. The first step is to type a scenario name into the Name field, in the upper left-hand corner of the dialog. Type Fixed Holdout Task 1. Next, you'll specify the three products that were included in that choice task. The area you type the products into looks like the grid of a spreadsheet. The first column is where you type the product label.

CBC Help

321

As you click the various columns associated with the attribute levels, the level codes will appear in the window below to remind you what codes are associated with each attribute's levels. The attribute level codes for the three products that were shown in the first holdout fixed task were: Product Name High-Flyer Pro Long Shot Magnum Force Brand 1 4 2 Performance Price 3 2 1 4 2 3

Specify these three products in the product entry grid. To add new rows to the grid (for additional products) click the Insert Product button. After you have specified these three products, there are a few other things we should do before running the simulation. First, we need to specify a simulation method. For the purposes of this tutorial, the method we'll select is called Share of Preference. This method works well when the products that are specified are basically unique, with little or no overlap in attribute levels used to define those products. The three products we entered are unique in terms of brand, performance and price, so our situation seems to fit this well. If some products shared a level specification (e.g. having the same performance), we would probably favor using the default Randomized First Choice method. Next, we need to tell the simulator to estimate a share of preference associated with the None product. (We generally suggest not estimating share of preference for the None product; but in this case, where we want to predict responses to a holdout task that included a None choice, it makes good sense). To do so, click the Advanced Settings... button. (The settings on this dialog are described in more detail in online Help). Specify a "None" Weight of 1, and click OK to close the dialog. Click OK again to close the Scenario Specification dialog and return to the main Market Simulator dialog. Choose a utility run to use for your simulation. You should have saved a run that computed main effects plus the interaction between brand and price. Select that run by highlighting it in the list of Utility Runs. To simulate shares for a scenario you have specified, place a check mark in the box next to the scenario name in the Simulation Scenarios list, and then click Compute!. The following report is displayed (we've inserted commentary within brackets):
Scenario: Fixed Holdout Task 1 Utility Run: Average Utility Values Rescaling Method: High-Flyer Pro, by Smith and Forester Magnum Force, by Durango Eclipse+, by Golfers, Inc. Long Shot, by Performance Plus Drives 5 yards farther than the average ball Drives 10 yards farther than the average ball Drives 15 yards farther than the average ball Main Effects + A1xA3 Diffs Total 47.62 36.42 -32.49 -51.56 -41.06 10.52 30.54

322

SSI Web v7
$4.99 for package of 3 balls $6.99 for package of 3 balls $8.99 for package of 3 balls $10.99 for package of 3 balls High-Flyer Pro... High-Flyer Pro... High-Flyer Pro... High-Flyer Pro... Magnum Force, ... Magnum Force, ... Magnum Force, ... Magnum Force, ... Eclipse+, by G... Eclipse+, by G... Eclipse+, by G... Eclipse+, by G... Long Shot, by ... Long Shot, by ... Long Shot, by ... Long Shot, by ... None 59.03 18.33 -7.17 -70.19

$4.99 for package of 3... -4.60 $6.99 for package of 3... -6.50 $8.99 for package of 3... 0.75 $10.99 for package of ... 10.35 $4.99 for package of 3... -11.64 $6.99 for package of 3... -20.31 $8.99 for package of 3... 1.08 $10.99 for package of ... 30.87 $4.99 for package of 3... -2.01 $6.99 for package of 3... 14.19 $8.99 for package of 3... -5.78 $10.99 for package of ... -6.40 $4.99 for package of 3... 18.24 $6.99 for package of 3... 12.61 $8.99 for package of 3... 3.96 $10.99 for package of ... -34.82 -0.38

<<The above average utilities are rescaled logit utilities using the zero-centered "diffs" method. The diffs method rescales utilities so that the total sum of the utility differences between the worst and best levels of each attribute across attributes (main effects) is equal to the number of attributes times 100. Note: the attribute utilities are influenced by the number of respondents in the simulation and respondent weighting, but are not affected by the product specifications you enter. After you have seen these once for a particular group of respondents, you may choose to omit them in subsequent simulations by un-checking the Display Utilities box in the Scenario Specification dialog.>>
Product Simulation Settings Simulation Mode: Simulation Model: Share of Preference None Weight: 1 Exponent: 1 Product Specifications High-Flyer Pro Long Shot Magnum Force Brand 1 4 2 Perf... Price 3 4 2 2 1 3

<<Above are the product level codes you specified for the three products in this simulation. Below are the simulated shares of preference (choice) for these products.>>
Shares of Preference for Products Total High-Flyer Pro 30.99 Long Shot 22.16 Magnum Force 22.00 None 24.85

In the table below, we've summarized the actual and predicted choices shares for the first holdout task. Notice that the simulated shares of preference are very similar to the actual share of choices respondents gave to fixed holdout task number one (which we analyzed earlier using Counts):

CBC Help Product Name High-Flyer Pro Long Shot Magnum Force None Actual Choice Shares 30.67 23.67 21.33 24.33 Simulated Choice Shares 30.99 22.16 22.00 24.85 Absolute Error 0.32 1.51 0.67 0.52

323

To quantify how closely the simulated shares match the actual choice shares, we've computed the absolute value of the difference between the actual and simulated shares in the last column. If we average the errors in the last column, we find the Mean Average Error (MAE) is 0.76. (We should note that the MAE for this example is much lower than is generally observed when comparing simulation predictions versus actual holdout shares). We won't take the space here to show the MAE computation for holdout task #2, though you may decide to specify another simulation scenario and compute predicted shares for that task. You can use MAE (or Mean Squared Error or Chi Square) to compare the results of different simulation models (i.e. Logit versus Latent Class or HB). You can also use it to tune the scaling parameter (exponent) or other advanced settings covered in more depth in the online help. The Market Simulator is a powerful tool for testing nearly an unlimited number of possible market scenarios. You can use the simulator to answer the strategic questions related to the fictitious golf ball situation posed at the beginning of this unit. For example, the main question facing Performance Plus was whether they could hope to compete with the two existing performance balls in this market. The simulation results suggest that, if they were able to provide the specified performance at the simulated price and could achieve the level of distribution and awareness of the existing brands (whose specifications and prices didn't change), they could expect a market share roughly equal to Magnum Force, and lower than the leader, High-Flyer Pro. A secondary question was whether they should market their new golf ball under the "Eclipse+, by Golfers Inc." name. If you specify the second holdout task as a new simulation scenario, the following shares result (displayed next to the previous simulation for fixed holdout task #1):
Simulated Shares of Choice Fixed Task 1 High-Flyer Pro Long Shot Magnum Force None 30.99 22.16 22.00 24.85 Fixed Task 2 High-Flyer Pro Eclipse+ Magnum Force None 29.20 26.65 20.73 23.42

By marketing their ball under the "Eclipse+, by Golfers, Inc." name instead of "Long Shot, by Performance Plus", share of choice increases from 22.16 to 26.65, representing a 20% relative increase. If you have a good memory, you'll note that this conclusion is similar to what we inferred from Counts data, and by comparing the actual choices of the two fixed holdouts. The simulated shares, however, should be both more reliable and more accurate. They are not subject to some inherent weaknesses of counting analysis, and they leverage much more data (15 randomized tasks) than isolating only the choices given to the two fixed holdout tasks. It is important to remember that the shares of preference resulting from conjoint predictions are not equivalent to actual market shares, and often look quite different. Many other factors in the real world factor into market shares and cannot be measured and reflected solely by conjoint data. Conjoint

324

SSI Web v7 assumes perfect information, equal distribution and availability, and that each respondent is in the market and able to purchase. Conjoint results reflect the potential market acceptance, given proper promotion, distribution and time.

CBC Help

325

2.2
2.2.1

Designing CBC Studies

CBC Questionnaires and Designs
CBC creates Web-based, CAPI, or paper-and-pencil interviews. CBC comes with a 50-question CiW system, that provides the capability of asking up to 50 other standard survey questions (e.g. numerics, open-ends, check-box). One can purchase additional capacity for standard survey questions by purchasing a larger CiW license.

The Choice Question

A CBC question is often referred to as a "task." A set of products (concepts) are displayed on the screen, and in the standard formatting the respondent chooses among the concepts. For example:

Choosing among concepts is the standard approach that is advocated by most experts in the industry. Constant-sum (allocation), where respondents type a number beneath concepts based on relative preference, percent of volume, or volume is also offered in CBC. Another format that is gaining popularity is to ask about the "none" concept in a second-stage question, called Dual-Response None. Two important decisions to make when constructing choice tasks are 1) how many concepts to present per task, and 2) how many total tasks to ask. These are sometimes difficult issues to answer, and ongoing research is helping to define guidelines and understand the consequences. From a statistical viewpoint, choice tasks are not a very efficient way to learn about preferences. Respondents evaluate multiple concepts, but only tell us about the one they prefer. We don't learn how strong that preference is relative to the other product concepts. Showing more product concepts per screen increases the information content of each task. Recent research has shown that respondents are quite efficient at processing information about many concepts. It takes respondents considerably less than twice as long to answer choice tasks with four concepts as with two concepts. Recent research has also pointed out the value of forcing at least some levels within attributes to repeat within each choice task (level overlap). This can be accomplished by showing more products on the screen than levels specified per attribute. In general, we recommend showing around four to six concepts per task. However, there may be instances (e.g. a beverage study with 25 brands plus price), where showing more product concepts per screen is appropriate and more realistically portrays the actual buying situation. With the Advanced Design Module for CBC, you can display up to 100 product graphics on the screen, presented as if they were resting on "store shelves."

326

SSI Web v7 With randomized choice designs, given a large enough sample size, we could model preferences at the aggregate level by asking each respondent just one choice task. In practice, researchers recognize that individuals are expensive to interview and that it makes sense to collect more information from each person in CBC studies. With multiple observations per respondent, one can model across-respondent heterogeneity in preferences, which leads to more accurate choice simulators. In a meta-analysis of 21 CBC data sets, we found that multiple observations per respondent are quite valuable, and that respondents can reliably answer up to at least 20 questions, and perhaps even more. However, we should point out that respondents process earlier tasks differently from later questions. Respondents pay more attention to brand in the first tasks, and focus more on price in later questions. (See Johnson and Orme's article entitled "How Many Questions Should You Ask in Choice-Based Conjoint?" available for downloading from the Technical Papers section of our home page: www.sawtoothsoftware.com/techpap.shtml). With CAPI or paper-based CBC research, we recommend asking somewhere in the range of 12 to 20 choice tasks. With Web interviewing, fewer tasks might be appropriate if there is an opportunity for increased sample sizes (which often is the case). If estimating individual-level utilities using CBC/HB, we'd recommend at least six choice tasks to achieve good results based on simulated shares, but about 15 choice tasks or more for developing robust predictions at the individual level (again assuming a typical design, and a healthy degree of sample heterogeneity). Before finalizing the number of concepts or tasks to be asked, we urge you to pretest the questionnaire with real respondents to make sure the questionnaire is not too long or overly complex. With CBC, all of the tasks are tied to the same layout, including header text and footer text. You cannot change the number of product concepts or other settings within your choice tasks midstream. In addition to the regular choice tasks, you can add tasks that have fixed designs ("holdout" tasks) described later. You can also add other text screens or generic survey questions between choice tasks.

The "None" Option

Choice-based conjoint questions can be designed to have a "None" option, sometimes referred to as the constant alternative. It is argued that the None option in CBC tasks better mimics the real world, since buyers are not required to choose products that don't satisfy them. The None option can also be used to reflect a status quo choice, such as "I'd continue to use my current long-distance service provider." CBC gives you the option of including a None option in the questionnaire. If the None option is present, a separate logit utility weight is computed for the None parameter (if using logit analysis). The more likely respondents are to choose None relative to product concepts, the higher the utility of the None option. The None parameter can be used in market simulations to estimate the proportion of respondents that would not choose any of the simulated product concepts. You can include more than one constant alternative in the questionnaire if using the CBC Advanced Design Module. In general, we recommend including None in CBC questionnaires, but paying less attention to (or completely ignoring) it in market simulations. The propensity to choose None can be a telling measure when comparing groups or individuals. We suggest that percent of Nones in Counts or simulations be viewed in a relative rather than an absolute sense. The use of the "None" concept in partial-profile CBC studies (only supported by the Advanced Design Module for CBC) is problematic. The None weight varies significantly depending on how many attributes are displayed in the partial-profile task. Patterson and Chrzan (2003) showed that as the number of attributes increases, the propensity to choose None also increases.

CBC Help

327

Randomized Versus Fixed Designs

Two schools of thought have developed about how to design and carry out choice-based conjoint studies. 1) Some researchers prefer fixed orthogonal designs. Such designs often employ a single version of the questionnaire that is seen by all respondents, although sometimes respondents are divided randomly into groups, with different groups receiving different questionnaire versions (blocks, or subsets of the larger fixed design). Fixed orthogonal designs have the advantage of high efficiency at measuring main effects and the particular interactions for which they are designed. It is interesting to note that for imbalanced asymmetric designs (where there are large differences in the numbers of levels among attributes) fixed orthogonal designs often can be less efficient than random designs. 2) Other researchers, particularly those accustomed to computer-administered interviewing, prefer random designs. The CBC software, for example, can design interviews with nearly-orthogonal designs, in which each respondent receives a unique set of questions. Attribute and concept order can be randomized across respondents. Such designs are often slightly less efficient than truly orthogonal designs (but can be more efficient with asymmetric designs), but they have the offsetting advantage that all interactions can be measured, whether or not they are recognized as important at the time the study is designed. Randomized plans also reduce biases due to order and learning effects, relative to fixed plans. CBC can administer either fixed, random designs, or designs generated from an outside source (via the ability to import a design from a .csv file). If a fixed design is chosen, then the researcher must specify that design. If a randomized design is chosen, then it will be produced automatically and saved to a design file that is later uploaded to your server if using Web-based data collection, or to another PC if using CAPI-based interviewing. If using paper-based interviewing, a limited set of questionnaire versions (a subset of the potential random plan) is often used. Most CBC users have favored the ease of implementation and robust characteristics of the randomized approach. We have tried to make CBC as easy to use and automatic as possible. The researcher must decide on the appropriate attributes and their levels, and compose whatever explanatory text and/or generic survey questions are desired during the interview. Absolutely no "programming" is involved. Every aspect of designing the interviews and conducting the analysis is managed through the point-and-click Windows interface. Thus, we hope that CBC will make choice-based conjoint analysis accessible to individuals and organizations who may not have the statistical or internet expertise that would otherwise be required to design and carry out such studies.

Random Design Strategies

Fixed experimental designs may be specified in CBC, but most users will rely on one of the four randomized design options. When CBC constructs tasks randomly, some efficiency is often sacrificed compared to strictly orthogonal designs of fixed tasks. But, any loss of efficiency is quite small, usually in the range of 5 to 10%. However, there are important compensating benefits: over a large sample of respondents, so many different combinations occur that random designs can be robust in the estimation of all effects, rather than just those anticipated to be of interest when the study is undertaken. Also, potential biases from learning and order effects can be reduced. The earliest versions of CBC offered two randomized design options: complete enumeration and the shortcut method. Though we refer to these as "randomized designs," these designs are chosen very carefully, as will be explained. The complete enumeration and shortcut methods generate designs conforming to the following

328

SSI Web v7 principles: Minimal Overlap: Each attribute level is shown as few times as possible in a single task. If an attribute's number of levels is equal to the number of product concepts in a task, each level is shown exactly once. Level Balance: Each level of an attribute is shown approximately an equal number of times. Orthogonality: Attribute levels are chosen independently of other attribute levels, so that each attribute level's effect (utility) may be measured independently of all other effects. Minimal overlap has benefits from a statistical standpoint in terms of increasing precision for main effects; but it is suboptimal for measurement of interactions between attributes. Also, we've recently recognized that including level overlap can lead to more thoughtful responses that reveal deeper preference information than minimal overlap. To help illustrate that point, consider the extreme case of a respondent who requires Brand A. If only one Brand A is available per task, the respondent's only option is to choose Brand A each time, and such questions are completed in a matter of a few seconds. But, if two Brand A products sometimes appear in the design, then the respondent is encouraged to ponder and express what additional aspects (other than brand) affect the decision. Level overlap can be added to designs simply by increasing the number of concepts per task. For example, if each attribute has four levels, then showing six concepts per task means that two levels must be repeated in each choice task. Six concepts per task also provides more raw information for estimating part-worth utilities than four concepts per task. (Yet another reason to advocate showing more rather than fewer concepts per task.) Another way to add level overlap within your designs is to use the Balanced Overlap or purely Random methods. Below, we describe the design methods in more detail, and conclude with more advice on when to use the different design approaches. Complete Enumeration: The complete enumeration strategy considers all possible concepts (except those indicated as prohibited) and chooses each one so as to produce the most nearly orthogonal design for each respondent, in terms of main effects. The concepts within each task are also kept as different as possible (minimal overlap); if an attribute has at least as many levels as the number of concepts in a task, then it is unlikely that any of its levels will appear more than once in any task. Complete enumeration may require that a very large number of concepts be evaluated to construct each task, and this can pose a daunting processing job for the computer. The base CBC software permits up to 10 attributes, with up to 15 levels each. Suppose there were 4 concepts per task. At those limits the number of possible concepts to be evaluated before displaying each task would be 4 x 15^10 = 2,306,601,562,500! This is far too great a burden for even the fastest computers today, and would take a great deal of time for the design generator to produce a design file. Generally, CBC design specifications in practice are more manageable and speed of computation is usually not an issue. Furthermore, with CBC, the researcher generates the experimental plan prior to conducting data collection. Thus, respondents do not need to wait for a new design to be generated--they are simply randomly assigned to receive one of the many questionnaire versions previously generated. The time required to compute designs under Complete Enumeration is more sensitive to the number of attributes in your study than to the number of levels for attributes. If you have just a few attributes, you should experience little delay in design computation, even if one of your

CBC Help attributes has scores of levels. Shortcut Method:

329

The faster "shortcut" strategy makes a much simpler computation. It attempts to build each concept by choosing attribute levels used least frequently in previous concepts for that respondent. Unlike complete enumeration, that keeps track of co-occurrences of all pairs of attribute levels, the shortcut strategy considers attributes one-at-a-time. If two or more levels of an attribute are tied for the smallest number of previous occurrences, a selection is made at random. With the shortcut method, as well as with complete enumeration, an attempt is made to keep the concepts in any task as different from one another as possible (minimal overlap). When there is more than one less-frequently-used level for any attribute, an attempt is made to choose one that has been used least in the same task. Designs composed using complete enumeration are of high quality, and those composed by the shortcut method are also quite acceptable. Balanced Overlap Method: This method is a middling position between the random and the complete enumeration strategies. It permits roughly half as much overlap as the random method. It keeps track of the co-occurrences of all pairs of attribute levels, but with a relaxed standard relative to the complete enumeration strategy in order to permit level overlap within the same task. No duplicate concepts are permitted within the same task. The balanced overlap method takes about the same time to compute as complete enumeration, and is slower than the random or shortcut methods. Random Method: The random method employs random sampling with replacement for choosing concepts. Sampling with replacement permits level overlap within tasks. The random method permits an attribute to have identical levels across all concepts, but it does not permit two identical concepts (on all attributes) to appear within the same task. The random method computes about as quickly as the shortcut method. Unless the primary goal of the research is the study of interaction effects, we generally do not recommend using the purely Random method. Recent research suggests that including level overlap in CBC designs is valuable. One can force some degree of level overlap even though using Complete Enumeration or Shortcut methods simply by increasing the number of concepts per task (to exceed the number of levels per attribute). If you cannot increase the number of concepts per task, balanced overlap directly adds a modest degree of level overlap and may generally perform a bit better than complete enumeration. Even though Complete Enumeration is the default in the current release of the software, we will probably change the default to be Balanced Overlap in the next major update. We strongly encourage CBC users to use Balanced Overlap, and/or to achieve more level overlap in their CBC questionnaires by displaying more concepts per task than the largest number of levels in any one attribute.

CBC's Design File

CBC automatically creates a design file that is a) uploaded to the server if using web-based data collection,

330

SSI Web v7 b) is copied to other PCs if using CAPI-based data collection, or c) is used to generate a questionnaire file if using paper-based data collection. During the process, you are asked to specify how many versions of the questionnaire should be written to that design file. For computer-based interviewing, we generally recommend that you include at least 100 versions, as there are benefits to fielding many versions of the questionnaire. Optimally, each respondent would receive his/her own version of the questionnaire. However, there are diminishing returns and including more than a few dozen versions often is of very little to no additional practical benefit. For that reason, we limit the number of versions that can be placed in the design file to 999. If you have more than 999 respondents, this means that some respondents will by necessity receive the same designs. For all practical purposes, once there are so many unique versions of the questionnaire in the total design pool, this poses no harm.

Suggestions for Randomized Designs

Generally, we recommend asking about 12 to 18 choice tasks, though this recommendation may vary depending on the attribute list and sample size. We prefer random task generation. Also, we caution against questionnaires with no level overlap. If a respondent has a critical "must-have" level, then there is only one product per choice task that could satisfy him/her. One can accomplish some level overlap by simply displaying more product concepts than levels exist per attribute. The "Balanced Overlap" design methodology may also be used. Another reason to include at least some degree of overlap in the CBC designs is when interaction terms are of interest. Overlap for an attribute can be added to a design simply by using more concepts than attribute levels in tasks. In summary, we suggest using complete enumeration (or its sister shortcut method) for main-effects only designs. If detecting and measuring interactions is the primary goal (and sample size is relatively large), then the random approach is favored. If the goal is to estimate both main effects and interactions efficiently, then overlap should be built into the design, at least for the attributes involved in the interaction. Using more concepts than attribute levels with complete enumeration, or utilizing the compromise balanced overlap approach would seem to be good alternatives.

Importing Other Designs

Some researchers will receive a specific design generated by a design specialist. You can import such designs into CBC.

Mixed Overlap Designs

Some researchers may want to force some attributes to have minimal overlap and other attributes to have some overlap (i.e. using Balanced Overlap). Imagine a situation in which the first attribute is to have minimal overlap (each level represented once per task) and the remaining attributes are to be designed using Balanced Overlap. The researcher could generate a design using all but the first attribute under Balanced Overlap, and export the design to a .csv file. Then, the researcher could modify the .csv file using Excel to insert an attribute in the first position where each level appears once per task. The last step is to import that new design into a CBC study that reflects all attributes. One could use a similar approach to combine designs where multiple attributes were generated using Balanced Overlap and multiple attributes were generated using Complete Enumeration. Of course, care should be taken to test the design efficiency of the final design.

CBC Help

331

2.2.2

Defining Attributes and Levels

You are limited to 30 attributes (depending on your license) with at most 15 levels of an attribute, though we encourage you to develop attributes that have about 5 or fewer levels whenever possible. (Hint: you can select a list of attributes or levels from Word or Excel and paste into SSI Web using the "paste from word processor or spreadsheet" icon. This can save a great deal of time.) An attribute is a characteristic of a product (e.g. color) which can take on various levels (e.g., red, yellow, blue). Every attribute must have at least two levels. The underlying theory of conjoint analysis holds that a buyer places a certain part-worth (or utility value) on each attribute level, and that the overall utility of any product is obtained by summing up the part-worth values of its specific attribute levels. In conjoint experiments, we show respondents product concepts described using different combinations of attribute levels and ask them to somehow express their preferences for those concepts. One way of thinking about conjoint analysis is that we are conducting a designed experiment for each respondent to see how preferences vary as we change attribute levels. After observing how respondents evaluate products in response to changes in the underlying attribute levels, we can estimate the impact (part-worth) each attribute level has upon overall product preference. Once we learn respondents' preferences for the various attribute levels, we can predict how buyers might respond to any potential combination of levels in our study, whether or not that actual product was ever displayed during the interview.

Conditional Relationships
(Not available within ACA studies.) The conditional relationships button lets you associate conditional prices or graphics with one or more attributes in your study. Conditional Graphics: Sometimes, you may want to associate a graphic with an attribute or multiple attributes. For example, if a combination of brand and form factor make up a graphic, one can specify the two attributes separately and associating a conditional graphic with their combined levels. This permits estimation of separate effects for brand and form factor.

Guidelines
In generating attributes and levels, consider the following guidelines: 1. Attributes should be independent. It is therefore important to economize; including attributes with overlapping meanings is wasteful and can lead to biased part-worth utilities. Furthermore, levels for related attributes often cannot combine naturally and freely with one another. Though it can lead to more realistic interviews, it is often detrimental (and sometimes fatal) to prohibit levels from occurring with others. 2. Levels within each attribute should be mutually exclusive. This point becomes clear when you specify products using the market simulator (during the analysis phase) and are forced to choose only a single level from each attribute.

332

SSI Web v7 Consider the following attributes for a study on optional automobile features: Optional Features: Sunroof GPS (Global Positioning System) Extended warranty This formulation doesn't permit simulating preference for a car that has both a Sunroof and a GPS. Similarly, we could not simulate preference for an automobile that had none of these features. There are two ways to resolve this quandary: a) Create an attribute with all potential combinations of these features. This results in an eight-level attribute, if you include the option that none of these features is available. b) Formulate three separate attributes each with two levels: (No Sunroof, Sunroof), (No GPS, GPS), (No Warranty, Warranty). Option (a) adds seven parameters to the model and forces the measurement of an explicit three-way interaction. With the more complex model definition, we can investigate whether there are diminishing returns by bundling the features. By splitting the options into three distinct binary attributes (Option b), only three parameters are added to the model; however, interactions are not measured. 3. Attribute levels should cover the full range of possibilities for existing products as well as products that may not yet exist, but that you want to investigate. Although the market simulator allows you to extrapolate and interpolate, only linear interpolation and extrapolation are possible. Although interpolation is likely to produce acceptable results, extrapolation is prone to error and should be avoided. One way to ensure that you are including the appropriate levels and ranges is to ask your client to specify ahead of time the market simulations to be run during the analysis phase of your study. That exercise can often reveal weaknesses in your attribute specifications. 4. Prohibitions, if at all possible, should be avoided. Specifying unnecessary or excessive prohibitions is one of the most common mistakes. The problem usually begins when either the analyst (or the analyst's client) notices that some product combinations displayed during the interview are not realistic, given what currently exists in the market. Sometimes a product is shown with all the best features at the lowest price; or two attribute levels that would not naturally occur in the real world are paired together. The inclination is simply to prohibit such combinations. We urge you to exercise restraint when considering prohibiting levels from occurring with one another. Too many prohibitions can lead to confounded effects and the complete inability to calculate stable utilities. It is better to prompt respondents that they will see combinations during the interview that are not yet available in the market or that seem unlikely. You can urge respondents to answer as if these products were actually available today. There are other strategies for dealing with prohibitions. Consider the example below with brands of soda and package types: Brand: Sawtooth Spritz Kong Kola Splut Package Type: 2-liter bottle 6-pack of 12-oz cans Suppose that Splut was only available in 6-packs of cans. Furthermore, you are displaying actual pictures of the products, and thus can only display actual (not potential) products. Rather than define a

CBC Help

333

prohibition between Splut and the 2-liter bottle, it would make more sense to combine these two attributes as a single attribute with five levels: Sawtooth Spritz in a 2-liter bottle Sawtooth Spritz in a 6-pack of 12-oz cans Kong Kola in a 2-liter bottle Kong Kola in a 6-pack of 12-oz cans Splut in a 6-pack of 12-oz cans Under this strategy, no prohibitions are required. 5. The number of levels you choose to define an attribute can have a significant bearing on the results. The first concern has been called the "Number-of-Levels Effect." All else equal, attributes defined on more levels tend to get more importance. The second concern is that you limit the number of levels on which quantitative attributes are described. We suggest not including more than about five levels to describe attributes such as price or speed. It's usually better to have more data at each price point than to have thinner measurements at more price points. Measuring too many points along a quantitative function can result in troublesome reversals. If you cover the entire range of interest with fewer levels, you can interpolate between levels within the market simulator to get finer granularity if needed. 6. Attributes that cannot be adequately described in words should be represented in multimedia. But if attributes do not require multimedia to adequately communicate their properties, it would probably be a mistake to make them multimedia. Though the interview might appear more attractive, it might bias the results in favor of multimedia attributes.

334

SSI Web v7

2.2.3

Specifying Prohibitions
This dialog lets you specify which combinations of attribute levels (or attributes altogether) may not be shown together within the same product concept in conjoint questions. For standard (within-concept) prohibitions, select an attribute in the upper pane and a second attribute in the lower pane. A grid appears, in which you can check which level combinations should be prohibited. For advanced prohibitions for CBC studies (described below), select either the Advanced Prohibitions or Null Level Prohibitions tabs (requires license to the Advanced Design Module).

A Warning
Prohibitions, if at all possible, should be avoided. Specifying unnecessary or excessive prohibitions is one of the most common mistakes. The problem usually begins when either the analyst (or the analyst's client) notices that some product combinations displayed during the interview are not realistic, given what currently exists in the market. Sometimes a product is shown with all the best features at the lowest price; or two attribute levels that would not naturally occur in the real world are paired together. The inclination is simply to prohibit such combinations. We urge you to exercise restraint when considering prohibiting pairs. Full-profile CBC studies are particularly sensitive to attribute prohibitions. Too many prohibitions can lead to imprecise part-worth estimation. It is better to prompt respondents that they will see combinations during the interview that are not yet available in the market or that seem unlikely. You can urge respondents to answer as if these products were actually available today. If you use any prohibitions, you must make sure to test your design and check the efficiency of the estimates for the parameters of interest.

How Many Prohibitions Can I Safely Add?

A common question that users ask is "how many prohibitions can I specify without seriously damaging the results of my study?" This cannot be answered without more information. For it is not the sheer number of prohibitions but the specific pattern of prohibitions that more directly affects the degree of correlation among the attribute levels in the design matrix, and thus the design efficiency. Assume that the researcher wants to specify 3 prohibitions between a 2-level attribute and a 3-level attribute. There are just 6 possible combinations that can occur when combining those two attributes. If the researcher prohibits 3 of those combinations from occurring, this eliminates half of the possible combinations between those attributes. There would be less damage to the efficiency of the design if three total prohibitions were allocated across three separate attribute combinations (one per pair of attributes). As a side note, the researcher in this instance may decide to specify the three non-prohibited combinations of those two attributes as a single attribute, rather than as two separate (and not independent) attributes. The main drawback of this approach is that after combining these two attributes, the researcher will not be able to compute the attribute importance or the relative partworths of each attribute independently.

CBC Help

335

Advanced Prohibitions (CBC Advanced Design Module Only)

For many years, our CBC software only supported within-concept prohibitions between the levels of pairs of attributes. In other words, within the same product concept, one level of one attribute would be prohibited from appearing with one level of another attribute. For example, we would prohibit Brand A from appearing with the color Green within the same product concept. Starting in v6 of CBC, we now support the following advanced prohibitions: n-way prohibitions between-concept prohibitions null-level prohibitions N-Way Prohibitions You can define prohibitions that simultaneously involve up to n attributes, where n is the number of attributes in your study. For example, a 3-way prohibition involves prohibiting the combination [Brand A, Style C, Green] from appearing (where brand, style, and color are three separate attributes). Between-Concept Prohibitions You can define prohibitions restricting what attribute combinations may occur in competition with another within the same choice task. For example, we may wish to prohibit a [Brand A, Style C] product from being shown within the same choice task in competition with a [Brand A, Style B] product. These may be defined as 1-way to n-way prohibitions. Null-Level Prohibitions Some researchers include "blank" levels within attributes (often by using the HTML syntax &nbsp; for the attribute level text, which displays a blank space on the screen). The null (blank) level reflects the absence of the attribute. However, if many attributes have null levels, the analyst may wish to limit how many of them appear together in their null state. CBC software lets you specify a minimum and maximum number of null levels that are allowed to appear together within a concept (though specifying that the number of null levels permitted should be constant leads to confounded designs and is not permitted). You must inform the software of which attribute levels are null levels. As you place constraints on the design to enforce a narrow range of permitted null levels per concept, the design efficiency is reduced. Furthermore, the greater the null level constraints, the worse that Shortcut, Complete Enumeration, and Balanced Overlap methods will perform relative to the purely Random method. With quite constrained solutions, the Random design method performs best. For this reason, it is very important that you test different design methods and compare the overall efficiency of the designs when using null-level prohibitions. Effect of Prohibitions on Design Efficiency Within-concept prohibitions are generally more damaging to design efficiency (the ability of the questionnaire to estimate the part-worth utility parameters with good precision) than between-concept prohibitions. A higher-order prohibition (for example, a single 3-way, within-concept prohibition) is less damaging to the precision of utility level estimates than a 2-way within-concept prohibition. This follows logically, since a single 3-way prohibition leaves many more attribute level combinations available to display than a single 2-way prohibition, which is more restrictive. Please note that to estimate interactions between two attributes, all possible level combinations of those two attributes must have been displayed within concepts to respondents (across all respondents). Therefore, specifying a 2-way, within-concept prohibition would preclude the ability to estimate an interaction effect between the two attributes involved in that prohibition. However, specifying a single 3-way prohibition would not prohibit estimating 2-way interaction effects (since all 2-

336

SSI Web v7 way combinations of attribute levels would still be represented).

CBC Help

337

2.2.4

Attribute Interactions
Most conjoint methods assume "main effects only" estimation. These are simple additive models in which the value of a product concept is equal to the simple additive sum of its parts (the attribute levels). The part-worths for each attribute are measured independently of all others. Main effects (the independent effect of each attribute) tend to capture the vast majority of variance to be explained in choice data. However, there are instances in which the presence of interactions between attributes makes the simple model not as accurate as it could be. Adding interaction terms to the main effects model, in these instances, could add important explanatory power and improve predictive accuracy. In a Sawtooth Software conference, the following explanation of an interaction was provided, "Interactions occur when the combined effect of two attributes is different from the sum of their two main effect utilities. For example, being stranded on a deserted island is pretty bad, say it has a utility of -40. Attending a party hosted by cannibals is also a bad thing, say with a utility of -50. But attending a party hosted by cannibals on a deserted island could be altogether worse, in grisly sorts of ways (utility -250)." ("An Overview and Comparison of Design Strategies for Choice-Based Conjoint Analysis," Keith Chrzan and Bryan Orme, 2000 Sawtooth Software Conference.) One of the strengths of CBC is its ability to estimate the effects of interactions. We think interactions would be found more often using conjoint-related methods if the sample sizes, experimental designs, and estimation methods allowed us to measure them with more precision. We think that CBC provides an excellent way to produce relatively precise results when attribute interactions are of concern. It can be demonstrated that interaction effects can be revealed through choice simulations using maineffect models that do not directly model interaction terms, if the source of the interactions between attributes is principally due to differences in preference among groups or individuals. If Latent Class or HB are used to model main-effects, the need to additionally model interactions should be lessened. Either way, the randomized choice designs offered in CBC are appropriate for either aggregate or disaggregate analysis; for main-effects or models that involve interactions. To obtain strongest results for interactions, some level overlap should be incorporated into your CBC designs. See the section entitled CBC Questionnaires and Designs for more information. Be careful about adding interaction terms to the model, as they come at a cost and can easily lead to overfitting. In other words, the extra parameters within the model may not be worth the amount of extra fit to the data, and worse predictions may result for market simulations involving new choice scenarios. It would be a mistake to simply add all interaction terms between attributes to the model simply because the analyst wanted to ensure that if interaction effects existed that they were captured.

338

SSI Web v7

2.2.5

Number of Attributes/Levels/Tasks in CBC

With full-profile presentation, respondents see concepts that are described on all the attributes. With full-profile presentation, there is a limit to how many attributes one should include in choice-based conjoint analysis. It is our opinion that concepts described by more than about eight attributes may confuse respondents, and may cause them to respond superficially. But, it is difficult to establish a general limit, since the motivation and knowledge of respondents, the length of the attribute text, and the use of graphics affect how many attributes will work well. If you must study more attributes than respondents can comfortably deal with in a CBC interview, you might consider some other method of conjoint analysis. Adaptive CBC may be a good alternative, since it is possible to drop entirely unimportant attributes from the product profiles. The Advanced Design Module for CBC can create partial-profile designs, where only a subset of the attributes is ever presented within a single choice task. Another conjoint technique, ACA (Adaptive Conjoint Analysis), can also handle larger numbers of attributes.

Number of Attribute Levels in CBC

The base CBC system lets you measure up to 15 levels for any one attribute (though the Advanced Design Module expands that number to 254 levels per attribute). Most projects will probably involve five or fewer levels per attribute, although attributes such as Brand (or SKUs reflecting brand x package combinations) may easily require many more levels for the purposes of a project. For typical CBC studies, it is usually better to have fewer levels on which attributes are described, along with approximate balance in the number of levels across attributes. With packaged-goods and beverage research, it may be reflective of real-world conditions to include dozens of levels to reflect brand or brand x package size combinations, but considerably fewer levels for variations in price. We caution against using more than about five levels to define quantitative functions such as Price or Speed if utilities will be estimated for each discrete level (CBC's standard approach). CBC's market simulator permits interpolation between levels, so many additional points along a function may be estimated. Defining a quantitative attribute on too many levels spreads the data thin and increases the likelihood of getting reversals (out-of-order) utilities that are counterintuitive and problematic in simulations. Constrained estimation (monotonicity constraints) can help out in these situations. However, you can fit linear terms to quantitative attributes. In that case, and if you believe the appropriate function is approximately linear, it may be justified to include more than five levels for quantitative attributes.

Number of Tasks in CBC

Because each respondent can receive a unique set of CBC tasks (questions) under randomized designs, as few as just one task may be asked per respondent while still allowing estimation of partworths. Not surprisingly, most CBC questionnaires include multiple tasks, since it seems a waste not to collect more information from each respondent. With multiple observations per respondent, one can model heterogeneity in preferences, which leads to more accurate choice simulators. In a recent meta-analysis of 21 CBC data sets, we found that multiple observations per respondent are quite valuable, and that respondents can reliably answer up to at least 20 questions, and perhaps even more. However, we should point out that respondents process earlier tasks differently from later questions. Respondents pay more attention to brand in the first tasks, and focus more on price in later questions. (See Johnson and Orme's article entitled "How Many Questions Should You Ask in Choice-Based Conjoint?" available for downloading from the Technical Papers section of our home page: www.sawtoothsoftware.com/techpap.shtml).

CBC Help

339

2.2.6

None Option/Dual-Response None

Choice-based conjoint questions can be designed to have a "None" option, sometimes referred to as the constant alternative. It is argued that the None option in CBC tasks better mimics the real world, since buyers are not required to choose products that don't satisfy them. The None option can also be used to reflect a status quo choice, such as "I'd continue to use my current long-distance service provider." CBC gives you the option of including a None option in the questionnaire. If the None option is present, a separate logit utility weight is computed for the None parameter (if using logit analysis). The more likely respondents are to choose None relative to product concepts, the higher the utility of the None option. The None parameter can be used in market simulations to estimate the proportion of respondents that would not choose any of the simulated product concepts. In general, we recommend including None in CBC questionnaires, but paying less attention to (or completely ignoring) it in market simulations. The propensity to choose None can be a telling measure when comparing groups or individuals. We suggest that percent of Nones in Counts or simulations be viewed in a relative rather than an absolute sense. The use of the "None" concept in partial-profile CBC studies (only supported by the Advanced Design Module for CBC) is problematic. The None weight varies significantly depending on how many attributes are displayed in the partial-profile task. Justification for the None concept refers to vertical justification of the None text within its concept area. Dual-Response None Some researchers have advocated asking the "None" choice as a second-stage question in discrete choice questionnaires (see "Beyond the Basics: Choice Modelling Methods and Implementation Issues" (Brazell, Diener, Severin, and Uldry) in ART Tutorial 2003, American Marketing Association). You can specify the dual-response None layout from the Specify CBC Interview Parameters dialog, on the None Option tab. Using this option does not complicate analysis, as the software handles this more sophisticated application of the None automatically (CBC/HB v4 or later required). The "Dual-Response None" approach is as follows. Rather than ask respondents to choose among, say, four alternatives {A, B, C and None}; respondents are first asked to choose among alternatives {A, B, and C}, and then next asked if they really would buy the alternative they selected in the first stage (yes/no).

340

SSI Web v7

The dual-response None dramatically increases the propensity of respondents to say "None," which many researchers would argue better reflects actual purchase intentions than when the "None" is asked in the standard way. But, no information is lost due to the selection of the "None," since respondents are first asked to discriminate among available products. Thus, the "Dual-Response" none can provide a "safety net": we can estimate a "None" parameter without worrying about the potential decrease in precision of the other parameters if the incidence of "None" usage is quite high. The dual-response None has its drawbacks. It adds a little bit more time to each choice task, since respondents must provide two answers rather than one. But, the additional time requirement is minimal, since respondents have already invested the time to become familiar with the alternatives in the choice task. It is also possible that asking the "None" as a separate question may bias the parameters of interest. Brazell et al. have suggested that the benefits of the dual response seem to outweigh any negatives. They have conducted split-sample experiments with respondents that demonstrate that the parameters (other than the "None") are not significantly different (after adjusting for scale) when using the standard "None" vs. "Dual-Response None" formats. For details regarding estimation of part-worth utility parameters with dual-response None in place, see the CBH/HB manual. Important Note: If using the dual-response None, estimation requires CBC/HB v4 or later (if using Sawtooth Software tools).

CBC Help

341

2.2.7

Specifying Fixed or Holdout Tasks

CBC lets you specify a single or multiple "fixed" tasks. Fixed refers to the fact that every respondent is shown the same choice task, with the product concepts defined in exactly the same way. You must define your own fixed tasks; CBC does not design them for you. (By default, all fixed tasks are initialized to level "1" for each attribute.) Most CBC users will opt for a randomized design, since they are quite efficient, automatic, and permit great flexibility in analysis. Some CBC users with design expertise may choose to implement a fixed design, and this is most easily done by import the design from a .csv file. A fixed design can be slightly more efficient than a randomized design in measuring the particular effects for which it was designed. For most CBC users we recommend using randomized tasks for part-worth estimation, and specifying one or more fixed holdout tasks that are not used for utility estimation. We think it is wise to include holdout choice tasks in conjoint interviews, even though they may not appear to be needed for the main purpose of the study. They almost always turn out to be useful, for these reasons: They provide a proximal indication of validity, measured by the utilities' ability to predict choices not used in their estimation. They provide a check on the scaling of the utilities. If the most popular concepts are overpredicted, then the scale parameter should be reduced. If the predictions are too flat, then the scale parameter should be increased. They permit identification and removal of inconsistent respondents (if using HB). They can be used for testing specific product configurations under consideration. Much value can be added by direct measurement of these concepts. They can be used for testing the accuracy of market simulators. They aid considerably in comparing alternative models (logit, Latent Class, or HB) and choice simulation strategies. (Note: if comparing the ability of different models to predict holdout choices, it is important to adjust the scale parameter to maximize the fit of each model prior to making comparisons.) If holdout concepts have been defined with differing degrees of product similarity, they can be used for tuning the appropriate correction for product similarity in Randomized First Choice modeling. It's hard to design good holdout concepts without some prior idea of respondent preferences. There's no point in asking people to choose among concepts where one dominates in the sense that everyone agrees it is best. And, similarly, it's good to avoid presenting concepts that are equally attractive, since equal shares of preference would be predicted by a completely random simulator. If you present triples of concepts, it's probably best if their shares of choices are somewhere in the neighborhood of 50/30/20. When conducting CBC studies, if you plan to do segmentation with latent class analysis, it's wise to consider the kinds of groups you expect to get, and to design products in holdout choice sets so that one alternative will be much more preferred by each group. This maximizes your ability to confirm the validity of the multi-group Latent Class simulator. If you plan to use the Randomized First Choice simulation model, it is helpful to include holdout tasks that reflect severe differences in product similarity. For example, in a holdout choice set featuring four product alternatives, two products might be identically defined on all except one or two attributes. By including products with differing similarities, the appropriate adjustment for product similarity can be tuned in the Randomized First Choice Model. It isn't necessary to have many holdout sets to check the validity of your utilities, or their scaling. However, if you want to use those choices to identify and eliminate inconsistent respondents, you need

342

SSI Web v7 several choice sets. Finally, if you do have several choice sets, it's useful to repeat at least one of them so you can obtain a measure of the reliability of the holdout choices. Suppose your conjoint utilities are able to predict only 50% of the respondents' holdout choices. Lacking data about reliability, you might conclude that the conjoint exercise had been a failure. But if you were to learn that repeat holdout tasks had reliability of only 50%, you might conclude that the conjoint utilities were doing about as well as they possibly could, and that the problem lies in the reliability of the holdout judgments themselves.

Analyzing Holdout Concepts

If you have specified fixed holdout choice tasks within the CBC questionnaire, you can analyze the results using the "Counts" technique offered within SMRT. Counts lets you specify which choice tasks to include in the analysis, so you can isolate the results for a single task at a time. Assume you had included a fixed holdout choice task (every respondent saw the same question), where the task displayed four product concepts. Further assume that each concept reflected a unique brand. You can tally the percent of the time each brand was chosen for that one task using the Marginals program in the Admin Module. These percentages will sum to 100%, and reflect the share proportions for the respective product concepts. Some researchers repeat choice tasks to achieve a measure of test-retest reliability. This type of analysis often is done at the individual level. If you plan to analyze holdout choice tasks at the individual level, you should export the data for analysis using another software program.

CBC Help

343

2.2.8

Specifying CBC Design Parameters

Experimental Design Creation Settings
Number of Random Choice Tasks The Random Choice Tasks are the questions/scenarios generated by CBC's design algorithms for the purposes of estimating part-worth utilities. The attribute levels making up the product concepts are formulated to have good statistical properties (such as level balance and independence of attributes). It is typical to include about 8 to 24 of these types of choice tasks in your study. Number of Fixed Choice Tasks If you want to specify specific choice scenarios for CBC to ask of every respondent, these are called Fixed Choice Tasks. If you add these to your questionnaire, you must specify the attribute level combinations that define the tasks (by clicking the Fixed Task Designs... button). Fixed Tasks are often used to include scenarios of particular interest to the client, and for collecting additional choice tasks that may be used to assess internal validity. If you are not inclined to include any fixed tasks in your study, you may set the number of fixed tasks to 0. Number of Concepts per Choice Task Here you specify how many product concepts (alternatives) to include in each choice task. Although it is tempting to want to make it easier on respondents by displaying fewer concepts per task, we urge you to ask respondents to consider many product concepts per task. For example, if each of your attributes had 3 levels, it would be a good idea to show 4 or 5 product concepts on the screen, so that some levels repeat within the choice task. This not only means each choice task provides more information, but including level overlap encourages respondents to provide more thoughtful responses and leads to stronger estimation of potential interaction effects. Random Task Generation Method This control lets you select which design method CBC should employ in determining the combinations of attribute levels to be shown in the Random Choice Tasks for your survey. We encourage you to read the section on CBC Questionnaires and Design Strategy to learn more about CBC's design strategies. For standard CBC studies, we generally suggest using either the Balanced Overlap or Complete Enumeration design strategies. (Note: these methods are computationally intensive, and for very large numbers of attributes and levels may require a long time to generate a design file.) Number of Questionnaire Versions CBC automatically creates a design file that is a) uploaded to the server if using web-based data collection, b) is copied to other PCs if using CAPI-based data collection, or c) is used to generate a questionnaire file if using paper-based data collection. During the process, you are asked to specify how many versions of the questionnaire should be written to that design file. For computerbased interviewing, we generally recommend that you include at least 100 versions, as there are benefits to fielding many versions of the questionnaire. Optimally, each respondent would receive his/her own version of the questionnaire. However, there are diminishing returns and including more than a few dozen versions often is of very little to no additional practical benefit. For that reason, we limit the number of versions that can be placed in the design file to 999. If you have more than 999 respondents, this means that some respondents will by necessity receive the same designs. For all practical purposes, once there are so many unique versions of the questionnaire in the total design pool, this poses no harm.

344

SSI Web v7 Design Seed This is the seed value used in the random number generator for the design algorithm. By using a different starting seed, a different design will result. With very many versions of the questionnaire, the overall design efficiency should be nearly identical no matter the random seed. With only a few versions of the questionnaire, using a different starting seed can have a meaningful effect on the overall efficiency.

Advanced Design Module Settings

Those with the base CBC license alone are limited to Traditional Full-Profile CBC Designs. Users with the Advanced Design Module may also choose Alternative-Specific Designs and PartialProfile Designs. These approaches are described later in this documentation.

Randomize Attribute Position within Concepts

Randomize Attribute Order Randomize Attribute Order specifies whether to present the attributes in random presentation order within a concept. If this is selected, the attribute list is randomized once per respondent, and all tasks within a respondent's interview will display the attributes in that given order. This can be useful to control order effects across respondents. First Randomized Attribute The first attribute in the range of attributes to be shown in random presentation order. Specify a "1" if you wish all attributes to be randomized. If, for example, you want the first and second attributes always to appear as the first two attributes in a product concept, specify a "2." Last Randomized Attribute The last attribute in the range of attributes to be shown in random presentation order. Specify the last attribute number if you wish all attributes to be randomized. If, for example, you had a total of five attributes in your study and you wanted the fourth and fifth attributes always to appear as the last two attributes in a product concept, specify a "3."

Concept Sorting within Task

Concept sorting refers to the order in which the product concepts are displayed in a task. For example, consider a choice task with three product alternatives. The default behavior in CBC software is to randomize the order in which these product concepts are displayed. Thus, a given brand may appear in the first concept in one task, but in the last concept position in the next task. This is Full Randomization. However, researchers do not always want the levels appearing in random concept positions within the task. There are two options for researchers to control concept ordering: Sort Concepts Using a Randomized Level Order. With this option, the researcher selects a specific attribute that is used to control the concept presentation order. For example, the

CBC Help

345

researcher may choose the "brand" attribute. The levels for brand are randomized once per respondent, and then the product concepts are displayed in that new order for all choice tasks for that respondent. For example, respondent #1 may see brands in the order 2, 3, 1 for the three concepts in the task (left, middle, right), for all tasks. Then, respondent #2 may see the brands in the order 3, 2, 1 for the concepts in his questionnaire. This strategy helps control order bias, without making respondents reorient themselves to new positions for, say, brands in each new task. Sort Concepts Using the Natural Level Order. With this option, the concepts are shown in the natural level order for a selected attribute, such as "brand." Brand 1 will always appear in the first concept, brand 2 in the second concept, etc.--assuming that the researcher uses a Complete Enumeration or Shortcut design strategy (which enforces minimal overlap), has no prohibitions, and shows as many concepts on the screen as there are brand levels in the study. This strategy is often used in "shelf facing" displays (Advanced Design Module only), or in cases in which the researcher doesn't want to make respondents reorient themselves to new positions for, say, brands in each new task. However, using a fixed concept order may introduce order biases.

Additional Settings
Show Levels of First Attribute no more than Once per Task Sometimes with attributes like brand, the researcher wants each brand to appear exactly once per choice task. If you have 4 brands and are showing 4 concepts per task, this happens naturally with Complete Enumeration or Shortcut strategies--unless you have included prohibitions in your study. When prohibitions are in place, it is possible for a brand to be repeated within the same choice task. To avoid this possibility, you can check this box, and the designer will ensure that levels of the first attribute appear just once per task. (Of course, this isn't feasible if you are trying to show more concepts per task than there are levels in the first attribute.)

Test Design Report Settings

Include Two-Way Frequencies in Test Design Report Check this to include an additional analysis within the Test Design Report of joint frequencies of levels. These joint frequencies reflect the number of times each attribute level appeared with each other attribute level within the same product concept across all concepts and tasks of your design.

Generate Design
When you click the Generate Design button, CBC automatically creates a design file using your current CBC settings that you upload to the server (for web-based data collection), or install on PCs (for CAPI-based collection), or that is used in generating questionnaire files for paper-andpencil collection. It also produces a test design report, letting you know whether the design can efficiently estimate the main effect utilities pertinent to your study. You should pay close attention to the design report to make sure the design efficiencies for the attribute levels in your study are acceptable and that the report does not state "Design Deficient" or have any asterisks in place of the estimated standard errors.

346

SSI Web v7 Please note that in some cases it may take a long time to generate a design if using the Complete Enumeration or Balanced Overlap methods. A typical study with few attributes and levels will face little delay (a few seconds) to generate hundreds of designs. CBC gives an estimate of the time remaining to generate the design. For testing purposes you may wish to generate few designs for the sake of speed, but once you are ready to post the survey to the Web (or use CAPI data collection) you may request that hundreds of designs be generated and saved to the design file.

Import/Export Design
Some users may choose to alter CBC's designs, or use their own design instead. Use the Export/Import Design capability when performing such advanced options.

CBC Help

347

2.2.9

Testing the CBC Design

Introduction
In CBC, a design refers to the sum total of the task descriptions across all respondents. The design contains information about the combinations of attribute levels that make up the product concepts within the tasks. The design is saved to a design file that you upload to your web server. Optimally efficient CBC designs can estimate all part-worths with optimal precision; meaning that the standard errors of the estimates are as small as possible, given the total observations (respondents x tasks), the number of product concepts displayed per task, and respondent preferences. CBC's random design strategies generally result in very efficient designs. These designs are not optimally efficient, but are nearly so. In the case of large sample sizes, a large number of questionnaire versions in the design file, and no prohibitions, one can confidently field a questionnaire without testing the design. However, there are conditions that can result in inefficient designs. Sometimes, a design can be so inefficient as to defy all attempts to compute reasonable part-worth utilities. We have heard of entire data sets with hundreds of respondents going to waste because the user neglected to test the design. Therefore, it is imperative to test your design whenever any of the following conditions exist: any prohibitions are included sample size (respondents x tasks) is abnormally small the number of versions you plan to use is few CBC's default Test Design capability (OLS Efficiency) only tests the efficiency of main effects (the separate utility estimate for each attribute level). It provides a good approximation of the relative efficiency of the CBC design with respect to each attribute level. Many researchers using standard CBC plans find this default test adequate for their purposes. For more sophisticated design testing, an Advanced Test option is available for simulating "dummy" respondent answers and reporting the standard errors (from a logit run) along with D-efficiency. Our design testing methods assume aggregate analysis, though most CBC users eventually employ individual-level estimation via CBC/HB. That said, CBC's design strategies can produce designs that are efficient at both the aggregate and individual levels (though we don't specifically provide valid measures of the standard errors for individual respondent estimates within our test design procedures). Prohibitions are often the culprit when it comes to unacceptable design efficiency. If your prohibitions result in unacceptably low design efficiency under the Complete Enumeration or Balanced Overlap Methods, you should try the Shortcut or Random design strategies. These latter two methods are less constrained than the more rigorous former ones, and will sometimes result in higher design efficiencies in the case of many prohibitions.

Testing the Efficiency of Your Design

When you choose Test Design from the Specify CBC Interview Parameters dialog, CBC automatically tests the design and displays the results within the results window (the results are also saved to a file named STUDYNAME_EXERCISENAMEtest.txt). CBC automatically generates a data file of simulated (dummy) respondent answers appropriate for advanced design testing. CBC includes two test procedures: Test Design (Frequencies and OLS Efficiency) (the default test routine used in previous versions of CBC, based on OLS theory)

348

SSI Web v7 Advanced Test (Simulated Data, Logit Report, and D-Efficiency) (a more rigorous test, based on conditional logit theory)

Test Design (Frequencies and OLS Efficiency)

Following is a sample report, as it appears in the results window:
CBC Design Efficiency Test Copyright Sawtooth Software Task Generation Method is 'Complete Enumeration' using a seed of 1 Based on 10 version(s). Includes 1000 total choice tasks (10 per version). Each choice task includes 3 concepts and 6 attributes. A Priori Estimates of Standard Errors for Attribute Levels --------------------------------------------------------------Att/Lev Freq. Actual Ideal Effic 1 1 75 (this level has been deleted) Brand A 1 2 75 0.2890 0.2887 0.9981 Brand B 1 3 75 0.2841 0.2835 0.9958 Brand C 1 4 75 0.2936 0.3062 1.0873 Brand D 2 1 2 2 2 3 3 1 3 2 3 3 4 1 4 2 4 3 5 1 5 2 5 3 6 1 6 2 6 3 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 (this level deleted) 0.2207 0.2182 0.2275 0.2182 (this level deleted) 0.2297 0.2182 0.2235 0.2182 (this level deleted) 0.2234 0.2182 0.2204 0.2182 (this level deleted) 0.2199 0.2182 0.2203 0.2182 (this level deleted) 0.2237 0.2182 0.2222 0.2182 0.9776 0.9200 0.9022 0.9533 0.9543 0.9806 0.9850 0.9809 0.9516 0.9648 1.5 GHz 2.0 GHz 2.5 GHz 3 lbs 5 lbs 8 lbs 60 GB Hard Drive 80 GB Hard Drive 120 GB Hard Drive 512 MB RAM 1 GB RAM 2 GB RAM $500 $750 $1,000

For each level, the number of times it occurs within the design is counted and provided under the column titled "Freq." Optimally efficient designs show levels within each attribute an equal number of times. For each attribute and level, an approximation is made of the relative standard error of each main effect under aggregate analysis and assuming that each version is seen just once across the total observations. Test Design uses ordinary least squares (OLS) rather than multinomial logit for this purpose, and it uses only the information about the design of the choice tasks, rather than respondents' answers. (A multinomial logit model is used in CBC's analysis modules.) This test design method gives relative standard error estimates similar to (but not identical to) those of multinomial logit. With this test, the emphasis is not a precise estimate of each standard error for a given number of respondents, but rather the pattern of their relative magnitudes with respect to one another.

CBC Help The Sample Output

349

We'll describe the output, using fragments of the file and showing the parts described in bold. Each line is labeled with the attribute and level in the first columns:
Att/Lev 1 1 1 2 Freq. Actual Ideal Effic 75 (this level has been deleted) 75 0.2890 0.2887 0.9981 Brand A Brand B

The number of times each level occurs in the design is displayed under the column labeled "Freq."
Att/Lev 1 1 1 2 Freq. Actual Ideal Effic 75 (this level has been deleted) 75 0.2890 0.2887 0.9981 Brand A Brand B

For estimation, it is necessary to omit one level from each attribute. The first level of each attribute is automatically deleted from this analysis:
Att/Lev 1 1 1 2 Freq. Actual Ideal Effic 75 (this level has been deleted) 75 0.2890 0.2887 0.9981 Brand A Brand B

The column labeled "Actual" gives estimated standard errors for the data file analyzed:
Att/Lev 1 1 1 2 1 3 1 4 Freq. Actual Ideal Effic 75 (this level has been deleted) 75 0.2890 0.2887 0.9981 75 0.2841 0.2835 0.9958 75 0.2936 0.3062 1.0873 Brand Brand Brand Brand A B C D

The column labeled "Ideal" gives an estimate of what those standard errors would be if the design were precisely orthogonal and had the same number of observations:
Att/Lev 1 1 1 2 1 3 1 4 Freq. Actual Ideal Effic 75 (this level has been deleted) 75 0.2890 0.2887 0.9981 75 0.2841 0.2835 0.9958 75 0.2936 0.3062 1.0873 Brand Brand Brand Brand A B C D

The column labeled "Effic" gives the relative efficiency of this design in terms of estimating each parameter, compared to the hypothetical orthogonal design (it is the square of their ratio):
Att/Lev 1 1 1 2 1 3 1 4 Freq. Actual Ideal Effic 75 (this level has been deleted) 75 0.2890 0.2887 0.9981 75 0.2841 0.2835 0.9958 75 0.2936 0.3062 1.0873 Brand Brand Brand Brand A B C D

When we consider the entire "Effic" column, we see that the randomized design had a median efficiency of about 97 percent, relative to a hypothetical orthogonal design. The estimates of standard errors for orthogonal designs are only approximate, and with a very small data file such as this there can be quite a lot of variability in estimation. Notice that the standard error estimated for attribute 1, level 4 is actually smaller than the value estimated for a hypothetical orthogonal design:
Att/Lev 1 1 1 2 1 3 Freq. Actual Ideal Effic 75 (this level has been deleted) 75 0.2890 0.2887 0.9981 75 0.2841 0.2835 0.9958 Brand A Brand B Brand C

350

SSI Web v7
1 4 75 0.2936 0.3062 1.0873 Brand D

Anomalies such as this are likely to occur when using small samples of test respondents, and shouldn't be of concern. It is important to test the design, since if too many prohibitions are included it is possible to develop designs that do not permit estimation of desired effects. When that occurs, the estimated standard errors for those effects will be infinite and their estimated efficiencies will be zero. Your attention is called to such occurrences by the presence of asterisks instead of numbers, and/or by error messages, that signal that additional thought is needed about what can be prohibited from occurring. If you see a warning stating that your design is deficient or if you see asterisks listed for the standard errors, your design is deficient. Especially if using few questionnaire versions, you will find that the quality of the design is affected by the design seed. You may want to try different design seeds to obtain slightly better designs.

Advanced Test
Rather than just offering a relative measure of efficiency, the Advanced Test design estimates the absolute precision of the parameter estimates under aggregate estimation, based on the combined elements of design efficiency and sample size (respondents x tasks). The estimated standard errors are only absolutely correct if the assumptions regarding the underlying part-worths and the error in responses are correct. The Advanced Test is useful for both standard and complex designs that include interaction or alternative-specific effects. It also reports a widely accepted measure of design efficiency called D-efficiency, which summarizes the overall relative precision of the design. Unlike the simpler default test, the Advanced Test takes into account that design efficiency for CBC studies depends on how the concepts are grouped in sets. The level contrasts within sets determine how much information that set contains with respect to the levels of interest. Technically, the utility balance among the concepts within those sets also affects overall design efficiency, and thus respondents' preferences need to be known to assess the efficiency of a design. However, most researchers are more comfortable planning designs that are efficient with respect to uninformative (zero) part-worth utility values, and that is the approach we take. The Advanced Test simulates random (dummy) respondent answers for your questionnaire, for as many respondents as you plan to interview. The test is run with respect to a given model specification (main effects plus optional first-order interactions that you specify). To perform the Advanced Test, you need to supply some information: Number of Respondents % None (if applicable to your questionnaire) Included Interaction Effects (if any) With this information, CBC simulates random respondent answers to your questionnaire. Using random respondent answers is considered a robust approach, because it estimates the efficiency of the design for respondents with heterogeneous and unknown preferences. Simulated respondents are systematically assigned to the versions of your questionnaire (the first respondent receives the first version, the second respondent the second version, etc.). If you are simulating more respondents than versions of the questionnaire, once all versions have been assigned, the next respondent starts again with the first version. If your study includes a None alternative, then the None is selected with expected probability equal to the value you previously specified.

CBC Help

351

Once the data set has been simulated, the Advanced Test performs an aggregate logit (MNL) run, estimating the effects you selected (by default, only main effects are considered). Sample results are shown below:
Logit Report with Simulated Data -----------------------------------------------------------Main Effects: 1, 2, 3, 4, 5, 6, 7 Interactions: 1x6 Build includes 300 respondents. Total number of choices in each response category: Category Number Percent ---------------------------------------------------1 787 21.86% 2 753 20.92% 3 778 21.61% 4 792 22.00% 5 490 13.61% There are 3600 expanded tasks in total, or an average of 12.0 tasks per respondent. Effect 0.01171 0.01427 -0.00202 -0.02396 0.01118 0.00600 -0.01717 -0.00534 0.01096 -0.00562 -0.02986 0.04165 -0.01179 -0.00618 -0.03820 0.04438 0.05398 -0.01099 -0.04300 0.09273 -0.01982 -0.07291 -0.11173 0.03271 0.07902 -0.02233 -0.00902 0.03135 0.04132 -0.00386 -0.03746 -0.45916 Aggregate Std Err 0.03186 0.03182 0.03195 0.03215 0.02638 0.02639 0.02654 0.02638 0.02631 0.02645 0.02655 0.02620 0.02644 0.02644 0.02661 0.02615 0.02610 0.02647 0.02669 0.04975 0.05101 0.05124 0.05083 0.05040 0.05060 0.05050 0.05104 0.05094 0.05038 0.05102 0.05164 0.04862 t Ratio 0.36757 0.44850 -0.06333 -0.74517 0.42372 0.22715 -0.64704 -0.20235 0.41654 -0.21251 -1.12501 1.58962 -0.44580 -0.23387 -1.43548 1.69683 2.06837 -0.41497 -1.61105 1.86389 -0.38851 -1.42294 -2.19797 0.64897 1.56170 -0.44214 -0.17680 0.61548 0.82021 -0.07574 -0.72529 -9.44349 Attribute Level 1 1 Brand A 1 2 Brand B 1 3 Brand C 1 4 Brand D 2 1 1.5 GHz 2 2 2.0 GHz 2 3 2.5 GHz 3 1 3 lbs 3 2 5 lbs 3 3 8 lbs 4 1 60 GB Hard Drive 4 2 80 GB Hard Drive 4 3 120 GB Hard Drive 5 1 512 MB RAM 5 2 1 GB RAM 5 3 2 GB RAM 6 1 $500 6 2 $750 6 3 $1,000 Brand Brand Brand Brand Brand Brand Brand Brand Brand Brand Brand Brand NONE A A A B B B C C C D D D by by by by by by by by by by by by $500 $750 $1,000 $500 $750 $1,000 $500 $750 $1,000 $500 $750 $1,000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

352

SSI Web v7
The strength of design for this model is: 3,256.006 (The ratio of strengths of design for two designs reflects the D-Efficiency of one design relative to the other.)

Details regarding the logit report may be found in the section entitled CBC Analysis: Counts and Logit. The beginning of the report lists the effects we are estimating (main effects for attributes 1 through 6, plus the interaction effect between attributes 1 and 6). All random tasks are included in estimation. Next, we see that 300 respondents each with 12 tasks were simulated using random responses, taking into account the expected probability for None. We specified that the None would be chosen with 15% likelihood, and indeed the None percentage is very close to that (13.61%). If we had used more respondents, the probability would have been even closer to 15%. The remaining choices are spread approximately evenly across the four other alternatives in the questionnaire. Next, the logit report based on the random responses is shown. You shouldn't pay any attention to the effects (utilities) in the first column, as we are using random data. The T-ratios are also not useful, for the same reason. The important column to study is the Aggregate Std Err (Standard Error) column. The standard errors reflect the precision we obtain for each parameter. Lower error means greater precision. This design included no prohibitions, so the standard errors are quite uniform within each attribute. If we had included prohibitions, some levels might have been estimated with much lower precision than others within the same attribute. For our simulated data above, the levels within three-level attributes all have standard errors around 0.026. The one four-level attribute has standard errors for its levels around 0.032. We have obtained less precision for the four-level attributes, since each level appears fewer times in the design than for the three-level attributes. The interaction effects have standard errors around 0.051. Suggested guidelines are: Standard errors within each attribute should be roughly equivalent Standard errors for main effects should be no larger than about 0.05 Standard errors for interaction effects should be no larger than about 0.10 The second two criteria are rules of thumb based on our experience with many different data sets and our opinions regarding minimum sample sizes and minimum acceptable precision. Ideally, we prefer standard errors from this test of less than 0.025 and 0.05 for main effects and interaction effects, respectively. These simulated data (300 respondents with 12 tasks each) almost meet that higher standard for this particular attribute list and set of effects. D-Efficiency D-efficiency summarizes how precisely this design can estimate all the parameters of interest with respect to another design, rather than how well the design can estimate the utility of each level of each attribute (as with the simpler default test). D-efficiency is described in an article by Kuhfeld, Tobias, and Garratt (1994), "Efficient Experimental Design with Marketing Research Applications," Journal of Marketing Research, 31 (November), 545-557. To arrive at D-efficiency, we should define a few terms: Xt xi pi v Zt Z = design matrix for task t with a row for each alternative = ith row of Xt = probability of choice of alternative i = probability-weighted means of rows: v = i pi xi 1/2 = matrix with ith row zi = pi ( xi - v) = matrix made by appending all Zt matrices

CBC Help Z'Z is known as the "Information Matrix" The determinant of Z'Z measures the strength of the design.

353

Because the magnitude of the determinant of Z'Z depends on the number of parameters estimated, to provide a measure of strength independent of p we consider the pth root of the determinant:

|Z'Z|1/p
Where Z is the probability-centered design matrix, Z'Z is the "Information Matrix," and p is the number of parameters estimated. The pth root of the determinant doesn't result in a single value bounded by 0 and 1.0 (as with the simpler test efficiency report), and this value is meaningless without reference to the same computed for comparison design. This value also depends on the number of respondents x tasks, so when comparing two designs, it is important to hold the number of respondents x tasks constant. We use the term "efficiency" to compare the relative strengths of two designs. The relative D-efficiency of one design with respect to the other is given by the ratio of the pth root of the determinants of their information matrices. The design with a larger value is the more efficient design. (Note: we only consider the precision of parameters other than the "None" parameter when computing the strength of the design.) Consider design A with no prohibitions and design A' with prohibitions. The pth root of the determinant of the information matrix is computed for both (holding the number of respondents, tasks, concepts per task, and None % constant). If design A' has a value of 2,500 and design A has a value of 3,000, design A' is 2,500/3,000 = 83.3% as efficient as design A. The inclusion of prohibitions resulted in a 1 0.833 = 16.7% loss in efficiency. Efficiency for Specific Parameters (Attribute Levels) Sometimes, you may be more concerned about the efficiency of the design for estimating a specific parameter (such as the utility for your client's brand) rather than an overall efficiency of the design across all parameters. Let's assume that your client asked you to implement prohibitions between the client's brand name and other levels. Further assume that the overall relative strength of the design with prohibitions relative to the design without prohibitions is 97%. On the surface, this seems like little overall loss in efficiency. However, you note that the standard error (from the logit report using simulated data) for your client's brand was 0.026 prior to implementing the prohibition, but 0.036 afterward. The relative efficiency of the design with prohibitions relative to the non-prohibited design with respect to this particular attribute level is: a2/b2 Where b is the standard error of the estimate for the client's brand name after the prohibition and a is the standard error prior to the prohibition. In this case, the relative design efficiency of the prohibited compared to the non-prohibited design with respect to this particular level is: 0.0262/0.0362 = 0.52 And the impact of these prohibitions on estimating your client's brand utility is more fully appreciated. Additional Note: The pattern of random answers for random respondent data will have a small, yet perceptible, effect on the reported Strength of Design (relative D-Efficiency). For the purpose of estimating the absolute standard errors of the parameters, we suggest using the same number of dummy respondents as you plan to achieve with your study. But, for comparing the merits of one design to another, you can reduce the effect of the random number seed by increasing the dummy respondent sample size significantly, such as to 5000 or 10000 respondents. Increasing the sample size will greatly reduce the variation of the Strength of Design measure due to the random seed used for respondent answers.

354

SSI Web v7

2.2.10

Importing/Exporting CBC Designs

One of the strengths of the CBC system is its ability to generate experimental designs (the combinations of attribute levels to show respondent) in an efficient and easy manner. Also, multiple versions of the questionnaire are generated, so different respondents see different combinations of attributes within the choice tasks. Some researchers wish to export the design, to examine ahead of time the combinations that will be asked of respondents (and perhaps to perform additional analysis regarding the design). Other researchers may have developed questionnaire designs using other software and wish to import those designs for fielding within the CBC system. They can do this through CBC's Import Design capability. For either purpose, the design file uses the same comma-separated (.CSV) layout. A .CSV file can be directly opened and saved with Excel, and the examples below assume you are working with the comma-separated file in Excel. (If using a text editor, the values in the file are separated by commas.) If you Import a design, any existing design will be overwritten. Import/Export File Format The file format is as follows. Column 1 Indicates the questionnaire version. Any fixed tasks in your study are listed as version 0 (all the examples below include a single fixed task). Fixed tasks are shown to every respondent and are only defined once in the design file (as version 0). Column 2 Indicates the task (choice set) number. Column 3 Indicates the concept (alternative) number. Columns 4 through n Shows the attribute levels associated with the current concept, task, and version number. Traditional CBC Layout: In the example below, each respondent receives a single fixed choice task (version 0, task 1). There are also two random tasks in the questionnaire (i.e. calibration tasks used for part-worth utility estimation). Only two versions of the questionnaire are shown below, though additional versions would continue, following the same pattern. There are three attributes in this study, and three concepts are shown per task.

CBC Help

355

Alternative-Specific Layout: In the example below, Attributes 2 and 3 are alternative-specific. When an attribute is not shown in a product concept, it receives a level "0". Attribute 2 only appears when level 2 of attribute 1 is present. Attribute 3 only appears when level 3 of attribute 1 is present.

Partial-Profile Layout: When an attribute is not shown in a choice task, it receives level "0". In the example below, two of the three attributes are shown per task.

356

SSI Web v7

Additional Rules: For Random tasks (tasks used in estimation), the version must start with 1 and proceed in that order. If random tasks are present in the file to import, the number of versions and number of tasks must match the number of versions and tasks specified in the software. The number of concepts must be constant across all tasks. The number of concepts must match the number of concepts specified in the software. If Partial Profile is specified in the software, the software setting for Number of Attributes Shown in Each Concept must match the number specified in each concept in the file (random tasks only). Any prohibitions specified in CBC will not be checked against the imported design.

CBC Help

357

2.3
2.3.1

Advanced CBC Designs

Conditional Pricing for CBC
In pricing research, it is sometimes very useful if prices for product concepts are made to depend on other attribute levels (such as brands). The first versions of CBC could handle this in a limited way by using prohibitions. For example, one could prohibit high prices from being shown for some products, and prohibit low prices from being shown with others. Such prohibitions, however, can lead to very inefficient designs. There are some work-arounds for dealing with situations where prohibitions with price seem to be needed. For example, if package size is an attribute, prices for the 64 oz package would be much higher than prices for the 16 oz size. One approach is to express price as a percentage above or below the "normal price" for that package size, such as "10% above the average price." Another is to show unit prices, rather than prices per package. For example, with detergent we might study 16, 32, and 64 ounce packages, but present prices in terms of "cents per ounce." These are ways of making a single set of price levels work for all package sizes without specifying any prohibitions, but it would be preferable to display actual prices appropriate for each package size. The conditional pricing option lets you create a look-up table to determine the prices to show for specific combinations of attributes. Drawing upon the example above, assume we have three attributes as follows to describe detergents: BRAND Brand A Brand B Brand C PACKAGE 16 oz. package 32 oz. package 64 oz. package PRICE "Low price" "Medium price" "High price" The text for the price levels above is not shown during the interview, and serves as a place-holder only. The price levels displayed during the interview vary according to the package sizes as follows: Conditional Pricing Lookup Table Low Price 16 oz. package 32 oz. package 64 oz. package $1.49 $2.79 $4.89 Medium Price $2.09 $3.99 $6.99 High Price $2.69 $5.19 $9.09

To construct this lookup table, we started with average prices for the three package sizes (middle column). To determine the prices in the "Low Price" column, we decreased the average price by 30% (and rounded to the nearest 9 cents). The "High Price" was calculated by increasing the average price by 30% (and again rounding to the nearest 9 cents).

358

SSI Web v7 The example above illustrates how to make price dependent on a single variable (package size). CBC lets you make price conditional on up to 3 attributes. You will need to deal with an additional level of complexity when analyzing the results of studies that involved conditional pricing. In the example above, even though nine unique prices were shown to respondents, CBC's analysis programs by default still regard these as just three levels of price: Low, Medium and High. However, we interpret the results keeping in mind that larger package sizes were displayed with higher prices on average. We interpret the part-worth of the 32 oz. package taking into account that it was shown at an average price roughly $1.90 cents higher than the 16 oz. package. The main effect contains information not only about how desirable one package size is versus another, but also about the relative price levels at which they are offered. Therefore, it is perfectly legitimate if the count proportion or logit effect for the 32 oz. package is lower than the 16 oz. package. This would indicate that respondents on average did not feel the larger size was worth the extra $1.90. The conditional pricing table must also be taken into account when using the market simulator. Unless the conditional pricing table is explicitly used in the market simulator, to simulate a 32 oz. package at $3.99, level 2 for price is specified, etc. However, with the most recent releases of SMRT, when you export the data for analysis in SMRT (the .CHO and .ATT files), the conditional pricing table is also exported for use in the SMRT market simulator. The conditional prices are automatically read from a special footer section in the .ATT file into the Assign Level Values area of the SMRT market simulator (please double-check them to verify that the information was interpreted correctly, as we parse text strings to extract the price values). Once the conditional prices are within the SMRT market simulator, you simply specify the actual conditional price during simulations. For example, to simulate a 32 oz. package at $3.99, instead of specifying level 2, you specify a level value of 3.99. Users should note that the use of Randomized First Choice (or the historical Share of Preference with Correction for Product Similarity) can be problematic with conditional pricing. These methods assume that products sharing the same attribute levels are identical on those levels and require some downward correction in share. But, with conditional pricing tables, even though two brands may share the same "level" for price, the conditional pricing table may have actually resulted in unique prices for brands within the questionnaire. Therefore, RFC may implement a correction for product similarity that isn't actually warranted, assuming that the brands' prices were identical when in fact they were not. To avoid this outcome, we suggest turning off the correction for product similarity with respect to price within the RFC advanced dialog. In our previous example, we used (near) constant deviations from average prices to build a proportional conditional table. Some researchers choose to use constant absolute price differences between levels of the conditional attribute(s) instead of proportional changes. In either case, the tables have symmetric properties, which are desirable from an analysis standpoint. Proportional or symmetric conditional pricing tables sometimes make it possible to adequately model the data using main effects only (should the interaction effects turn out to be not significant.) Otherwise, specification of interactions may be necessary to fit the data properly, and the additional parameters estimated are done so with relatively less precision than main effects. If your conditional pricing table reflects significant deviations from symmetry, you must specify appropriate interaction effects to correctly fit the data, even if the data don't justify the additional parameters added to the model. A two-way interaction can correctly fit data collected with an asymmetric conditional pricing table where price is dependent on one other attribute. If price was dependent on more than one attribute and the table is not proportional, a three-way interaction would be required to correctly model the effects. Our logit, latent class and CBC/HB programs are limited to modeling only main effects and two-way interactions. Therefore, to analyze three-way interactions, you would need to reformat the .CHO file and collapse the two conditional attributes (using your own data processing techniques) prior to specifying the interactions with price. Up to 254 levels per attribute are permitted in CBC with the advanced design module, so this affords some flexibility in this area.

CBC Help

359

Specifying Conditional Pricing

Before you can specify conditional prices, you first must have defined your list of attributes and levels. Instead of specifying the text for actual prices for your price levels, you should have used placeholder text such as "low price," "medium price," and "high price." When the questionnaire is administered, the placeholder text you assigned is overwritten by the text string you specify in the conditional pricing table. When you create a conditional price lookup table in CBC, you must specify the number of attributes that participate in the relationship (there should be at least two). If price is dependent on one other attribute (such as package size), two attributes participate in the relationship: package size and price. Click Compose | Write Questionnaire... and then edit any CBC question within the CBC Exercise, then click the Attributes tab and select the Conditional Relationships button. A dialog appears in which you specify attributes involved in the relationship. First, select your price attribute. The level text for this attribute is overwritten by the prices that you specify in the conditional pricing table. Then, select the attribute(s) upon which price is dependent. The conditional pricing table is displayed, with as many rows in the table as required to completely define the prices for all combinations of price and the dependent attribute(s). You can cut and paste conditional pricing information directly from a spreadsheet program into the conditional pricing grid. There can be a maximum of one conditional pricing table per study. However, some users have discovered that they can use conditional display tables just as they would use conditional pricing tables. In that case, more than one conditional pricing variable may be included. The only difference is that the conditional prices specified as conditional display variables are not automatically carried forward to the SMRT market simulator. In that case, you may need to use relative indices (such as 0.8, 1.0, 1.2, etc.) when defining products in the market simulator, rather than having SMRT automatically use the conditional pricing table within the simulator.

360

SSI Web v7

2.3.2

Conditional Display
Conditional display is often used for showing a graphic when levels from one or more attributes appear together within a concept in a CBC task. For example, you may have two separate attributes for brand and package size, but a single picture reflects the joint brand/package size concept. Where the text for a particular brand and package size might appear in a CBC concept, you may wish to replace that with a graphic. But conditional display is not limited to just graphics, as it simply adds (or replaces, depending on your Attribute Visibility setting) the existing level text. Other possibilities include: Multi-media files Text JavaScript Attribute Label (Optional) If you are using Conditional Display and have chosen to show the graphic (or other conditional display "element") at the top or bottom of the choice task, you can optionally specify an attribute label to be shown in the choice task (on the left-hand side of the task). This attribute label is only used (and is only applicable, otherwise it is "grayed out") if you have specified that attribute labels are to appear in the choice task. You specify that attribute labels should appear within the choice task by clicking Compose | Write Questionnaire and editing one of your choice tasks from the List of Questions. Then, from the Format tab check the Show Attribute Labels box. Note: if in Step 3 of the Conditional Relationships Wizard, you specified that the conditional display element is to replace the text of a certain attribute within the task, then the attribute label used for the conditional element is the original attribute label for the replaced attribute text (that you originally typed within the Specify Attributes and Levels dialog).

CBC Help

361

2.3.3

Alternative-Specific Designs
(The capabilities described in this section are only available if you own the CBC Advanced Design Module.) Some CBC projects require more flexible designs than the standard approach used with our CBC system. With standard CBC, if there are six attributes in the study, all six appear within each product concept. Researchers have employed a specialized type of CBC design wherein some or all product alternatives have their own unique sets of attributes. Conditional Pricing is one example of such a design, where each brand might have its own set of prices. But, designs can be much more complex than that. For example, consider modes of transportation: cars versus buses. Each option has its own set of features (beyond just price) uniquely associated with that mode of transportation. Consider the following attributes for cars and buses: Car Parking fee $5.00/day Parking fee $8.00/day Parking fee $10.00/day Bus Picks up every 20 min. Picks up every 15 min. Picks up every 10 min. Picks up every 5 min. 25 cents per one-way trip 50 cents per one-way trip 75 cents per one-way trip $1.00 per one-way trip

Assume we wanted to measure three other (constant) alternatives: walking, biking, or "I'd choose another way to get to work." The CBC question might look like the following: If you lived 1 mile away from your workplace in downtown New York and these were your choices for getting there, which would you choose? Ride the bus Picks up every 10 min. $1.00 per one-way trip Bike Drive my car Parking fee $8.00/day Walk I'd choose another way to get to work

To implement this design using CBC's Advanced Design Module, you define the attributes as follows: Attribute 1: Walk Bike Ride the bus Drive my car Attribute 2: Picks up every 20 min. Picks up every 15 min. Picks up every 10 min.

362

SSI Web v7 Picks up every 5 min. Attribute 3: 25 cents per one-way trip 50 cents per one-way trip 75 cents per one-way trip $1.00 per one-way trip Attribute 4: Parking fee $5.00/day Parking fee $8.00/day Parking fee $10.00/day Before proceeding, it is useful to introduce some terminology to describe the different roles attributes can play in alternative-specific designs. Primary attribute: an attribute whose levels appear in every product concept and upon which conditional attributes (described below) depend. In the example above, attribute 1 is the primary attribute. Conditional attribute: attributes only displayed with a particular level or levels of the primary attribute. Attributes 2 through 4 are conditional attributes. Common attribute: attributes displayed with all levels of the primary attribute. (The example above doesn't include a common attribute.) In traditional CBC designs, all attributes are common.

Specifying an Alternative-Specific Design

Choose Compose | Write Questionnaire... and the select a CBC question within the desired CBC Exercise to edit. This will bring up the CBC Exercise Settings dialog. From that dialog, click the Design Tab and then check the Alternative-Specific CBC Design radio button. To create the "None" alternative, select Traditional None Option under the Format tab and specify the text: "I'd choose another way to get to work." Additional constant alternatives are defined as levels within the primary attribute. To complete the design, set the following attribute level prohibitions (make sure to check "alternativespecific design" prior to setting these prohibitions): Attribute 1, Level 1 Attribute 1, Level 2 Attribute 1, Level 3 Attribute 1, Level 4 Attribute 1, Level 4 with with with with with Attributes 2 through 4 (all levels) Attributes 2 through 4 (all levels) Attribute 4 (all levels) Attribute 2 (all levels) Attribute 3 (all levels)

Additional Guidelines for Alternative-Specific Designs

For alternative-specific designs, we generally suggest the Complete Enumeration strategy (unless you have so many attributes that the design generation process is prohibitively slow). If the time to generate the design is too long, then you might consider 1) generating fewer total designs (e.g. 20 instead of 300) or 2) the Shortcut design strategy. In either case, use the advanced test design procedure (estimated standard errors via logit and relative D-efficiency test) to determine the effect of one strategy versus the other.

CBC Help

363

Recall that within our paradigm for alternative-specific designs, main effects for conditional attributes are measured specific to particular levels of the primary attribute. If the measurement of higher-order effects is the primary concern, the Balanced Overlap or purely Random design method can work well given a large sample size. The "None" alternative is always displayed in the last concept position (or in the question's header or footer). Other constant alternatives you define (with exhaustive prohibitions) within the primary attribute are positioned randomly within the task by default, but you can control their order of presentation on the Design tab of the CBC Exercise Settings dialog. Depending on how many concepts are displayed per task and the design method, it is possible that a constant alternative may not appear in some tasks. If you want the constant alternatives to appear in every task, you should display at least as many concepts as levels in the primary attribute and then choose the Shortcut design strategy. It is possible to have multiple primary attributes and multiple conditional "branches" in an alternativespecific design. Conditional attributes can be shared (common) between two or more levels of the primary attribute. You can choose to have those common attribute aligned in the same row within your choice task by checking Align Common Attributes on Same Row from the Design tab. Fixed alternative-specific tasks (or entire designs) may be specified. Primary attributes must appear in the attribute list before conditional attributes.

Design Efficiency and Alternative-Specific Designs

Alternative-specific designs usually require a complex set of prohibitions. Therefore, we strongly encourage you to test the design to ensure acceptably stable estimation of the part-worths. For such designs, we suggest you use the advanced test design procedure (estimated standard errors via logit and relative D-efficiency test). When testing the design, you should check is that it doesn't return the message: "WARNING - THE DESIGN IS DEFICIENT." You should pay particular attention to the standard errors of the part-worths. Lack of convergence or very large standard errors are an indication that your design is deficient. Hint: One rule of thumb is to look for standard errors from logit for main effects of no greater than about 0.05, and standard errors for interaction effects to be no greater than about 0.10 (alternativespecific attribute effects have standard errors roughly the same magnitude as first-order interaction effects). If your design is like those described in our transportation example where prohibitions are between all levels of the conditional attribute(s) with a particular level (or levels) of the primary attribute, you will likely be on safe ground. If you define additional prohibitions, you may encounter problems during analysis. You can specify two-way interactions for alternative-specific designs, though in our example it doesn't make sense to specify two-way interactions between primary and conditional attributes, since the effects measured are already "specific" to the reference levels of the primary attribute. However, for this example design, it is possible to study the interaction between the frequency of bus pick-up and the cost per bus trip, since there were no prohibitions between those two attributes. If a common attribute were defined (applying to all levels of the primary attribute), it is possible to specify other interactions, such as between the primary and common attribute. Analysis proceeds as with any other CBC study. The only difference is that when specifying product concepts in the market simulator, you type "N/A" for conditional attributes that aren't applicable.

364

SSI Web v7 Logit (limited to 30 attributes), Latent Class (limited to a maximum of 100 attributes), or HB (limited to 1000 attributes) estimation are possible with alternative-specific designs.

CBC Help

365

2.3.4

Partial-Profile Designs
(The capabilities described in this section are only available if you own the CBC Advanced Design Module.) In spite of their advantages, CBC questions can overwhelm some respondents, particularly if there are many attributes. The base CBC system can measure up to ten attributes (shown in full profile), but more than about eight attributes may overwhelm respondents in some situations. Some researchers have proposed "partial-profile" choice experiments as a way to estimate preferences for a large set of attributes. With partial-profile designs, each choice task includes a subset of the attributes (typically around five). Across all tasks and respondents, a much larger list of attributes is evaluated. The CBC Advanced Design Module permits up to 30 attributes. Partial-profile choice designs are relatively new to the industry, but are becoming more mainstream over the last few years. A growing body of research suggests that they can be valuable for some situations. We haven't yet formed a definite opinion about their performance versus other methods like Adaptive CBC (ACBC) and ACA for dealing with large numbers of attributes. We hope that including partial-profile in the Advanced Design Module encourages further research and experimentation. With partial-profile designs, we assume respondents can evaluate the product concepts holding all attributes not represented constant. If respondents cannot maintain this ceteris paribus mind set, the resulting data may be incorrect. Therefore, when asking the choice question, we suggest including language such as, "Please assume that all features not shown are alike in all other ways," or "Please assume that these toasters differ only on the features shown." This may help respondents answer appropriately, but it is still no guarantee. The use of the "None" concept in partial-profile CBC studies is problematic. The None weight varies significantly depending on how many attributes are displayed in the partial-profile task. Analysis methods for partial-profile include logit (limited to 30 attributes), Latent Class (limited to 100 attributes), and HB (limited to 1000 attributes). The success of the methods (particularly for Latent Class and HB) hinge upon how much information is available from each respondent on each attribute. We caution that individual-level estimation may not be stable for partial-profile designs if the information available from each respondent relative to the number of parameters to be estimated is low. Given enough information per respondent relative to the number of parameters to be estimated, Hierarchical Bayes estimation may provide useful individual-level utilities for use in market simulations where the results are summarized across respondents. Still, choice data are not as rich in terms of statistical information content as ratings-based data. The individual-level estimates for large partialprofile designs may contain a significant amount of noise and counter-intuitive relationships (reversals). If the goal of the research is to estimate stable utilities for individual-level profiling and predictions, the partial-profile approach alone may be insufficient. For sparse partial-profile designs, it may be helpful in CBC/HB software to use a lower prior variance assumption and a higher degrees of freedom for prior covariance matrix to avoid potential overfitting. Many split-sample partial profile / full profile methodological studies have shown that the parameters derived from either experiment are quite similar, including Price. Some recent methodological experiments presented at the Sawtooth Software conference suggested that the parameters were quite similar, but that the importance (slope) of Price was understated using partial-profile. Therefore, we suggest users should be aware of this potential outcome and proceed with caution when using large partial-profile designs in CBC for pricing research.

Specifying Partial-Profile Designs

Choose Compose | Write Questionnaire... and edit a CBC question within the CBC exercise to bring up the Conjoint Settings | CBC Settings... to bring up the CBC Exercise Settings dialog. From that

366

SSI Web v7 dialog, click the Design Tab and then click the Partial-Profile CBC Design option. There are a number of control parameters governing partial-profile designs. For purposes of illustration, let's assume there are 12 total attributes in the study. The researcher wants to display 5 attributes per task with attributes 1 and 12 to appear in every choice task. Rotate Attributes into Concepts Starting with: In this example, attribute 1 appears in every choice task. Therefore, attributes rotate in and out of the choice tasks starting with attribute 2. (If you want all attributes to rotate into the tasks, always specify 1). Note that, all else equal, attributes displayed in every task are measured with greater precision than those rotated into the tasks. and Ending with Attribute: In this example, attribute 12 appears in every choice task. Therefore, attributes rotate into choice tasks ending with attribute 11. (If you want all attributes to rotate into the tasks, specify the last attribute number.) Number of Attributes Shown in each Concept: In this example, 5 attributes are displayed in each choice task, so we specify a 5. Recent research suggests that between 2 to 4 attributes may be optimal to use in partial-profile studies. The length of the level text has a bearing on the appropriate number, along with the familiarity and interest respondents have for the product category. If the Randomize Attribute Order box is checked, the attributes appear in random order (held constant within respondent) within the concept. If un-checked (default), the attributes appear in their natural order.

Design Strategies for Partial-Profile Designs

With partial-profile designs, the design selection has two stages. The first stage involves choosing the subset of attributes displayed in the choice task. To formulate the first task, a subset of attributes is randomly chosen. For all subsequent tasks, the two-way joint frequency table of attribute presentation within choice tasks is examined. Attributes are chosen so that the off-diagonal cells in this table remain approximately balanced. This strategy is identical for the two methods. The second stage of the design selection involves deciding which levels of the selected attributes are displayed for each concept. For partial-profile designs, we generally suggest the Complete Enumeration strategy (unless you have so many attributes that the design generation process is prohibitively slow). If the time to generate the design is too long, then you might consider 1) generating fewer total designs (e.g. 20 instead of 300) or 2) the Shortcut design strategy. In either case, use the advanced test design procedure (estimated standard errors via logit and relative D-efficiency test) to determine the effect of one strategy versus the other. The purely Random design method should only be used if the measurement of interactions (through aggregate analysis) is the primary goal, the attribute list is relatively small and the sample size is quite large. However, partial-profile designs make it much more difficult to measure interactions between attributes, when these attributes may only occur together within a product concept a minority of the time.

CBC Help

367

Notes for Partial-Profile Designs

Estimation of main effects and especially interaction terms for attributes rotated into choice tasks is significantly less precise than with manageable full-profile CBC designs. We strongly suggest you test your design prior to fielding to ensure you can estimate reasonably stable part-worths. The Test Design module provided with CBC may be used. When using the default Test Design routine (Frequencies and OLS Efficiency), you should recognize that the design efficiency is estimated with respect to a hypothetical full-profile orthogonal design. The efficiency of part-worths for attributes rotated into the design is naturally lower than with a full-profile design (where the attribute is always present), since comparatively less information about that attribute is available. We suggest using the advanced test design procedure (simulated respondent data, logit estimation report, and relative D-efficiency) to assess the quality of different partial-profile designs. Fixed partial-profile tasks (or entire designs) may be specified. Partial-profile designs are more robust in the face of level prohibitions than full profile designs. For example, consider a study with 10 attributes, where 4 attributes are shown at a time in product concepts. Further consider that there are prohibitions between the levels of attributes 1 and 2. When attributes 1 and 2 appear together within the same choice task, the prohibitions introduce correlations (and resulting lower precision) within the design matrix. However, for the many tasks in which attributes 1 and 2 appear separate from one another, the prohibitions have no negative effect on the design efficiency. These tasks in which attributes 1 and 2 appear separately provide uncompromised information (assuming no bias due to partial-profile displays) for stabilizing the parameters for attribute 1 and 2.

368

SSI Web v7

2.3.5

Shelf-Facing Display
The capabilities described in this section are only available if you own the CBC Advanced Design Module.) If you own the Advanced Design Module for CBC, you can use the "Shelf Facing" Display option. The shelf display option is particularly useful for beverage and packaged goods research. Here is an example of a shelf facing display:

To achieve this look, the user supplies graphics in which each product is situated on a shelf "segment," such as:

When the graphics are placed side-by-side (with no space between them) and CBC by default

CBC Help

369

automatically places a border (usually black for most browsers) underneath the graphic, the resulting display looks like a continuous shelf. Note: when using the "shelf display" option, you will probably not want the brands to "shuffle" around randomly from task to task, but will want them always located in the same position. Make sure to select Sort Using the Natural Order option on the Design tab, and specify to sort by the natural order of the Brand attribute using the Settings... button.

Shelf Facing Settings

To create a shelf display look, you must tell CBC how many total rows (shelves) are to be used, and how many product concepts to display per row. For example, if you have 45 total products to display, you might choose to display them on 5 rows (shelves). The number of products shown per row might be something like: Row # 1 2 3 4 5 Number of Concepts on Row 7 8 10 9 11

Note that the widths of the shelves are entirely controlled by the width of the graphics used in the product concepts, together with the number of graphics you have chosen to display per row. CBC doesn't re-size your graphics--they are always displayed in their natural size.

Shelf Facing Format Settings

CBC controls the way the HTML tables are specified for shelf facing display. (We have provided CSS "hooks" within the HTML for shelf facing display.) There are a number of settings that control how the "shelf display" functions work. The following picture illustrates the area each setting affects:

For Top Attribute Background Color, we suggest you pick the same color as is used in the background of your graphics (Note: this color will only show on the screen if the graphics you develop have different heights, or if you chose a transparent background color in your graphics). It is usually a good idea to pick the same background color as used for the Task Background Color, which is selected on the

370

SSI Web v7 Format tab. Separation between Shelves adds a blank line between the shelves.

None Option Position in Shelf Facing Tasks

In typical CBC task displays, the "None" option is always shown as the last concept in each task. You can also use that option with shelf facing. However, shelf facing display permits more flexibility. You can move the "none" text and the radio button associated with the text into the CBC tasks' header or footer. This is not the "global" HTML header, but is the regular text header or footer that you edit under Write Questionnaire | Edit. To place the "None" concept in the header or footer of the task, type the function [%CBCNone( )%] in either the header or footer question text boxes of the CBC choice task, accessed through Write Questionnaire | Edit. You may want to supply some HTML to format this "None" concept exactly as you want it.

CBC Help

371

2.3.6

Customized CBC Questions Using "Free Format"

Warning: This is an advanced feature that if not executed properly can result in errors in design, data collection, and part-worth estimation. It requires careful, manual checking of Free Format HTML syntax, and double-checking the validity of your .CHO file prior to fielding the study.

Introduction
This section describes how to create custom CBC surveys that go beyond the standard layout formatting provided in SSI Web. To customize your CBC survey, you specify HTML in Free Format questions that build the custom look and "tap into" CBC's experimental design plans (to display the appropriate attribute text or graphics on the screen) using SSI Script system commands.

Designing/Programming the Questionnaire

1) Create a CBC Web study the standard way by entering attributes and levels, generating a design, and adding the CBC questions within the SSI Web questionnaire. (You can include the standard CBC layout, Constant-Sum layout, or Dual-Response None.) 2) To add custom HTML, create a Free Format question (one corresponding to each CBC task) starting with the prefix such as FFTASK indexed by the "Random#" portion of the reserved CBC question names (such as FFTASKRandom1, FFTASKRandom2, etc.) and place each Free Format question on the same page as its corresponding CBC question (preferably above it). We suggest you start from a standard CBC questionnaire and copy the relevant source HTML (using the View Source function in your browser) for that layout as produced by SSI Web (this is a good starting point, but will require additional modifications). If you are applying Styles to your survey, you may wish to refer to the styles documentation to see how to make your Free Format CBC questions use the style settings that are applied to standard CBC questions. 3) In the HTML that you create for the Free Format question, insert SSI Scripting that "taps into" the attribute text and the CBC experimental design. Use the following function to return the text for levels in the CBC design: [%CBCDesignLevelText (ExerciseName_CBCQuestionName, ConceptPosition#, LevelTextPosition#) %] For example [%CBCDesignLevelText (ExerciseName_Random3, 2, 1) %] returns the text for the first row of level text (which is usually the 1st attribute) of the 2nd concept of the 3rd random task. (In designs that use conditional display, it is possible for a graphic to be displayed as a new "row position" in the choice task in addition to the standard attribute text, and in this specific case LevelTextPosition# doesn't always correspond to attribute#.) We would suggest creating just one Free Format question and testing it to ensure it is correct. Then, after everything is working properly, you should copy that structure and create the other custom CBC questions. One useful way to test is to compare (preferably using the Local Test Server) your custom Free Format question with the regular CBC question on the same page. The text and graphics for the standard CBC concepts and the custom CBC concepts should match. Once you are confident that all is working as expected, you can then copy the structure, creating a similar Free Format question corresponding to each CBC question in the survey. Search and replace to change the syntax within your SSI scripting commands to the correct indices. For example you would need to change all the occurrences of Random1 to Random2 etc.

372

SSI Web v7 You must carefully check your HTML syntax for each custom CBC question to ensure that you have referred to the variable names correctly for each choice task. Warning: a mistake can invalidate a task, or potentially your entire CBC study. You can also visually inspect each custom CBC task, running under Local Test Server, comparing each standard CBC task with each custom built task. The attribute levels presented in each task should match. 4) Insert proper syntax for your radio buttons (or your numeric values, if using constant-sum tasks). Recall from Step 2 above that we suggested the free format questions be called FFTASKRandom1, FFTASKRandom2, etc. For each of your free format questions (one corresponding to each random choice task), create a Free Format variable named "_response". If you are using the constant-sum display for your CBC tasks, you will need to name the variables "_response1", "_response2", etc to match all of the possible concepts including any possible none-option. Standard CBC Tasks: Because CBC can sort the concept order by a user-specified attribute order prior to displaying the concepts on the screen (a setting on the Design tab of the CBC Exercise Settings dialog), recording the respondent answers correctly requires some special syntax. For standard radio buttons (discrete choice), format each button as follows: Concept 1: <input name="FFTASKRandom1_response" type="radio" value="[%CBCDesignConceptValue(ExerciseName_Random1, 1)%]"> Concept 2: <input name="FFTASKRandom1_response" type="radio" value="[%CBCDesignConceptValue(ExerciseName_Random1, 2)%]"> Concept 3: <input name="FFTASKRandom1_response" type="radio" value="[%CBCDesignConceptValue(ExerciseName_Random1, 3)%]"> ...for as many concepts in your question (not including the None concept). When showing radio buttons for random CBC task #2, make sure to increment to "Random2" within the script syntax. The None concept is always shown in the final position, therefore the radio button for a traditional None concept is specified as follows (assuming that the None concept is the 4th concept in the questionnaire): <input name="FFTASKRandom1_response" type="radio" value="4"> Constant Sum CBC Tasks: First, make sure to specify that you wish to use constant-sum CBC format, under the Format tab of the CBC Exercise Settings. For CBC Constant Sum the function is used like this: Concept 1: <input name="FFTASKRandom1_response[%CBCDesignConceptValue(ExerciseName_Random1, 1)%]" type="text" size="3"> Concept 2: <input name="FFTASKRandom1_response[%CBCDesignConceptValue(ExerciseName_Random1, 2)%]" type="text" size="3">

CBC Help

373

Concept 3: <input name="FFTASKRandom1_response[%CBCDesignConceptValue(ExerciseName_Random1, 3)%]" type="text" size="3"> (Note: SSI Web's syntax checker will warn you that it cannot find a "_response1" variable in the HTML syntax etc. corresponding to the variable names you specified in the free format interface. That is because you are dynamically inserting the integers at runtime using the CBCDesignConceptValue command syntax. Therefore, you can ignore this warning.) The numeric entry box for the None concept is done as follows (assuming that the None concept is the 4th and final concept in the questionnaire): <input name="FFTASKRandom1_response4" type="text" size="3"> For CBC Constant Sum you might also want to display a totals box. To do so, use this function: [%DisplayTotal(FirstQuestionName, LastQuestionName, Width)%] For example: [%DisplayTotal(FFTASKRandom1_response1, FFTASKRandom1_response4, 3)%] will display a totals box that will total the responses from FFTASKRandom_response1 through FFTASKRandom_response4. (Note: only use this function once per question. Calling this function multiple times for the same question will cause it not to work.) Dual-Response None CBC Tasks: First, make sure you have specified that you wish to use the Dual-Response None Option on the Format tab of the CBC Exercise Settings dialog. Add another variable within your FFTASKRandom1 free format question called "none" (FFTASKRandom1_none). The responses should be coded: <input name="FFTASKRandom1_none" type="radio" value="1">Yes<br> <input name="FFTASKRandom1_none" type="radio" value="2">No<br> 5) Test that the correct attribute level combinations are appearing in your custom CBC questions, by running the questionnaire in Test mode and comparing the attribute levels displayed in your free format questions with those displayed directly below them in the standard CBC tasks. 6) Suppress the regular CBC questions. You don't actually delete them from the questionnaire. We have created a fast way to toggle them to a suppressed mode. To suppress the regular CBC questions, add the following HTML comment to the Header 1 field of your CBC random choice tasks:
<!-SSI_Comment: CUSTOM_CBC_DESIGN Free_Format_Base_Name: FFTASK Free_Format_Variable_Name: response Verification: ON -->

If the above text is included in Header 1 of a random CBC task, the regular CBC question will not appear (note: it must be placed within the Header 1 field). The verification (that respondents have answered the question, or that the allocations sum to a particular value) are taken from your corresponding CBCRandom question settings. 7) Finally, test run the survey again, this time making sure that your answers are being recorded properly. If you have specified a concept sort order (on the Design tab of the CBC Exercise Settings

374

SSI Web v7 dialog), CBC sorts the concept order prior to displaying the concepts, so you need to verify the final data file to ensure that the answers correspond to the actual product concepts you chose. We suggest the following: while taking the questionnaire, print each question (with your answer). Download your data, export the data to a .CHO or .CHS file (.CHS is used for constant-sum layout), and compare the attribute levels recorded in the .CHO or .CHS file along with your answers to the printed questionnaire. We strongly encourage you to test your custom CBC questionnaire thoroughly, including checking the final data, to ensure that the attribute levels are presented correctly in the custom choice tasks, that the corresponding design information is being correctly saved to the .CHO or .CHS file, and that respondent answers to the custom choice tasks are also being stored correctly in the .CHO or .CHS file.

CBC Help

375

2.3.7

Volumetric CBC
(This is an advanced topic, and requires customized data processing of the .CHS file. See the section in this documentation on .CHS File Format for file format information.) In some situations, it might make sense to use allocation-based choice tasks, but to not force the answers to sum to any particular value. For example, for FMCG (fast-moving-consumer-goods) we might ask respondents how many of each product within a richly defined task they would purchase. There is some debate in the industry whether respondents can complete such a task reliably; but, if they could provide accurate answers to these questions, it would allow the researcher to examine whether the overall category volume would increase or decrease given a certain competitive scenario. Furthermore, we would be able to report absolute shares of unit volume using the market simulator rather than just relative shares of preference. We do not necessarily know the best way to model such data, but we offer the following suggestion (which has been applied by other experienced users as well): 1. Use a script-based *program to modify the .CHS file. First, change the header for each respondent so that it reflects that a "None" concept was asked in each task (though you didn't actually display a "None" concept in the questionnaire). 2. Add a new row of zeros to each choice task, reflecting the "None" concept. 3. For each respondent, scan the choice tasks and determine the largest volume of product ever "purchased" (the sum of allocations across concepts within a choice task). Call that volume max_volume (each respondent has a different value for max_volume). For each respondent, set the "None" alternative to receive zero allocation for that one choice task reflecting the highest max_volume. 4. For each respondent, set the allocation for the "None" concept equal to max_volume minus the volume "purchased" in each other task. 5. Estimate individual-level part-worth utilities using CBC/HB. 6. Import the part-worth utilities into the SMRT market simulator. Merge a weighting variable (through SMRT's File | Merge Variables) for each respondent, equal to max_volume. Use max_volume as the weight in weighted simulations (select Weights... from the Scenario Specification dialog). Include the "None" concept in market simulations (set the "None" weight to 1.0 under the Advanced Settings within the Scenario Specification dialog.) The resulting shares of preference are shares of volume. The share allocated to the "None" concept reflects potential sales volume not captured by the products in this particular simulation scenario. Notes and Cautions: One should take care that the use of max_volume as a weight does not result in a few respondents exerting too much weight on the final population estimates. Furthermore, relying on max_volume as the definitive potential volume "purchased" for each respondent and the relative weight against which the "None" parameter is scaled places a great deal of reliance on this one data point from a single choice task. If a respondent accidentally records an erroneous and unrealistically high volume for a single task, it can greatly affect the scaling of this respondent's data with respect to the "None" parameter and this respondent's influence upon the simulated shares of preference. CBC/HB normalizes the allocations within each task to sum to a constant value. For this reason, one should reapply the max_volume weight in weighted simulations. * We have written such a program to perform steps 1-4 above, and it is available for free download from http://www.sawtoothsoftware.com/products/ssiweb/tools.

376

SSI Web v7

2.4

CBC Paper-and-Pencil Studies

SSI Web lets you run your CBC studies via Web, over PCs (CAPI installation), or as paper-and-pencil questionnaires. The process of running paper-based CBC studies is very similar to computer-based studies. You should set up your attributes, levels, and compose your choice tasks in the same manner as if you were going to run a computerized study. However, you should probably choose plain black (text) on white (background) for questionnaire colors, unless you plan to print the questionnaires in color. For CBC, even though you will not have a typical "randomized" design with paper-based questionnaires (with, say, 500 questionnaire versions), you should still use CBC's random choice tasks. Rather than having so many possible versions of the questionnaire (perhaps a unique version for each respondent), you will create just a few versions (design blocks) of the questionnaire and assign respondents (randomly) to take one of the different questionnaire versions. We generally recommend that you include enough versions of the questionnaire so that the number of random choice tasks times the number of questionnaire versions is greater than or equal to 80 (assuming no prohibitions, and typical attribute level specifications). However, we urge you to spend extra time looking at the design efficiency report that was created when you generated your paper-andpencil design plan. With very few questionnaire versions, the design seed can have an effect on design efficiency. Also, you may want to take further steps to generate "dummy" response data (holding the number of respondents constant) under different design plans (e.g. three versions vs. four versions, etc.). Please see more information on testing your design in this documentation.

Saving the Questionnaire

When you click the Field | Create Paper & Pencil Interviews button, each version of your questionnaire is saved to a separate file within the Paper-and-Pencil folder within your study directory. The files are named STUDYNAME_EXERCISENAME_V#.htm, where STUDYNAME is the fourcharacter or less study name for your project, and # is the version number of the questionnaire (always beginning with version #1). These files are in HTML format and may be opened with most any recent word processing software. You should do any additional formatting you like to make the questionnaire appear as you wish prior to printing. Note: It is helpful to print the version# and task# with each task in a paper-and-pencil study. This gives an additional reference while performing data entry, helping ensure that you do not introduce errors when specifying the version numbers seen by respondents. An easy way to automatically label your choice tasks is to add a function [%CBCVersion( )%] in the header or footer of the choice task. When you do this, each task includes a label such as D - 1, T - CBCRAN1, meaning "Design #1, Task CBCRAN1."

Fielding the Questionnaire

Make sure when fielding the questionnaire that you are able to match the different questionnaire versions correctly with respondent answers. You should also try to randomly distribute the questionnaire versions among respondents, so that roughly equal numbers of respondents complete each version.

CBC Help

377

Data Entry
After you have collected the data, you prepare a .csv file containing the respondent answers. This file follows a required .csv (comma separate values) format, with specific labels in the first row of the file. To see that format, prepare a template for yourself by clicking Field | Create Accumulated Data Template File.... A file named Accumulated Data.csv is saved to your study folder. Use this template to arrange your data, and make sure to use the prescribed header row within your data file. The layout is: Column 1: Respondent# Column 2: Version# Column 3: CBC answer #1, etc. Any missing data are coded as blank (meaning, if you viewed the data file using Excel, the cell would be empty; if you viewed the file with a text editor, the missing value would look like two consecutive commas with no text in between). Accumulate Respondent Answers: Click Field | Accumulate Paper & Pencil Data... and browse to the .csv file containing your respondent data.

378

SSI Web v7

2.5
2.5.1

Analyzing the Data

Preparing Conjoint Data for SMRT
To analyze the conjoint-related results of your CBC study, you must prepare (export) the basic data files for use within the SMRT software system. You received a copy of SMRT for analyzing CBC data with your purchase of CBC. The two files that CBC will automatically create for you for use in the SMRT software are: STUDYNAME_EXERCISENAME.CHO file ( text-only .CHO data file) STUDYNAME_EXERCISENAME.ATT file (text-only text file of attribute level labels) Prior to preparing the .CHO and .ATT files, you should have downloaded your data from the Web server to your study directory on your hard drive. When you click File | Export Data | Prepare CBC Data Files (*.cho & *.att), the .CHO and .ATT file are written to your study directory. Please see the SMRT documentation (hard copy) for instructions related to analyzing the data within SMRT. The .CHO and .ATT files can also be used within our Latent Class and Hierarchical Bayes estimation software modules for CBC. And, if you have used constant-sum (allocation) responses for CBC, you can estimate part-worth utilities using CBC/HB. (Please see Appendix A for more information on interpreting conjoint analysis results.) Question Types This section lets you filter which types of choice tasks will be exported to the .CHO (or .CHS) file (Random, Fixed, or both). Most users will wish to export both types of choice tasks (Random and Fixed). All Sawtooth Software analytical systems for analyzing CBC results (SMRT, Latent Class, CBC/HB, HB-Sum) include a way to select which choice tasks will be used for utility estimation. Therefore, even if you include both Random and Fixed choice tasks in the .CHO file, you can set within the analytical software filters to selectively use the Random tasks for estimation. Respondent Numbering You can choose from five options for assigning respondent numbering. This lets you specify how the respondent numbers (case IDs) are to be generated. User Name is one of the passwords from the passwords module. You should only choose this if these values were purely numeric (no letters). Password is also one of the passwords from the passwords module. You should also only use this if the passwords were purely numeric (no letters). Internal Interview Numbers are those that the system assigns when respondents complete the survey over the Web. There can be gaps in the sequence if some respondents started a survey but didn't finish. Sequentially Assigned Numbers (default) are integers 1 through n. Other lets you choose another numeric variable to use for respondent numbering. This last option is useful if you are linking surveys from another source and need to merge the data sets by a "passed-in" variable. Include All Respondents Choose this option to export all respondents: both complete and incomplete. Data for questions that were not completed are left blank. Qualified/Completes Only Choose this option to export only complete data records. A survey is considered qualified complete when respondents reach a terminating question that is marked with a qualified/complete status. Disqualified and Incompletes Only Choose this option to export only disqualified and incomplete data records. A survey is considered disqualified when respondents reach a terminating question that is marked with a

CBC Help

379

disqualified status. A survey is considered incomplete if respondents have not reached a terminating question.

380

SSI Web v7

2.5.2

Moving Data from SSI Web to SMRT for Analysis

If conducting conjoint analysis studies (CBC, ACBC, ACA and CVA) with SSI Web, you'll often want to move the data from SSI Web into the SMRT software package for analysis (market simulations, counting analysis, logit, etc.). Sometimes data are moved directly into SMRT, and other times data are moved from SSI Web into a preliminary analysis program such as hierarchical Bayes, and then afterward to SMRT.

CBC (Choice-Based Conjoint) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. Export the CBC data (just the data directly involving the choice tasks) to a studyname.CHO (text-only format) file. From the SSI Web menu, click File | Export Data | Prepare CBC Data files (*.cho/*.chs and *.att). 2. Start the SMRT software (a different program from SSI Web, also downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the .CHO file into SMRT, by clicking (from the SMRT software menu) File | Import, browsing to your .CHO file, and clicking Import. The raw choice data (respondent answers and experimental design) are now within the SMRT study. You can proceed with counting analysis or logit within SMRT. Note: If you use CBC/HB or Latent Class software to estimate part-worth utilities, use the .CHO (or .CHS, in the case of constant sum responses to be analyzed with CBC/HB) and .ATT files with those systems (see their respective manuals for details) to produce a data file containing respondent partworths (a .HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .HBU or Latent Class probability file (.Pxx). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

ACBC (Adaptive Choice-Based Conjoint) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. Estimate part-worth utilities within SSI Web by clicking Analysis | Calculate ACBC Utilities Using HB.... This process creates a text-only format file called STUDYNAME.hbu. 2. Start the SMRT software (a different program from SSI Web, also downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new

CBC Help study functions independently of your original SSI Web study.

381

4. Import the STUDYNAME.hbu file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CBC utilities (selecting .HBU file type). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use CBC/HB or Latent Class software to estimate part-worth utilities, use the .CHO (or .CHS, in the case of constant sum responses to be analyzed with CBC/HB) and .ATT files with those systems (see their respective manuals for details) to produce a data file containing respondent partworths (a .HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .HBU or Latent Class probability file (.Pxx). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

ACA (Adaptive Conjoint Analysis) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. If using the default OLS (Ordinary Least Squares) part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate ACA Utilities using Ordinary Least Squares. This process creates a text-only format file called STUDYNAME.utl. 2. Start the SMRT software (a different program from SSI Web, downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing ACA utilities (selecting ACA v4.x Utilities *.utl as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use ACA/HB software to estimate part-worth utilities, you can either run ACA/HB as a separate program, or if you own ACA/HB v3 or later, you can run the program as an integrated component from within the SSI Web interface. If running the estimation outside of SSI Web, from the SSI Web menu, select File | Export Data | Prepare ACA Data Files (*.acd and *.aca)... to export the ACA information to a STUDYNAME.acd format into the same folder containing your ACA/HB system. See the ACA/HB manual for details of producing a data file containing respondent part-worths (a .HBU file). After estimating part-worth utilities using either ACA/HB approach (within SSI Web menu or outside program), import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .hbu file. Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

382

SSI Web v7

CVA (Conjoint Value Analysis) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. If using the OLS (Ordinary Least Squares) part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate CVA Utilities using Ordinary Least Squares. If using the Monotone Regression part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate CVA Utilities using Monotone Regression. Either process creates a text-only format file called STUDYNAME.utl. 2. Start the SMRT software (a different program from SSI Web, downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CVA utilities (selecting CVA Traditional Conjoint Utilities *.utl as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use CVA/HB software to estimate part-worth utilities, run the program as an integrated component from within the SSI Web interface. After estimating part-worth utilities using the CVA/HB approach, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .hbu file. Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

Moving Segmentation (Generic Survey) Data from SSI Web to SMRT

We assume you have first followed the procedures above for estimating part-worth utilities and establishing those as a utility run in a new project within SMRT. 1. From the SSI Web system, export the data you wish to merge into SMRT. From the SSI Web menu, select File | Export Data | All Data, select the respondent numbering method (the numbering method must match the method you chose when exporting the conjoint data or estimating utilities), and export the data you wish to merge into SMRT (only numeric data may be merged). Select either a .CSV file (comma-delimited text-only file) or a fixed text-only file (.TAB). 2. Within the SMRT software, open the project previously established that contains the conjoint data/utility run. Use File | Merge Variables | Add to merge the data into your SMRT project (see the SMRT manual or online help within SMRT for details). After merging the data, if using SMRT v4.11 or earlier, you'll additionally need to specify Custom Segmentation Variables (Under Analysis | Custom Segments) if you wish to use the new variables as banner points or respondent filters during simulations.

CBC Help

383

2.5.3

.CHO File Format

You can export CBC data (discrete choice) from SSI Web to a .CHO format. The .CHO format files are used in many other Sawtooth Software programs for analyzing CBC data, including Latent Class, and Hierarchical Bayes (HB) applications. Some power-users modify the .CHO file (usually with an automated script outside of SSI Web) for certain advanced applications. The format is as follows:

8960 6 1 3 1 2 1 3 3 4 2 2 32 . . .

2 2 3 1

6 3 1 2

12 2 3 1 3 1 2

(lines 3-7 repeated, for 11 more tasks for this respondent) Line 1: Respondent record information 8960 2 6 Respondent Number "Extra" Variables Number of Attributes

12 Number of Choice Tasks

1 None option 0=N, 1=Y, 2=Dual Response None

Line 2: "Extra" Variables (may include as many numbers as specified on Line 1) 6 1 Interview Duration Segmentation Variable

Lines 3-7 are repeated for each choice task Line 3: Task 1 format 3 1 Number of concepts in first task Depth of preference in first task (always 1)

Line 4: Levels displayed in first concept 2 1 2 Level 2 of Attribute 1 Level 1 of Attribute 2 Level 2 of Attribute 3

3 Level 3 of Attribute 4

2 Level 2 of Attribute 5

3 Level 3 of Attribute 6

Note: Missing levels (partial profile and alternative specific designs) are indicated with "0"'s.

384

SSI Web v7 Line 5: Levels displayed in second concept 3 3 3 Level 3 of Attribute 1 Level 3 of Attribute 2 Level 3 of Attribute 3

1 Level 1 of Attribute 4

3 Level 3 of Attribute 5

1 Level 1 of Attribute 6

Line 6: Levels displayed in third concept 4 2 1 Level 4 of Attribute 1 Level 2 of Attribute 2 Level 1 of Attribute 3

2 Level 2 of Attribute 4

1 Level 1 of Attribute 5

2 Level 2 of Attribute 6

Line 7: Choice information 2 32 Concept Chosen Task duration

* In this example, the none option, if chosen, would be represented as concept 4. ** Missing answers are indicated by Concept Chosen set to "0."

Coding Dual Response None Answers

In the standard .CHO layout, the coding of each task is completed by a line with two values: "Choice" and "Task Duration." With the "Dual-Response None" format, each choice task is completed by a line with four values, such as: Line 7: Dual-Response None Choice information 3 27 1 4 1st stage choice Task duration Buy=1 No Buy=2 Task duration

CBC Help

385

2.5.4

.CHS File Format

The .CHS file format is very similar to the .CHO file format, and is used for constant sum (allocation) style CBC studies. The .CHS format files are used in Sawtooth Software's CBC/HB program. Some power-users modify the .CHS file (usually with an automated script outside of SSI Web) for certain advanced applications. The format is as follows:

8960 6 1 3 2 1 3 3 4 2 . . .

2 2 3 1

6 3 1 2

12 2 3 1 3 1 2

0 7 3 0

(lines 3-6 repeated, for 11 more tasks for this respondent) Line 1: Respondent record information 8960 2 6 Respondent Number "Extra" Variables Number of Attributes

12 Number of Choice Tasks

0 None option 0=N, 1=Y

Line 2: "Extra" Variables (may include as many numbers as specified on Line 1) 6 1 Interview Duration Segmentation Variable

Lines 3-6 are repeated for each choice task Line 3: Task 1 format 3 Number of concepts in first task Line 4: First concept information 2 1 2 Level 2 of Attribute 1 Level 1 of Attribute 2 Level 2 of Attribute 3

3 Level 3 of Attribute 4

7 Point allocation

Level 2 of Level 3 of Attribute 5 Attribute 6

Note: Missing levels (partial profile and alternative specific designs) are indicated with "0"'s. Line 5: Second concept information 3 3 3 Level 3 of Attribute 1 Level 3 of Attribute 2 Level 3 of Attribute 3

1 Level 1 of Attribute 6

3 Point allocation

Level 1 of Level 3 of Attribute 4 Attribute 5

386

SSI Web v7 Line 6: Third concept information 4 2 1 Level 4 of Attribute 1 Level 2 of Attribute 2 Level 1 of Attribute 3

2 Level 2 of Attribute 4

1 Level 1 of Attribute 5

2 Level 2 of Attribute 6

0 Point allocation

Note: if a "None" concept is present, it is included as the last alternative in the task, with all attribute level codes as "0."

Adaptive CBC (ACBC) Help

387

3
3.1
3.1.1

Adaptive CBC (ACBC) Help

Getting Started with ACBC
Welcome to Adaptive CBC!
We're excited about Adaptive CBC (ACBC) and think it will change the way we think about and conduct stated choice research for complex problems involving about five or more attributes. Sawtooth Software's products have benefited immensely from the interaction between practitioners and the academic community. We are not secretive about the algorithms and methodologies so that others may scrutinize and test our work. At our conferences and in the academic literature, the methods have been examined and tested. Some investigations have been favorable toward our work, while others have been critical. When the criticism has merit, we strive to improve the techniques. The first version of ACBC continues this tradition. Over the last two years, we have presented the methodology and results at both academic and practitioner research conferences. We have submitted it for review among leading academics in discrete choice modeling. It has undergone a thorough beta test process with over 50 users. Over 50 studies were fielded and analyzed during an eight-month beta test program. The feedback from the beta testers (and their clients) has been more than positive. Since the release of ACBC in March 2009, many more projects have been completed. Two papers were delivered at the Sawtooth Software conference in March that highlighted successful applications. Two additional case studies were delivered at the May 2009 joint SKIM/Sawtooth Software event in Prague, Czech Republic on ACBC. We are proud to add ACBC to our suite of conjoint-related tools, and expect that it will quickly become a valued standard in the industry.

388

SSI Web v7

3.1.2

ACBC Specifications
ACBC can be used to study up to: 100 attributes 250 levels per attribute Unlimited number of respondents Typical studies in practice will cover about 5 to 12 attributes, with about 2 to 6 levels per attribute. However, because the attribute list can be "constructed" (customized) for each respondent, it is possible to study many more attributes and many more levels per attribute (for example, levels of "brand") than would be prudent using traditional full-profile conjoint methods. But whenever omitting attributes and levels from a respondent's design, some assumptions must be made, and we recommend the user carefully review the section entitled: Customized (Constructed) Attribute and Level Lists.

Adaptive CBC (ACBC) Help

389

3.1.3

Overview
Adaptive CBC (ACBC) is a component within the SSI Web suite of Internet interviewing products. ACBC is used for conducting adaptive Choice-Based Conjoint (CBC) studies over the web or in nonweb "CAPI" interviewing mode (but not paper-and-pencil questionnaires). ACBC questions are usually integrated within a longer SSI Web questionnaire that includes other standard survey questions. The number of additional questions that can be added depends on your license. An ACBC study can include additional standard survey questions that collect up to 50 data fields. An unlimited number of additional questions can be used if the largest CiW interviewing license is acquired. ACBC studies are used for learning about respondents' preferences for the combinations of features that make up products or services. We recommend that ACBC be used in instances where there are about five or more attributes. Studies with few attributes (such as brand+package size+price) commonly done with standard CBC are probably not appropriate for ACBC. ACBC analysis can help with (among other things) product design, line extensions, pricing research, and market segmentation. Your license entitles you to free technical support via phone or email (support@sawtoothsoftware.com). We encourage you to contact us at support@sawtoothsoftware.com or 360/681-2300 if you cannot find the answer to your problem or question in this documentation. We are available Mon-Fri, 7:00 am to 5:00 pm Pacific time. (Note: we provide full technical support for composing and testing your Internet survey on your local PC, and provide written instructions on how to install your questionnaire on your server. We cannot take responsibility for or be expected to provide technical support to debug any customized HTML or JavaScript you may add to your survey. You are also responsible to make sure that Perl is running and that you can run Perl (CGI) scripts on your web server. We can provide some assistance related to SSI Web's use with your server once the fundamental Perl functionality is in place.) A sample ACBC questionnaire (lapt.ssi) that includes 250 respondent records is available in the \Samples folder within your SSI Web installation. You can easily access this example study by clicking Help | Sample Studies | ACBC Sample. This questionnaire lets you see an example of an ACBC survey, investigate how it is set up, and perform analysis. If you know a little HTML (which is not necessary to use ACBC), you can employ some "tricks" to help you customize the look of your survey beyond the standard options provided by the software. By opening the sample study and looking at the settings and text we've used, you can become familiar with some of the possibilities. Please note that ACBC is the most advanced conjoint analysis system that we've designed, and the user should have the appropriate experience and training to ensure good results. ACBC is also a new methodology, so we recommend paying close attention to the published research on ACBC. Using ACBC requires the following skills and knowledge base: The analyst managing the survey should be proficient in conjoint analysis, including defining attributes and levels properly, correctly phrasing the questionnaire text and response scales, interpreting the part-worth utilities, and running simulations. It is not necessary to be a statistician to use our software effectively, but knowledge of basic statistics for market research is highly recommended. The questionnaire is first composed locally on a PC running under Windows XP, Vista, or Windows 7. The individual inputting/formatting the survey should have working knowledge of standard Windows applications such as word processing, Web browsers and/or spreadsheets. Though it is not necessary, some knowledge of HTML is valuable. For web-based data collection, one must have access to an Internet/intranet server running Perl 5.003 or later. This can be on your company's own Web server, or your ISP's (Internet Service Provider's) Web server. The user must know how to use FTP software to create directories, upload files to those directories, and set permissions on the files and directories. Click here to receive detailed instructions on the directories, files and permissions. (Sawtooth Software also

390

SSI Web v7 provides web hosting services for SSI Web studies.)

Relationship between ACBC and SMRT

ACBC is used for designing, creating, and fielding ACBC studies (either web-based or non-web "CAPI" interviewing). But you actually perform market simulations using the SMRT software. These two software systems are installed and run independently--but SMRT imports the .HBU file containing partworth utilities. If you don't already own SMRT software, you receive a special copy with your ACBC license to analyze your ACBC data.

Steps in Conducting an ACBC Study

1) Formulate attributes and levels and type them into the ACBC interface. 2) Make decisions regarding your study design: decide how to organize and lay out your BYO, Screener, Choice Tasks and (optionally) Calibration sections. Using the point-and-click ACBC interface, add those sections with their associated questions to your SSI Web study. 3) Test your questionnaire in preview mode, by using the Local Test Server included in SSI Web, or by uploading the files to your Web server. Compute utilities for very small sample sizes (even n=1) using the method of Monotone Regression. Examine the utilities for your test questionnaires. Make modifications and re-test. 4) If using web-based data collection, upload your study to an Internet server, and continue testing--by yourself, with colleagues, and later preferably with a small number of pilot respondents. Receive feedback from colleagues and pilot respondents. Examine the data to make sure all data are being stored properly using the Admin Module, and by downloading the data to your hard drive. Compute utilities using monotone regression (or using HB if your sample size is large enough). Import the results into the SMRT software package for market simulation analysis. 5) After ensuring that your ACBC questionnaire works properly, that the data are being stored correctly, and that your questionnaire produces reasonable results, invite respondents to take the survey. 6) Monitor the data collection using the Admin Module. Download the data, and then conduct analysis: tabulate the BYO choices, as well as the indications of "Unacceptables" and the "Must-Haves" for levels. Estimate part-worth utilities in preparation for running market simulations. See the SMRT manual or online help within the SMRT software for documentation regarding market simulations.

Adaptive CBC (ACBC) Help

391

3.1.4

Motivation for Adaptive CBC

Background
Choice-Based Conjoint (CBC) has been the most widely used conjoint technique among our user base since about the year 2000. The marketing research community has adopted CBC enthusiastically, for several reasons. Choice tasks seem to mimic what actual buyers do more closely than ranking or rating product concepts as in conventional conjoint analysis. Choice tasks seem easy for respondents, and everyone can make choices. And equally important, multinomial logit analysis provides a welldeveloped statistical model for estimating respondent partworths from choice data. However, choice tasks are less informative than tasks involving ranking or rating of product concepts. For this reason, CBC studies have typically required larger sample sizes than ratings-based conjoint methods. In CBC studies, the respondent must examine the characteristics of several product concepts in a choice set, each described on several attributes, before making a choice. Yet, that choice reveals only which product was preferred, and nothing about strength of preference, or the relative ordering of the non-preferred concepts. Initially, CBC questionnaires of reasonable length offered too little information to support multinomial logit analysis at the individual level. More recently, hierarchical Bayes methods have been developed which do permit individual-level analysis, but interest has remained in ways to design choice tasks so as to provide more information.

Problems with Traditional CBC

In recent years marketing researchers have become aware of potential problems with CBC questionnaires and the way respondents answer CBC questions. The concepts presented to respondents are often not very close to the respondent's ideal. This can create the perception that the interview is not very focused or relevant to the respondent. Respondents (especially in Internet panels) do choice tasks very quickly. According to Sawtooth Software's experience with many CBC datasets, once respondents warm up to the CBC tasks, they typically spend about 12 to 15 seconds per choice task. It's hard to imagine how they could evaluate so much information on the screen in such short order. It seems overwhelmingly likely that respondents accomplish this by simplifying their procedures for making choices, possibly in a way that is not typical of how they would behave if buying a real product. To estimate partworths at the individual level, it is necessary for each individual to answer several choice tasks. But when a dozen or more similar choice tasks are presented to the respondent, the experience is often seen to be repetitive and boring, and it seems possible that respondents are less engaged in the process than the researcher might wish. If the respondent is keenly intent on a particular level of a critical attribute (a "must have" feature), there is often only one such product available per choice task. Such a respondent is left with selecting this product or "None." And, respondents tend to avoid the "None" constant, perhaps due to "helping behavior." Thus, for respondents intent on just a few key levels, standard minimal overlap choice tasks don't encourage them to reveal their preferences much more deeply than the few "must have" features. Gilbride et al. (2004) and Hauser et al. (2006) used sophisticated algorithms to examine patterns of respondent answers, attempting to discover simple rules that can account for respondent choices. Both groups of authors found that respondent choices could be fit by non-compensatory models in which only a few attribute levels are taken into account. We find that when choice sets are composed so as to have minimal overlap, most respondents make choices consistent with the hypothesis that they pay attention to only a few attribute levels, even when many more are included in product

392

SSI Web v7 concepts. In a recent study with 9 attributes, 85 percent of respondents' choices could be explained entirely by assuming each respondent paid attention to the presence or absence of at most four attribute levels. Most CBC respondents answer more quickly than would seem possible if they were giving thoughtful responses with a compensatory model (a model that assumes that respondents add the value of each feature and choose the product concept with the highest overall utility). Most of their answers can be accounted for by very simple screening rules involving few attribute levels. Combine those facts with the realization by anyone who has answered a CBC questionnaire that the experience seems repetitive and boring, and one is led to conclude there is a need for a different way of asking choice questions, with the aim of obtaining better data. There has been a lot of effort dedicated to designing efficient CBC experiments featuring orthogonality and high D-efficiency. These efforts have assumed that respondents answer using an additive process consistent with the logit rule. We have become increasingly convinced that most respondents to complex conjoint studies employ non-compensatory heuristics at odds with the logit rule, and that efforts to improve design efficiency assuming compensatory processing may be misdirected. ACBC's design strategy is effective for respondents that employ various degrees of non-compensatory and compensatory processes. In terms of the traditional design criteria, ACBC's designs would be judged inferior. But this is an inappropriate standard given that so many respondents apply cutoff rules, such as screening based on unacceptable and must-have levels.

The Quest for Better Data

We believe CBC is an effective method that has been of genuine value to marketing researchers, but that it can be improved. And we believe the greater need at this point is not for better models, but rather for better data. Adaptive CBC is a new, promising method that responds well to the problems above. It uses the trusted full-profile method of presenting product concepts. Its surveys are more engaging for respondents. Our first few rigorous comparison studies to traditional CBC suggest that our adaptive form leads to data that are more accurate and predictive of choice behavior. It captures more information at the individual level than traditional CBC surveys and may be used even with small (for example, business-to-business) samples. Adaptive CBC is best applied for conjoint-type problems in which there are about five attributes or more. Studies involving few attributes (such as the brand+package+price studies commonly done in FMCG research) wouldn't seem to benefit from ACBC. The store-shelf approach in our standard CBC software would seem more appropriate for such studies.

Adaptive CBC (ACBC) Help

393

3.1.5

Sections and Flow for Adaptive CBC

The objectives of the Adaptive CBC interview are as follows: Provide a stimulating experience that will encourage more engagement in the interview than conventional CBC questionnaires. Mimic actual shopping experiences, which may involve non-compensatory as well as compensatory behavior. Screen a wide variety of product concepts, but focus on a subset of most interest to the respondent. Provide more information with which to estimate individual partworths than is obtainable from conventional CBC analysis. The interview has several sections, with each section quite different from the previous.

BYO (Configurator) Section:

In the first section of the interview the respondent answers a "Build Your Own" (BYO) question to introduce the attributes and levels, as well as to let the respondent indicate the preferred level for each attribute, taking into account any corresponding feature-dependent prices. A typical screen for this section of the interview is shown below:

394

SSI Web v7

An alternate display incorporates combo boxes rather than radio buttons:

Adaptive CBC (ACBC) Help

395

Past research has shown that respondents enjoy BYO questionnaires and answer them rapidly, and that the resulting choices have lower error levels than repetitive choices from CBC questionnaires. Based on answers to the BYO questionnaire, we create a pool of product concepts that includes every attribute level, but for which attribute levels are relatively concentrated around the respondent's preferred attribute levels. Notes: The BYO-selected product concept can be shown in later sections of the SSI Web survey using SSI Script Functions. Some attributes with obvious a priori preference order would seem to be out of place within a BYO question. It may not make sense in some situations to assign price premiums to the preferred levels, and it would be "obvious" to ask respondents to indicate their preferred level. In those cases, you can drop such attributes from the BYO question type (but these attributes appear within the remaining sections of the ACBC survey). For attributes without obvious a priori preference order, it would generally make sense to include them within the BYO question to learn respondents' preferences. Then, ACBC could focus on product concepts similar to those preferences. But, researchers may choose to exclude such attributes from the BYO section, if they desire.

Screening Section:
In the second section of the interview the respondent answers "screening" questions, where product concepts are shown a few at a time (we recommend showing 3 to 5 at a time per screen, for about 7 to 9 total screens of concepts). In the Screening Section, the respondent is not asked to make final choices, but rather just indicates whether he/she would consider each one "a possibility" or "not a possibility." A typical screen from this section of the interview is shown below:

The Screening Section is also used to estimate the "None" parameter threshold.

396

SSI Web v7

Unacceptables:
After a few screens of concepts have been evaluated, we scan previous answers to see if there is any evidence that the respondent is using non-compensatory screening rules (meaning, that there are cutoff rules that are absolutes and cannot be compensated for by the presence of enough other good features). For example, we might notice that he/she has avoided some levels of an attribute, in which case we ask whether any of the consistently avoided levels is an "Unacceptable". Here is a typical screen for this question:

Past research with ACA has suggested that respondents are quick to mark many levels as unacceptable that are probably just undesirable. We considered that the same tendency might apply here. To avoid this possibility, we only offery cutoff rules consistent with the respondent's previous choices and we allow the respondent to select only one cutoff rule on this screen. After each screen of typically three to five products has been screened (as a "possibility" or not), another "unacceptable" screen is shown and the respondent has another opportunity to add a subsequent cutoff rule. If the respondent identifies any "unacceptable" levels, then all further concepts shown will avoid those levels. Unacceptable questions provide a less aggressive way for respondents to indicate non-compensatory, cut-off rules than "Must-Haves" (described below), and for that reason we recommend giving it precedence in the ACBC questionnaire. A recent split-sample research project we conducted suggested that the Unaccepable question be asked first. We suggest waiting to ask Must-Have questions until at least two Unacceptable questions have been asked. By default, we do not ask respondents if a certain price level is unacceptable (for Summed Pricing), but you can change that if you'd like (on the Pricing tab). If you include price in Unacceptables questions, ACBC scans the previous answers to determine the highest price ever selected as "a possibility" in the Screener or BYO questions. That highest price is offered as an unacceptable threshold. The fact that a level has been marked unacceptable can be used in skip patterns (such as to ask a follow-up question regarding why a level was marked unacceptable) using SSI Script Functions.

Must Haves:
If the only products that a respondent has marked "a possibility" contain certain attribute levels (or

Adaptive CBC (ACBC) Help

397

ranges of levels for ordered attributes), we ask whether that level is a Must-Have. For example:

After each screen of typically three to five products has been screened (as a "possibility" or not), another "must have" screen is shown and the respondent has another opportunity to add a subsequent cutoff rule. If the respondent identifies any "must have" levels, then all further concepts shown will satisfy those requirements. The fact that a level has been marked a must-have can be used in skip patterns (such as to ask a follow-up question regarding why a level was marked a must-have) using SSI Script Functions.

Choice Tasks Section:

Once the respondent has completed the planned number of screens of Screening questions (typically 7 to 9 screens, where each screen includes 3 to 5 concepts), we transition to the Choice Tasks Section (tournament). The respondent is shown a series of choice tasks presenting the surviving product concepts (those marked as "possibilities") typically in groups of three, as in the screen below:

398

SSI Web v7

At this point, respondents are evaluating concepts that are close to their BYO-specified product, that they consider "possibilities," and that strictly conform to any cutoff (unacceptable/must-have) rules. To facilitate information processing, we gray out any attributes that are tied across the concepts, leaving respondents to focus on the remaining differences. Any tied attributes are typically the most key factors (based on already established cutoff rules), and thus the respondent is encouraged to further discriminate among the products on the features of secondary importance. The winning concepts from each triple then compete in subsequent rounds of the tournament until the preferred concept is identified. If displaying concepts in triples, it takes t/2 choice tasks to identify the overall winner, where t is the number of concepts marked as "possibilities" from the previous section. The winning product concept can be shown in later sections of the SSI Web survey using SSI Script Functions.

Calibration Section (Optional):

The fourth section of the interview is optional, and may be used to estimate a different "None" parameter from that provided by the Screening Section. The respondent is re-shown the concept identified in the BYO section, the concept winning the Choice Tasks tournament, and (typically) four others chosen from among both previously accepted and rejected concepts. For each of those concepts we ask how likely he/she would be to buy it if it were available in the market, using a standard five-point Likert scale, with a screen similar to the one below:

Adaptive CBC (ACBC) Help

399

This section of the interview is used only for estimation of a partworth threshold for "None." Partworths from other sections of the interview are used to estimate the respondent's utility for each concept, and then a regression equation is used to produce an estimate of the utility corresponding to a scale position chosen by the researcher, such as, for example, somewhere between "Might or Might Not" and "Probably Would." Within the market simulator, if the utility of a product concept exceeds the None utility threshold, it is chosen. The order of Calibration Concepts is as follows: BYO "Not a possibility" concept A "winner" from the tournament A "loser" from the tournament (Repeat pattern of last three, if needed) Winning concept from the tournament

Summary:

400

SSI Web v7

Based on our experience, we recommend the following flow for an Adaptive CBC questionnaire: Page # 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. Description Introductory Text BYO (or Most Likelies) Transition Text Screener #1 (showing about 4 concepts) Screener #2 (showing about 4 concepts) Screener #3 (showing about 4 concepts) Unacceptables #1 Screener #4 (showing about 4 concepts) Unacceptables #2 Must Have #1 Screener #5 (showing about 4 concepts) Unacceptables #3 Must Have #2 Screener #6 (showing about 4 concepts) Unacceptables #4 Must Have #3 Screener #7 (showing about 4 concepts) Unacceptables #5 Must Have #4 Screener #8 (showing about 4 concepts) Transition Text Choice Tasks Set #1 Choice Tasks Set #2 Choice Tasks Set #3, etc. until winning product is identified Transition Text Calibration Concept #1 (optional) Calibration Concept #2 (optional) Calibration Concept #3 (optional) Calibration Concept #4 (optional) Calibration Concept #5 (optional) Calibration Concept #6 (optional)

While the schematic above gives a general recommendation regarding an appropriate ACBC interview, the recommended number of questions in each section depends on the number of attributes in the study and how accurate results are desired at the individual level. For more detailed recommendations, please see the next section entitled Design Tab (ACBC). In our experience, ACBC questionnaires take about 7 to 15 minutes (median survey length) when respondents are asked to trade off 9 to 10 attributes each having from 2 to 6 levels. While this is

Adaptive CBC (ACBC) Help

401

longer than traditional CBC questionnaires, respondents find the process more engaging and we capture more information to model each respondent accurately. The interview as a whole attempts to mimic the actual in-store buying experience that might be provided by an exceptionally patient and interested salesperson. For example, after the BYO section we might explain that this exact product is not available but many similar ones are, which we will bring out in groups to see whether each is worthy of further interest. The Choice Tasks section is presented as an attempt to isolate the specific product which will best meet the respondent's requirements. If the respondent has answered conscientiously, he/she will find that the final product identified by ACBC as best is actually more preferred than the original BYO product. This occurs because the overall prices of the products generated in the product pool are varied from the fixed BYO prices (assuming "Summed Pricing" has been used). Therefore, typically at least one of the product concepts will have better features than the BYO product at the same price, the same features at a lower price, or a combination of these benefits. This makes it seem that the ACBC interview has actually done a good job finding a product that exceeds the quality of the BYO product and fits the needs of the respondent.

402

SSI Web v7

3.1.6

Design Tab (ACBC)

For general information about ACBC interview setup, please refer to the section entitled: Sections and Flow for Adaptive CBC. The key decisions to make when designing your ACBC study are: How many total product concepts (cards) will each respondent evaluate? This is equal to the Number of Screening Tasks x Number of Concepts per Screening Task. We generally recommend asking each respondent to evaluate between 18 to 40 product concepts, depending on the attribute list length and complexity (see recommendations in the table below). A good rule of thumb is to include enough cards so that each level taken forward to the ACBC survey appears at least twice, and preferably three times per respondent. How many attributes to vary from the BYO selections when generating product concepts (cards)? At the beginning of the interview, the respondent indicates which levels are "best" in the product they'd buy. We call this configured product the "BYO Concept." When generating new product concepts for respondents to evaluate, we choose near-neighbors to the BYO Concept, by varying a subset of the attributes that have been included in the BYO Concept. For example, assume that there are 8 total (non-price) attributes in your study, and all were included in the BYO question. If you choose to vary from 2 to 4 of the attributes (Minimum Attributes to Vary = 2; Maximum Attributes to Vary = 4), a new near-neighbor product concept could be generated by varying 3 of the attributes from their BYO-selected levels and retaining the other 5 attributes at their BYO-selected levels. This idea is illustrated below. To generate a near-neighbor concept, attributes 3, 4, and 7 have been selected to be varied from their BYO-selected levels. The changed levels are indicated in green, italics: Attribute # 1 2 3 4 5 6 7 8 BYO Concept Level 3 Level 2 Level 4 Level 1 Level 2 Level 3 Level 5 Level 1 Near-Neighbor Concept Level 3 Level 2 **Level 1** *Level 3** Level 2 Level 3 **Level 4** Level 1

The software will not allow you to vary more than half (+1) of the attributes included/chosen in the BYO exercise. How many Unacceptables and Must-Haves questions to ask? The choice of how many Unacceptables and Must Haves questions largely depends on how many Screening Tasks you decide to display. Please refer to Sections and Flow for Adaptive CBC for illustrations of the typical layout, and how the Unacceptables and Must Haves are positioned within an ACBC study. Generally, we suggest asking the first Unacceptables question after three Screening Tasks have been shown. Then, we recommend showing the first Must Haves question after the second Unacceptables question. This leads to the easy rule of thumb: # Unacceptables = (# Screening Tasks - 3) # Must-Haves = (# Screening Tasks - 4) The more Unacceptable and Must-Haves questions that are asked, the larger the percent of levels

Adaptive CBC (ACBC) Help

403

respondents mark as Unacceptables will result. One can increase the number of Unacceptables and Must-Have questions by building a larger pool of product concepts (cards), asking more Screener questions, and by showing fewer concepts at a time in each Screener question. How many concepts should be brought into the Choice Tournament? The choice tournament shows either two (pairs) or three product concepts at a time (triples), and not-selected concepts are eliminated until an overall winner remains. It takes t/2 such choice tasks to identify an overall winner if showing triples, and t-1 tasks if showing pairs. Thus, if a respondent takes all 40 product concepts into the choice tournament (all 40 are marked "a possibility" in the Screening section), then 20 Choice Tournament tasks will be asked if shown in triples, or 39 if shown in pairs. We generally recommend that you limit the number of concepts taken into the choice tournament to a little over half of your total product concepts (cards) and that you show the tasks in triples. Thus, if you plan to show respondents 32 total product concepts, we'd recommend bringing up to about 20 products into the tournament (leading to a maximum of 20/2 = 10 choice tasks in that section). Of course, if respondents mark fewer product concepts as "possibilities" in the Screener section, then fewer choice tasks will be needed to identify the overall winner.

Recommendations
Based on our experience with ACBC, we can offer some recommendations regarding questionnaire design settings. Most studies will probably bring all attributes and levels into the ACBC section, so the table below reflects the idea that each person has an identically-sized design. In all cases, we recommend that no more than about 7 total levels per attribute be brought into the ACBC part of the survey (other levels could be dropped as "undesirable" or "not available" using prior CiW questions and constructed list logic). We also recommend that no more than about 10 attributes be brought into the ACBC part of the survey (irrelevant attributes with virtually no importance could be dropped using prior CiW questions and constructed list logic). Depending on the number of attributes in the ACBC section of the survey, we recommend you consider the following Design settings (as approximate guidelines): Attributes brought into ACBC: (Not counting a "summed price" attribute) Questions: Number of Screening Tasks Number of Unacceptables Number of Must-Haves Maximum Number of Concepts in Choice Tournament Number of Calibration Concepts (optional) 6 3 2 12 *6 6 3 2 14 *6 7 4 3 16 *6 8 5 4 20 *6 8 5 4 24 *6 3 5 7 10 12

Screening Task Generation: Number of Concepts per Screening Task Minimum Attributes to Vary from BYO Selections Maximum Attributes to Vary from BYO Selections BYO-Product Modification Strategy 3 1 2 Mixed 4 1 2 Mixed 4 2 3 Mixed 4 2 4 Mixed 5 2 4 Mixed

404

SSI Web v7 *Only if estimation of a new "None" parameter using Purchase Likelihood intent scale is required. A "None" parameter will automatically result from the Screening Tasks. Note: if purely individual-level analysis is required (monotone regression) and/or sample sizes are quite small (n<50), we recommend increasing the Number of Screening Tasks by about 2 in each case (and also increasing the Maximum Number of Concepts in Choice Tournament by 4 in each case).

Adaptive CBC (ACBC) Help

405

3.1.7

Attributes and Levels for Adaptive CBC

For the most part, designing attributes and levels for ACBC follows the same guidelines as for other conjoint methods. We'll assume the reader already is acquainted with this practice (refer to the relevant sections in the ACA, CVA, and CBC documentation). But, there are some key differences related to the treatment of price and the possibility of using constructed lists to customize the attribute list each respondent sees. Most projects will probably present each respondent the full list of attributes and levels in the study. However, ACBC gives you the option to drop attributes and/or levels from consideration (you need to do this prior to the respondent starting the ACBC questions). This is facilitated by SSI Web's constructed list building logic. As you enter your attributes and levels for your ACBC study, you'll be specifying them within Predefined Lists. The attribute list is a Predefined List with as many elements as attributes in your study. Each element in the attribute list is associated with a Predefined List that enumerates the levels. Generally, we recommend that no more than about 12 attributes with no more than about seven levels each be taken forward to the ACBC questionnaire. If your study involves more attributes and levels than this, you may want to use preliminary questions within SSI Web (CiW) to eliminate any attributes that are completely unimportant and any levels that are completely irrelevant. Although it is possible to include a Price attribute as an attribute with discrete levels (such as $10, $15, $20), we generally recommend using ACBC's Summed Price approach and to treat price as a continuous variable. This is described in the next section of this documentation.

Sequence Order and Preference Order

In the "Unacceptables" and "Must Have" questions, we need to understand something about the nature of the levels (sequential order and/or preference order) to phrase our questions correctly and to make correct logical deductions regarding which levels to eliminate from future consideration. For example, assume we are conducting a study on automobiles and one of the attributes is fuel efficiency (in Miles Per Gallon) with levels: 20 MPG 25 MPG 30 MPG 35 MPG If after the first few screens of product concepts, the respondent only indicates that automobiles with at least 30 MPG are "possibilities," then the correct way to phrase the "Must Have" question would be something like this: The automobile must feature... "Fuel Efficiency: At least 30 MPG" For ACBC to know whether to use text such as "at least" or "at most" requires knowledge of whether the attribute has sequential (such as numeric) order, and which direction the order goes. With the fuel efficiency attribute, the sequential progression is from low to high values. For ACBC to know whether to eliminate more extreme levels from consideration if a less extreme level is marked "unacceptable" or "must have" requires understanding if the attribute involves a rational preference order. In this case (all else equal) there is rational preference order from worst to best. So, if the respondent confirms that "at least 30 MPG" is a cutoff rule, then we know that both 20 MPG and 25 MPG are unacceptable. When defining attributes, you must specify whether each attribute has: Sequential (e.g. numeric) order (low to high or high to low)

406

SSI Web v7 Preference order (worst to best or best to worst) The preference order is only used in the questionnaire for determining how to offer and enforce cutoff rules. It is not automatically applied as a utility constraint within estimation. You have the opportunity to enforce monotonicity (utility) constraints separately within the estimation modules. Here are some examples of attributes and their classifications: Non-ordered: Brand Coke Pepsi Sprite Sequential Order (high to low), Preference Order (Best to Worst): Printer Speed 40 pages per minute 25 pages per minute 15 pages per minute 10 pages per minute Sequential Order (low to high), No Preference Order: Square feet in home 1250 square feet 1750 square feet 2250 square feet 3000 square feet 4000 square feet 5000 square feet (Note that a person may not prefer a 5,000 square foot home to a 2,250 square foot home, because it involves extra work to clean, and is more expensive to furnish and maintain.)

Adaptive CBC (ACBC) Help

407

3.1.8

Price in Adaptive CBC

Price is often included in choice studies, and initial evidence suggests that ACBC should be a strong approach for studying the impact of price on choice.

Summed Prices
In traditional conjoint studies, the researcher applies price as another attribute (factor) in the study design, and specifies typically 3 to 5 levels of price (e.g. $100, $150, $200, $250). (One certainly could take this approach with ACBC.) The problem with such an approach is that $100 is sometimes shown with a collection of high-end features and $250 is sometimes shown with low-end features. With our standard CBC software, conditional pricing allowed the researcher to specify that certain (up to 3) attributes should be conditional on price, and always include a premium or discount. With ACBC, we have taken the idea of conditional pricing further, by allowing the researcher to specify incremental prices for up to all attributes in the study. When displaying the total product price, we sum the prices associated with the levels across all attributes in the product concept, and then we vary that summed price by a randomly drawn price variation (such as anywhere from -30% to +30%), as specified by the researcher. The researcher can also indicate that prices (after being disturbed randomly) should be rounded to, say, the nearest $100, or $5, $1, or $0.10 (for whatever currency symbol is appropriate). To specify prices per attribute, do the following: Add a level to your attributes list for the Price attribute From the Attributes tab, use the drop-down dialog to indicate that the attribute is a "<Summed Pricing Attribute>" Use the Pricing tab to specify any base price, plus prices per attribute levels in your study Using the "summed" pricing approach leads to product concepts that show realistic prices (and therefore reflect higher utility balance across product concepts relative to traditional CBC). Products with high-end features will generally carry higher prices, and products with low-end features will generally carry lower prices. Under summed pricing, thousands of potential unique prices will have been shown to respondents, and the utility function is estimated by fitting a linear (or non-linear function). Under summed pricing, we may estimate price as a continuous function (described further in the section entitled "Modeling the Price Function"). When treating price in this manner, we are able to partial-out the effect of price vis-a-vis the effects of other attributes' levels. Therefore, one can interpret the utilities for the other levels independent of the price increments that were associated with them (which one cannot do when using conditional price in our CBC software).

Choosing Price Increments

A challenge with using summed price is selecting appropriate price levels to associate with different attribute levels. For example, for the attribute RAM, incremental prices might be: 1 GB RAM +$0 2 GB RAM +$200 4 GB RAM +$500 This indicates that 2GB laptops should be shown at a price $200 higher (on average) than 1GB products. This imbedded price for RAM will be transparent to respondents within the Screener, Choice Tasks, and Calibration Sections, as only the total price will be shown for the product concept. However, within the BYO task, respondents will see that selecting 2GB RAM adds $200 to the base price of the product, and 4GB RAM adds $500 to the base price. In essence, in the BYO (configurator) task respondents are trading off the value of improving RAM vs. increasing the price. This provides

408

SSI Web v7 one tradeoff of RAM vs. price among the many choice tasks trading off product features (including RAM) with price across the total ACBC survey. Some attributes may not require associated levels of price, as the inclusion of one level or another really doesn't affect the price of the product in the real world. Examples include colors, styles, and often brands. Such attributes could be given incremental prices of zero, and choosing levels for these attributes within the BYO task would not change the overall price of the selected concept. Researchers will naturally be concerned that the price levels they choose to associate with levels in the study might have a significant effect on the outcome. If we could instruct a computer to answer the ACBC questions according to known cutoff rules and known utilities (with some level of respondent error), the choice of price levels (within reason) attached to attributes should not bias the estimates of utility for attribute levels and the price function. It would be ridiculous to use outlandish prices attached to RAM for a laptop such as it costing an extra $1,000 for each level of RAM. Such a mistake would certainly affect the final utilities. In one of our methodological (split-sample) experiments, we varied the price levels attached to some of the attributes in our study, to see if the resulting part-worth utilities and predictive ability (in terms of hit rates and share prediction accuracy) resulting from the two questionnaire versions would be affected. Our experiment showed that after estimating price as a separate linear function, the remaining partworths associated with the attributes that we manipulated were essentially the same. The hit rates for one variant of the experiment vs. the other were not significantly different. But, we need to see additional experiments on this point before we declare that it doesn't matter what prices (within reason) you associate with attribute levels when using summed pricing. So, our recommendation is to select reasonable incremental prices for price levels, but not to be overly concerned if your incremental prices deviate from average willingness to pay for the different attribute levels. If you find it very difficult to decide on incremental levels for some attributes, preliminary qualitative research could be conducted to help determine appropriate prices.

When Level-Based Prices Don't Make Sense

For some projects you face, the idea of assigning price increments to specific attribute levels and showing these to respondents just doesn't seem to work out. Perhaps it's impossible to think about price premiums by feature, or specific prices cannot be agreed upon with the client. If that is the case, you can specify a single base price for the product concept (along with the desired range of price variation to test, such as from -30% to +30%). Then, you could use the BYO question as if it was simply a series of select questions asking respondents which levels are preferred. However, this might seem unusual for ordered attributes, where one level is clearly superior. If that occurs, another option in the software is to omit the attribute from the BYO question. In that case, the software assumes no BYO level has been selected and will sample equally across all levels of that attribute in the generation of near-neighbor concepts (see the section entitled How Concepts Are Chosen for more details).

Adaptive CBC (ACBC) Help

409

3.2

Hands-On Tutorials
ACBC has flexible options for conducting a variety of studies. We assume the user already has significant experience using the SSI Web system, including CBC. In these tutorials, we give outline instructions and avoid click-by-click detail. Tutorial #1 (Beaches) Tutorial #2 (Pianos)

410

SSI Web v7

3.2.1

Tutorial #1 (Beaches)
Tutorial Specifications: Subject Matter: Choosing which beach to visit on vacation Design Summary: 8 attributes (no price attribute involved)

Note: We assume you are already familiar with SSI Web programming and standard CBC (ChoiceBased Conjoint) research. If not, we recommend you complete the Tutorials for CiW and standard CBC. Setting: You have been asked to work with Dr. Stephen Leatherman, "Dr. Beach," to determine which characteristics people most want in a beach. Dr. Leatherman is a professor at Florida International University that specializes in the study of beaches, and he is interested in refining his annual "Best Beaches" report. Part 1: Attribute List After discussions with Dr. Leatherman, you come up with a list of attributes that you both feel are the primary drivers of what makes a beach attractive. Attributes: Sand softness: Water temperature: Water safety: Beach cleanliness: Public Facilities: Bottom structure: Typical demographic: Crowds: Levels: Sand softness: Sand and rocks Coarse sand Medium sand Fine sand Water temperature: 60F/15C average (wetsuit required) 70F/21C average 80F/27C average 90F/32C average (bathwater) Water safety: No rip currents, few dangerous animals Occasional rip currents, occasional dangerous animals Frequent rip currents, occasional dangerous animals Beach cleanliness: Trash, glass and manmade rubble commonly found Trash, glass and manmade rubble occasionally found Trash, glass and manmade rubble rarely found Public Facilities: No public restrooms, showers or changing facilities

Adaptive CBC (ACBC) Help Public restrooms available, but no showers or changing facilities Public restrooms, showers & changing facilities available Bottom structure: Sandy bottom underwater past shoreline Rocky bottom underwater past shoreline Coral bottom underwater past shoreline Mud bottom underwater past shoreline Typical demographic: Popular with young partygoers Popular with young families Popular with mature adults Crowds: Uncrowded - ample open space Somewhat crowded - some open space Crowded - very little open space

411

Step 1: Create a new study

To begin programming this survey, open SSI Web and click File | New Study Name the survey "beaches".

Step 2: Add attributes and levels as Lists (List Manager)

Once you've created your new survey, we need to add the attributes and levels we've just defined. Use the List Manager (Compose | Lists) to specify 9 total (predefined) lists: 1. 2. 3. 4. 5. 6. 7. 8. 9. attributes sandsoft watertemp watersafety cleanliness facilities bottom demographic crowds

For list: attributes, add the 8 attribute labels (as List Members): Sand softness: Water temperature: Water safety: Beach cleanliness: Public Facilities: Bottom structure: Typical demographic: Crowds: For list: sandsoft, add the following List Members: Sand and rocks

412

SSI Web v7 Coarse sand Medium sand Fine sand Etc. Until you have fully specified all 9 lists.

Step 3: Attribute Settings

Open the Write Questionnaire dialog, then on the second page, add (using Add) an ACBC Exercise named "beach." The ACBC Exercise Settings dialog appears, with six tabs: Attributes, Pricing, Prohibitions, Design, Shared Question Settings, and Conditional Display. Select the Attributes tab. For the List of Attributes, select attributes. A dialog opens with one row per attribute (corresponding to each list element of attributes). For each attribute, use the drop-down to specify the associated list. For example, for attribute Sand softness: select sandsoft. When you are done, the dialog should look like this:

Next, we need to inform ACBC if each attribute has sequence order (low to high or high to low) and if so, the preference order (best to worst or worst to best). We also need to specify if each attribute should be included in the BYO section. Two of the attributes (Water safety and Beach cleanliness) would seem to be obvious questions for most every respondent (respondents would generally prefer safer, cleaner beaches). It wouldn't seem to make much sense to ask respondents about these two attributes in the BYO question.

Adaptive CBC (ACBC) Help Specify the following settings on the Attributes tab: Attribute Sand Softness Water temperature Water safety Beach cleanliness Public facilities Bottom structure Typical demographic Crowds Sequence Order Preference Order None Low to high Low to high High to low None None None None None None Best to worst Worst to best None None None None Include In BYO Yes Yes No No Yes Yes Yes Yes

413

Step 4: Establish Necessary Prohibited Pairs

We generally discourage the use of prohibitions in conjoint exercises. However, we also recognize that, occasionally, a few prohibitions might be necessary to avoid illogical combinations. And, very modest prohibitions will not affect design efficiency much. (Warning: too many prohibitions can invalidate your study, so be careful!) In our study, it might be unreasonable to believe that an uncrowded beach would be popular with partying college students. We'll therefore add a prohibition between "Popular with young partygoers" and "Uncrowded - ample open space". On the Prohibitions tab, Click Add Prohibition and specify a prohibition between Attribute 7, Level 1 and Attribute 8, Level 1. This looks like:

414

SSI Web v7 Back on the Prohibitions tab, the prohibitions grid should now look like:

(Of course, you could have just directly typed these values into this grid, rather than use the Add Prohibition button.)

Step 5: Design Settings

Go to the Design tab. This is where the bulk of your settings that control the flow of the questionnaire are found. The defaults that are shown are:

Let's reduce a few settings (to shorten the questionnaire, for the purposes of this tutorial): Screening Tasks: change from 8 to 6 Minimum Attributes to Vary from BYO Selections: change from 2 to 1 Maximum Attributes to Vary from BYO Selections: change from 4 to 3 Number of Unacceptables: change from 5 to 3 Number of Must Haves: change from 4 to 2 Maximum Number of Product Concepts in Choice Tournament: change from 20 to 16

Adaptive CBC (ACBC) Help When you've made those edits, the dialog should look like:

415

This will lead to 6 Screening Tasks x 4 Concepts per Screening Task = 24 total "cards/concepts" in the design. The respondents will get 3 Screening Tasks prior to seeing their first Unacceptables question. The number of choice tasks in the Choice Tournament will be limited to 16/2 = 8. Note that in the BYO question the respondent is only asked to specify their preferred beach on six of the eight attributes (we omitted Water Safety and Beach Cleanliness from the BYO question). The general rule is to vary no more than half of the BYO attributes from their BYO selections when generating the pool of near-neighbor concepts. Thus, we have changed the Min and Max attributes to vary to a more modest 1 and 3 attributes (instead of the default 2 and 4 attributes). For each new near-neighbor concept, between 1 to 3 of the six attributes included in the BYO question will be changed from their BYO selections. The other two attributes (Water Safety and Beach Cleanliness) will have levels chosen at random (controlled for balance) to complete each full concept.

Step 6: View Questionnaire Layout and Edit Questions

From the ACBC Exercise beach dialog, click OK to return to the Write Questionnaire dialog.

416

SSI Web v7 You should see the following question order for the ACBC section: Question Name beach_BYO beach_Screener1 beach_Screener2 beach_Screener3 beach_Unacceptable1 beach_Screener4 beach_Unacceptable2 beach_MustHave1 beach_Screener5 beach_Unacceptable3 beach_MustHave2 beach_Screener6 beach_ChoiceTask18 Question Type Build-Your-Own Screening Task Screening Task Screening Task Unacceptables Question Screening Task Unacceptables Question Must-Have Question Screening Task Unacceptables Question Must-Have Question Screening Task Choice Task Tournament

Edit BYO Question

Double-click the BYO question to edit it. First, modify the text in Header 1 to say: Please describe the beach you would most want to visit during summer vacation. Click your preferred choice for each feature below. Then, it would be helpful to inform respondents about the two attributes that we decided to drop from the BYO question (because they would have obvious answers). Now, let's click Preview to see what we've got to this point. Go to the Question Settings dialog to examine the various formatting options you have for this question. For example, you can change from a radio-button layout to a combo-box layout by selecting: BYO Layout = Combo Box. Try that, then click Preview to see the result. You can modify other settings if you would like to investigate further formatting options.

Insert HTML Filler Question (optional)

We'd recommend that you insert a new HTML Filler question directly following the BYO question (on a separate page) that informs respondents about the two attributes that we omitted from the BYO question. Place the following into the Header 1 of that HTML Filler question. For your information, in the next stages of this interview, we'll also be asking you about beaches that differ in the following ways:<p>

Adaptive CBC (ACBC) Help <b>Water Safety:</b><br> <ul> <li>No rip currents, few dangerous animals <li>Occasional rip currents, occasional dangerous animals <li>Frequent rip currents, occasional dangerous animals </ul><p> <b>Beach Cleanliness:</b><br> <ul> <li>Trash, glass and manmade rubble commonly found <li>Trash, glass and manmade rubble occasionally found <li>Trash, glass and manmade rubble rarely found </ul>

417

Edit Screener Tasks

Double-click any of the Screener tasks to edit the layout of these questions. Edit Header 1 to say: Here are a few beaches you might like. Do any of these look like possibilities? For each, indicate whether it is a possibility or not. Click Preview, to see what we've got with the defaults. You will note that the questionnaire might not fit well on your screen. It might be good to make the concepts a bit wider, so there isn't so much text wrapping across multiple lines. The Style that is selected for the study controls the layout and look of each question. You could try changing styles to see if a different style provides a better layout for ACBC questions. Or, you can directly override the style to control the width of the tasks and the font sizes. To override the font size used for the level text, click the ... button on the Format row of the grid. On that dialog, you can specify a font size (in pixels), which overrides the font size controlled by the study Style. Specify a "12" for text with a 12 pixel point size.

Edit Unacceptable Questions

Double-click any of the Unacceptable questions to edit the layout of these questions. The text header for this question varies depending on whether it is the first time this type of question has been seen by a respondent (a longer description) or whether the respondent has already seen this question type before (requiring a shorter description). Thus, we dynamically bring the text in using the function: [%ACBCUnacceptableText()%] You edit the text supplied in this question using the Question Settings tab. Edit the No Rules Exist Label to specify the text to display the first time the respondent encounters the question. Click the button to edit the text. Write: Would any beach having the features below be <b>totally unacceptable</b>? If so, mark the <b>one feature</b> that is most unacceptable to you, so I can focus better on

418

SSI Web v7 beaches that meet your needs. Edit the Rules Exist Label to specify the text to display on subsequent Unacceptable questions (after the respondent has become familiar with this type of question). Write: In addition to these unacceptable characteristics:<br><br> [% ACBCUnacceptableRules() %] <br><br>Is there another unacceptable feature? Next, edit the None of the Above Label to say: None of these is entirely unacceptable Let's insert a graphic of an interviewer to the left of this question (change Side Panel Text from Right to Left). Click the button for Side Panel Text, click the "camera" icon, and browse to a graphic (use "Kylie, business, small format" the graphic's name is "kylie_300_bus_4.jpg") Next, click Preview to see what we've got to this point.

Edit Must Have Questions

Double-click any of the Must Have questions to edit the layout of these questions. The text header for this question varies depending on whether it is the first time this type of question has been seen by a respondent (a longer description) or whether the respondent has already seen this question type before (requiring a shorter description). Thus, we dynamically bring the text in using the function: [%ACBCMustHaveText()%] You edit the text supplied in this question using the Question Settings tab. Edit the No Rules Exist Label to specify the text to display the first time the respondent encounters the question. Click the button to edit the text. Write: I don't want to jump to conclusions, but I've noticed you've chosen beaches with certain characteristics shown below. Are any of these <b>absolute musts</b> for the beach you would like to visit? If so, mark the <b>one most important feature</b>, so I can focus better on beaches that meet your needs. Edit the Rules Exist Label to specify the text to display on subsequent Must-Have questions (after the respondent has become familiar with this type of question). Write: In addition to:<br><br> [% ACBCMustHaveRules() %] <br><br>Are there any other absolute musts? Next, edit the None of the Above Label to say: None of these is an absolute must Let's insert a graphic of an interviewer at the right of this question. Change the Side Panel Position from Left to Right. Click the button for Side Panel Text, click the "camera" icon, and browse to a graphic (use "Kylie, business, small format" the graphic's name is "kylie_300_bus_3.jpg"). You can get these graphics from our website, at: http://www.sawtoothsoftware.com/download/ssiweb/graphics/ Next, click Preview to see what we've got to this point.

Adaptive CBC (ACBC) Help

419

Edit Choice Task Tournament Questions

Double-click any of the Choice Task Tournament questions to edit the layout of these questions. Edit Header 1 to say: Among these three, which beach would you most want to visit for summer vacation? (I've grayed out any identical features, so you can just focus on the differences.) You can also change Level Display Format so that the text is 12 point text always, irrespective of what Style is applied to the study.

Step 7: Test Questionnaire on Local Test Server, and view contents of bchdat.dat file
When testing the questionnaire, make sure to answer reasonably, using a realistic strategy that a respondent might follow. After you have completed at least one test survey, using the Admin Module, download the bchdat.dat file to the study folder (this is the main respondent data file). We recommend that you examine the contents of the file, to review some diagnostics about the pool of near-neighbor concepts that were created for your interview. Open the bchdat.dat file with a text editor. Search for the following text: "sys_ACBC_beach_levelCnt". After that label, you'll find an array of numbers. These record how many times each level in your study appeared in the questionnaire for this respondent. For example, for the "beach" study, the counts array may look like the following (after the first set of 24 cards are generated): sys_ACBC_beach_levelCnt1,[[4,2,3,15],[3,2,4,15],[8,8,8],[8,8,8],[4,3,17], [3,2,16,3],[4,16,4],[3,17,4]] The first four values within brackets [4,2,3,15] records how many times each level of the first attribute (sand softness) was included in the 24 product concepts. Level 4 (fine sand) was the BYOchosen level, and thus it occurs many more times (15) than the others. For this sample record, Unacceptable and Must Have rules were imposed, resulting in 7 replacement cards generated (for a final pool of 31 cards), and the final counts array is: sys_ACBC_beach_levelCnt3,[[4,4,3,20],[3,4,4,20],[12,11,8],[9,11,11],[4,5, 22],[4,4,19,4],[5,22,4],[4,22,5]] For strong individual-level estimates, we recommend that each level appear at least twice and preferably three times. If you plan to have relatively large sample size and are mainly interested in population estimates and predictions, then you should be able to use a relatively shorter questionnaire (where each level isn't necessarily shown at least 3 times). For this questionnaire, if the respondent doesn't establish any Unacceptable or Must Have rules, 24 cards are generated. Three of the levels in the design have only been represented 2 times (attribute 1 level2, attribute 2 level 2, and attribute 6 level 2). We could be satisfied with this questionnaire design if we had a relatively large sample size and accuracy of individual-level prediction was not a principal goal.

420

SSI Web v7

Step 8: Randomize the Attribute List

To control for order bias, let's randomize the order of presentation of attributes (randomized once per respondent, and the order remains constant through the interview). Create a new "Constructed" list called Rattributes (with "parent list" = attributes). Use the following instructions to build Rattributes: ADD (attributes) RANDOMIZE() Then, on the Attribute tab within the ACBC exercise, choose Rattributes as the attribute list instead of attributes. You could do a similar trick to randomize levels within any attribute you wanted as well.

Adaptive CBC (ACBC) Help

421

3.2.2

Tutorial #2 (Pianos)
Tutorial Specifications: Subject Matter: Grand pianos for home use Design Summary: 5 attributes Summed Pricing, with additional price adjustments Screening questions to develop list of considered brands (constructed list)

Note: This is an advanced ACBC tutorial. If you are beginning with ACBC, we suggest you first complete the "Beaches" Tutorial #1. Setting: This project involves pricing research for grand piano purchases for home use, in the 6-foot (parlor grand) to 7.5-foot (small concert grand) range. There are very large price differences between brands, with the Steinway brand in particular charging a very high premium due to the hand-crafted nature of its pianos. We would like you to consider that you would be interviewing only sophisticated buyers in this niche market (who are both in the market and are highly knowledgeable about this product category). Attribute List: Attributes: Brand: Length: Finish: Bench: Price: Levels: Brand: Baldwin Bechstein Bosendorfer Boston Chickering Estonia Kawai Kimball Knabe Petrof Samick Steinway Wurlitzer Yamaha Young-Chang Length: 6-foot 6.5-foot 7-foot 7.5-foot Finish: High-gloss black finish Flat black finish

422

SSI Web v7 Wood grain finish Bench: Standard bench Concert bench Standard + concert bench Price: "Summed Price" per specifications below Summed Pricing Grid Specifications: Level Prices: $7,000 $6,000 $10,000 $20,000 $12,000 $4,000 $5,000 $8,000 $4,000 $2,000 $6,000 $2,000 $30,000 $4,000 $8,000 $0 $0 $5,000 $10,000 $18,000 $0 $0 $0 $0 $0 $250 Steinway (+$15,000); Bosendorfer (+$10,000) Steinway (+$25,000); Bosendorfer (+$18,000) Steinway (+$40,000); Bosendorfer (+$25,000)

Level: Base Price Baldwin Bechstein Bosendorfer Boston Chickering Estonia Kawai Kimball Knabe Petrof Samick Steinway Wurlitzer Yamaha Young-Chang 6-foot 6.5-foot 7-foot 7.5-foot High-gloss black finish Flat black finish Wood grain finish Standard bench Concert bench Standard + concert bench

Additional Price Adjustments:

Adaptive CBC (ACBC) Help

423

The table above provides the data to compute the total price for each grand piano (prior to varying its price by a random variation in the range -30% to +30%). For example, a 7-foot Yamaha with a standard bench would cost: Base Price ($7,000) + Yamaha ($8,000) + 7-foot ($10,000) + Standard bench ($0) = $25,000 But, Steinway and Bosendorfer are elite brands whose prices escalate (beyond the other brands) for the larger instruments in the 6.5 to 7.5-foot range. The additional premiums are given as Price Adjustments. For example, the price of a 7-foot Steinway with a standard bench is: Base Price ($7,000) + Steinway ($30,000) + 7-foot ($10,000) + Adjustment for Steinway @ 7-foot (+$25,000) + Standard bench ($0) = $72,000 Dynamic Brand List (Constructed Lists): We recommend that not more than about seven levels of any attribute be brought into the ACBC exercise. Because this project involves 15 separate brands, we should drop some brands for each respondent and focus just on relevant brands. It is rare for an informed grand piano buyer to consider more than a handful of brands. Therefore, we will ask some preliminary screening questions to elicit the brands under consideration (and cap the maximum number of brands at seven). ACBC estimates a full set of utilities for the respondent, even though most brands are excluded from each respondent's ACBC exercise. For each respondent, we inform utility estimation that the notconsidered brands are inferior to the considered brands. Synthetic choice sets are added to the BYO portion of the design to indicate that the not-considered brands were rejected in favor of considered brands at their respective prices (if price premiums per brand are specified). It is very important to consider that the data for the BYO section is coded as if respondents are making choices for each attribute separately traded off versus any price premiums for that same attribute (from the "Summed Pricing" grid). Therefore, if preliminary screening questions are used to establish which brands are chosen for inclusion in the ACBC exercise, these screening questions should refer to any price premiums (or discounts) that are associated with the levels. For example, the screening question for brand could be represented as follows: Q1. Please consider your upcoming purchase of a grand piano (6-foot to 7.5-foot). For your reference, the typical price for a 6-foot piano for different brands is as follows:

424

SSI Web v7

Typical Price for 6-foot Piano:

Baldwin Bechstein Bosendorfer Boston Chickering Estonia Kawai Kimball Knabe Petrof Samick Steinway Wurlitzer Yamaha Young-Chang $13,000 $17,000 $27,000 $19,000 $11,000 $13,000 $15,000 $11,000 $9,000 $13,000 $9,000 $37,000 $11,000 $15,000 $7,000

If these were your only options, which of the following brands would you consider? <show list of brands, multi-select format> Baldwin Bechstein Bosendorfer Boston Chickering Estonia Kawai Kimball Knabe Petrof Samick Steinway Wurlitzer Yamaha Young-Chang

Programming the Project in SSI Web

As with the previous Beaches exercise (Tutorial #1), use the List Manager to establish pre-defined lists for the attributes and levels. (For the summed Price attribute, it should be included as a list element of the Attribute list.) The lists should be: List: Attributes Levels: Brand: Length:

Adaptive CBC (ACBC) Help Finish: Bench: Price: List: Brands Levels: Baldwin Bechstein Bosendorfer Boston Chickering Estonia Kawai Kimball Knabe Petrof Samick Steinway Wurlitzer Yamaha Young-Chang List: Length Levels: 6-foot 6.5-foot 7-foot 7.5-foot List: Finish Levels: High-gloss black finish Flat black finish Wood grain finish List: Bench Levels: Standard bench Concert bench Standard + concert bench

425

Add an ACBC exercise to your study, and use the Attributes tab to specify your list of attributes and the level lists associated with each attribute. When you are finished, it should look like:

426

SSI Web v7

(Note that for now, we are specifying the "Brand" list, which contains all 15 brands. Soon, you'll write screener questions and build a constructed list that includes up to 7 considered brands for each respondent. You'll select that constructed list as the List of Levels for brand rather than the pre-defined Brand list that is currently shown above. Next, specify your Summed Pricing Grid using the Pricing tab:

Adaptive CBC (ACBC) Help

427

Using the drop-down control, select each attribute and cut-and-paste or directly type the price values into the Component Prices column. Note that we inserted the base price of $7,000 into the Base Price field. We have specified to vary summed prices by +/-30%, rounded to the nearest $250. Next, you need to establish the Price Adjustments for Steinway and Bosendorfer. Click the Price Adjustments... button.

428

SSI Web v7

The grid above gives you great flexibility for inserting additional price adjustments. In our situation, the price adjustments are quite simple, because they are based on just two attributes (Attribute #1: Brand & Attribute #2: Length). For example (row 1 above), we add $15,000 to the cost of Attribute 2, Level 2 (Length: 6.5-foot) if Attribute 1, Level 12 (Steinway) occurs. The last major step to programming this exercise is to create a screener question (Q1) that asks respondents to provide a list of brands they consider. It could also be done in two steps: a question that asks for the one most likely brand, and then a follow-up question that asks for other considered brands. In either case, you develop a new constructed list of brands with no more than 7 brands on the list. On the Attributes tab, you'll select the constructed list of brands for Attribute 1 rather than the predefined brands list. (We show an example using constructed lists for a Brand attribute in the section entitled Customized (Constructed) Attribute and Level Lists.)

Adaptive CBC (ACBC) Help

429

3.3
3.3.1

ACBC Design Algorithms

How Concepts Are Chosen
ACBC creates an array of product concepts for the respondent to evaluate within the Screener and Choice Tasks sections of the questionnaire. These concepts are chosen as "near-neighbors" to the product concept the respondent chooses in the BYO task, but still include the full range of levels taken into each respondent's ACBC survey. Because the BYO-specified product concept differs across respondents, and the number of attributes and levels taken into each respondent's ACBC exercise can be dynamic (though in most projects the list will be static), it isn't possible to create an experimental design prior to fielding the study (as is done in non-adaptive CBC studies). Customized designs must be generated on-the-fly for each respondent. Because dozens or even hundreds of respondents might simultaneously be completing surveys over the web, we are concerned about the amount of computing resources demanded of the server to generate designs. Therefore, we've developed a relatively quick algorithm. The algorithm cannot be said to produce optimal designs, but its designs are nearorthogonal and have proven to work exceptionally well in our methodological studies to date comparing ACBC to standard CBC.

Inputs
The respondent provides the following input to the customized design: C0, a vector with as many elements as attributes included in this respondent's BYO question, describing which levels were included in the BYO concept. The analyst provides some inputs that control the design: T, the number of total product concepts to generate, Amin, the minimum number of attributes to vary from the BYO concept, Amax, the maximum number of attributes to vary from the BYO concept (restricted to be no more than 1/2 the number of attributes in the BYO question +1, not including Summed Price), If a "summed price" attribute is used, a range is provided specifying how much price should vary (randomly) from the summed components' price (e.g. 30% below to 20% above summed price).

The Design Algorithm

Near-orthogonal designs are generated using a controlled, randomized process. The steps involved in selecting each of T concepts in the design are as follows: 1. Randomly select an integer (Ai) from Amin to Amax that specifies how many attributes within C0 will be modified to create new (near-neighbor) concept Ci. 2. Randomly select Ai elements within C0 to modify. 3. Randomly select new (non-BYO selected) levels for the attributes chosen in step 2 (all other attributes remain at the BYO-selected levels). 4. Check to ensure that the concept chosen doesn't violate any prohibited pairs and is not a duplicate to another concept previously selected for this respondent. If prohibited or duplicate, discard the concept and return to step 1.

Issues Related to Level Balance and Efficiency

"Counts" arrays (at the individual level) are maintained to ensure that the designs have much greater balance than would be achieved if the above strategy involved purely random selections. A counts

430

SSI Web v7 array keeps track of how many times each element has been selected or modified. For example, we maintain a counts array for each attribute that records how many times each level within that attribute (other than the BYO-selected level) has been included in the design. When a relative deficit occurs in the target frequency of selection, we increase the likelihood that the element with the deficit will be selected in the next concept (the mechanism for doing this is described in the section Deviation from Target Frequencies). This allows for "controlled" randomized designs, and leads to a relatively high degree of level balance (but not perfect level balance). Our approach leads to a high degree of balance for: a) how many times Amin to Amax attributes were varied when generating T concepts, b) how many times each attribute was varied, and c) how many times each level (other than the BYO-specified level) was included across the T concepts. For attributes without logical preference order (such as color, style, and brand), we try to achieve level balance across the non-BYO chosen levels. For attributes featuring levels with numeric/logical order (such as speed, size, weight, etc.), we overweight by a factor of two the selection of levels directly adjacent to the BYO-chosen level. Thus, the counts array describing the frequency of level occurrence across T=36 concepts for four levels of speed might be as follows (assuming the respondent chose level three in the BYO concept): 3, 6, 21, 6. Levels 2 and 4 are oversampled by a factor of 2x relative to level 1. When a "summed price" attribute is in place, we first sum the total price for the concept (according to the level-based prices specified in the questionnaire setup). We then draw a random price variation (continuous random variable, rectangular distribution) within the range of price variation specified by the questionnaire author (for example a -30% to +30% price variation involves drawing a price multiplier from 0.7 to 1.3). Our randomization strategy ensures a high degree of balance across the price variations. We divide the range of price variation into quartiles, and randomly select (without replacement) which quartile will be used for the next random price draw (a random continuous value within that quartile). For example, the range for the price multiplier of 0.7 to 1.3 is broken into quartiles 0.7 to 0.85, 0.85 to 1.0, 1.0 to 1.15, and 1.15 to 1.3 (for tracking purposes). If we need to draw a price variation from the fourth quartile in creating concept Ci, we draw a random price multiplier anywhere within the range 1.15 to 1.3. After multiplying the summed price by that random price multiplier, the resulting price can then be rounded to the nearest x dollars (or monetary units of choice), where x is controlled by the survey author. As examples, one can round to the nearest 100 dollars, the nearest 1 dollar, or the nearest 0.1 dollars. One can also choose to add a constant after the rounding, such as 0.01, such that rounding to the nearest 0.1 dollars and subtracting 0.01 leads to prices ending in 9s. The actual price shown is recorded in the data file for use in analysis. Certain constraints (such as attribute prohibitions) can make it difficult or impossible to maintain a desired degree of balance. If the design algorithm is unable to find a new concept Ci with the counts balancing goals in place, it will relax the balancing criteria in order to generate a valid concept. In that case, a message will be written to a special ACBC log file name studyname_ACBC_log.cgi stored within the Admin folder for your information and review. From the standpoint of achieving maximum statistical efficiency, the optimal approach would vary all attributes independently, as in the traditional orthogonal array. By choosing only a narrow subset of the attributes to vary (from the BYO-specified concept) when creating each new concept, lower statistical efficiency results. However, ACBC's approach has three important benefits to counteract the loss in statistical efficiency: 1) Respondents can answer with less noise when fewer attributes are varying within each concept, 2) The concepts seem more relevant and plausible to the respondent, since they are near-neighbors to the BYO-specified product, and 3) The design concentrates on learning about preferences for the levels directly surrounding the respondent's highest levels of preference. In our previous methodological tests with about eight or nine total attributes, we have found that using 2 and 4 for Amin and Amax, respectively, works quite well. We tried varying from 2 to 5 attributes when generating each concept in one of our experiments, and didn't see an increase in performance. However, the optimal amount of variation to use when generating concepts depends on the number of attributes in the study and should be investigated further. Because the designs are adaptive, one cannot estimate the design efficiency prior to fielding the study.

Adaptive CBC (ACBC) Help

431

Norms established from previous studies will point to effective strategies. Simulated respondents may also be used, followed by aggregate logit estimation to investigate the size of standard errors (reflecting the precision of estimates) for pooled parameters.

When Attributes Are Dropped from the BYO Section

There are instances in which an attribute with a priori preference order seems out-of-place in a BYO question. If price premiums aren't assigned to such an attribute, it seems painfully obvious to ask respondents which level they would prefer in their BYO product. In such cases, you can drop attributes from the BYO question. The software will also let you drop attributes from the BYO question even if you don't establish a priori preference order (though this is an exception rather than typical practice, and you lose an opportunity to learn which level is preferred via the BYO question). When attributes with known a priori order are skipped in the BYO section, for software developer's convenience, the software will automatically fill the missing BYO answers as if the respondent had picked the best level. For attributes dropped in BYO without known a priori order, the software will automatically fill the missing BYO answers with randomly selected levels for those attributes. However, in either case, the design algorithm doesn't treat these automatically filled responses as legitimately chosen BYO levels for purposes of utility estimation or design generation. All levels of the skipped attributes are sampled roughly evenly in the near-neighbor design. "Dropped" attributes from the BYO section are not included in the vector C0, and are thus not counted as an attribute to be modified from its BYO level.

Strategies for Selecting Attributes to Vary (Attribute Balance vs. Mixed Approach)
The first three methodological tests we ran comparing ACBC to standard CBC used the strategy described above, where for the generation of each new concept we randomly selected a subset of attributes to vary from the BYO-selected levels such that each attribute had been varied from its BYOselected level about an equal number of times. However, it has occurred to us that such a strategy misses an opportunity to improve the amount of information we gain from each respondent in the typical cases in which we have asymmetric designs (where the number of levels per attribute varies). For studies with asymmetric designs, attributes with fewer levels have their levels probed more often than attributes with more levels. Consider an attribute with 6 levels versus another attribute with 2 levels. If the number of times we decide to vary the attributes from the BYO-selected levels is equal for these two attributes, it is clear that the number of times that the 1 non-BYO selected level from the 2-level attribute will be probed (included in the design) is greater than for the 5 non-BYO levels from the 6-level attribute. However, if our strategy is to reduce uncertainty over the entire attribute level space, it would follow that we should select the 6-level attribute more often for varying from its BYO-selected level than the 2-level attribute. This can be accomplished simply by changing the manner in which we increment the counter array that keeps track of how many times each attribute has been varied from its BYO-selected level. For the first three studies we ran, every time we altered an attribute from its BYO-selected level, we added a "1" to the counts array with respect to that attribute (where the counts array has as many elements as attributes in the study). The goal was to select subsets of attributes to modify from BYO-selected levels such that the counts array was balanced, for example, [18, 18, 18, 18, 18, 18] for a study involving six attributes, to indicate that each attribute had been varied 18 times from its BYO-specified level. If we want to select attributes to vary such that there is greater balance in the number of times each non-BYO level has been probed, we could add 1/k to the counts array each time we select an attribute for variation from its BYO-selected level, where k is equal to the number of levels in that attribute. This

432

SSI Web v7 would reduce the uncertainty across the entire attribute level space, but at the cost of adding a potential psychological bias. If one attribute is shown to vary from its BYO-selected level much more often than another across the product concepts, then undue attention may be called to that attribute. For that reason, we have implemented a mixed strategy that is a middling position between these two approaches (attribute-selection balance versus level-selection balance). It puts more attention toward testing non-BYO levels for attributes with more levels than for attributes with fewer levels; but it doesn't enforce strict level balance. We believe that the mixed approach should be better than the attribute balance approach, but look to future empirical research regarding this issue. Such research must involve human respondents rather than computer-generated respondents, since it involves a potential psychological bias.

Generating Replacement Cards

A key aim of ACBC questionnaires is to identify any levels that are "unacceptable" or "must haves." Rather than ask the respondent up front regarding cutoff rules, we formulate hypotheses regarding what levels might be unacceptable or must-haves based on the respondent's observed choices within the Screener section. For example, if we notice that the respondent only selects as "a possibility" concepts featuring Brand A, we might suspect Brand A is a "must-have" level. After we have observed a pattern of such choices, we present a list of "unacceptable" or "must have" rules that we suspect the respondent might be employing. If the respondent confirms a non-compensatory rule on the list, we then mark as "not a possibility" any concepts not yet evaluated that would fail to satisfy that rule. This leads to more efficient questionnaires and the opportunity to create "replacement cards" that probe deeper within the respondent's relevant preference space. For example, if after a respondent has evaluated the first 10 concepts (marking each as a "possibility" or "not a possibility"), the respondent verifies that Brand A is a "must-have" rule, this might eliminate from consideration 8 of the upcoming concepts (all featuring a brand other than Brand A) that we had planned to ask the respondent to evaluate. We generate 8 replacement concepts (all featuring Brand A) according to the design algorithm specified above, the exception being that the brand attribute will be removed from consideration as a potential selection in step 2 (above).

Concept Presentation in the First Few Screener Tasks

We rely heavily upon respondents' evaluations of the first few screens of concepts within the Screener section to identify cutoff rules. It is typical to show four or five product concepts per screen within the Screener section. It seems useful to us that the first two screens of concepts should reflect the full variety of levels included in each respondent's attribute list. Therefore, after generating T concepts as described above, we attempt to select sets of concepts to show in the first two screens that contain the full range of levels included in the study. We employ a simple algorithm that involves counting how many levels are represented on each screen, and selects sets from our T concepts that lead to a relatively high score on that count.

Concept Presentation in the Choice Tasks Tournament

The concepts the respondent indicates are "possibilities" in the Screener section are taken forward to the Choice Tasks section where they are compared in triples until an overall winner is identified. It takes t/2 triples to identify the overall winner, where t is the number of concepts to be evaluated (in the case that t is odd and t/2 is not an integer, one rounds down to determine the required number of triples). It is easier to compare the concepts if some of the attributes are tied (share the same level) across the triple. When this happens, we "gray-out" the row so that respondents can focus on the differences when choosing the best concept. We can reduce the cognitive difficulty of the task if we assemble the concepts so that a healthy number of rows are "grayed-out." To that end, we take modest steps to increase the likelihood of tied attributes, but stop well short of maximizing the "amount

Adaptive CBC (ACBC) Help

433

of gray." This sacrifices some design efficiency (in the traditional sense, assuming compensatory processing), but the tasks are more easily completed and the responses will contain less error.

434

SSI Web v7

3.3.2

Deviation from Target Frequencies

In the previous section, we described the method that ACBC uses to account for how many times each non-BYO level has been used in the design, and how many times each attribute has been varied from its BYO-specified level. We described that the algorithm increases the likelihood that a level or attribute will be chosen if a deficit in the "counts array" is noted. We also said that the process tends to lead to good, but not perfect balance and target frequencies. With ACBC, we are not approaching design generation in a traditional sense (we are using an adaptive procedure). Strict attribute/level balance is contrary to our aims and to the goal of reducing uncertainty in the utility estimates. There are a few reasons why the target level frequencies or number of times each attribute has been varied from its BYO-specified level may not perfectly match the goals of the algorithm: 1. If prohibitions are in place, it may be impossible to achieve target frequencies. 2. When respondents specify certain unacceptable or must-have rules, certain levels are forced in/out of the design for replacement cards, which by necessity leads to imbalance. 3. Even under the "Attribute Balance" design generation option, attributes with very few levels tend to be chosen less often (for modification from their BYO levels). That is because there are fewer levels to choose from to generate new, unique concepts. If duplicate concepts are generated, the algorithm chooses new attributes to try. This outcome is probably favorable, because better overall level balance occurs if we spend more effort investigating (relative to BYO choices) attributes with more levels than attributes with fewer levels. 4. When subsets of attributes are chosen to be modified from their BYO-specified levels and levels are chosen within the attribute that is being varied, we scan to see which one of those chosen attributes or levels has the highest representation in the current respondent's design. We then discard that choice of attribute or level in favor of the attribute or level that has the least representation in this respondent's design. This process nudges us in the direction of achieving targeted frequencies. But because only one attribute or level per draw may be discarded in favor of one under-represented attribute or level, it may not be able to "fill the holes" fast enough to achieve the desired attribute/level frequencies. But, if we were to allow the algorithm to catch up as quickly as needed in favor of perfectly matching target frequencies, patterns of attribute/level inclusion could emerge to the detriment of randomization and design efficiency. 5. If for some reason ACBC is unable to generate enough concepts following its design algorithm to satisfy the demands of the questionnaire, the attribute/level frequency targets are ignored, and new concepts are generated following a randomization strategy. A warning message is written to the error log (studyname_ACBC_log.cgi) in the /admin directory.

Adaptive CBC (ACBC) Help

435

3.4
3.4.1

Advanced ACBC Surveys

Customized (Constructed) Attribute and Level Lists
Warning: this is an advanced area. We caution you to proceed carefully if using constructed lists. ACBC includes the ability to customize the list of attributes and levels taken into the ACBC survey for each respondent (using SSI Web's List Building). We have had success using as many as 10 attributes with 33 total levels within ACBC studies. We don't know whether respondents can effectively manage even more attributes than this, and future research may answer that question. ACBC is designed to handle as many as 100 attributes, with as many 250 levels per attribute. This capacity is provided because computers can handle the computational and memory requirements, but not because we'd ever recommend doing such a large study. For most studies in practice, each respondent will evaluate the entire list of attributes and levels within the ACBC sections of the survey. However, because the list of attributes and levels within each attribute are organized as lists within SSI Web, it is possible to create constructed (customized) lists of attributes and levels for each respondent. It is possible to exclude entire attributes from consideration in ACBC (because we believe these attributes to be entirely irrelevant to respondents and carry no importance) or to exclude certain levels from attributes (such as excluding irrelevant brands that are unacceptable or not available). Any of these exclusions must be done prior to beginning the ACBC sections. We must stress that we don't know the best way to ask a respondent to indicate which attributes are irrelevant to him or her so that we could exclude them from the attribute list. This is left to your judgement (though we recommend that the range of levels associated with an attribute should be shown to respondents when asking them to mark irrelevant attributes). We also don't know the best way to ask regarding which levels (such as levels of brand) are unacceptable. However, if levels have associated level-based component prices in your design, you should display the level-based prices when eliciting which levels to take forward to the ACBC questionnaire. The utility estimation procedure assumes that levels were accepted/rejected considering their component-based prices. Please consider the following assumptions that we make when dealing with customized (constructed) lists of attributes and levels. 1. If an attribute is omitted from a respondent's customized attribute list, we assume it has zero importance (all levels will carry a part-worth of zero for that attribute). 2. If a level is omitted from a respondent's customized attribute list, you must indicate whether the omitted level should be assumed to be: a) Generally inferior to all included levels (but not absolutely unacceptable), b) Absolutely unacceptable, c) Missing randomly, and preferences should be imputed from the population distribution. Notes: ACBC can handle the case of just one level being brought into a constructed list for an attribute. But, zero levels within a constructed list for an attribute will produce a fatal error in the interview; therefore, you'll want to program skip logic to ensure that respondents either reconsider their answers (leading to at least one level in the level list) or that they skip over the ACBC questionnaire. Be careful if dropping selected levels for a sequentially ordered attribute with a preference direction. If you do this, you must bring a contiguous set of attribute levels into a respondent's design, and you should impose utility constraints upon this attribute when estimating part-worths.

436

SSI Web v7

Building Constructed Lists Using Unverified Perl

If you are conditionally dropping attributes or levels from consideration, you will sometimes need to use Unverified Perl when building the constructed lists (a constructed attribute list, or a constructed level list for an attribute). For example, consider the case in which you are studying too many brands of grand pianos to include in each respondent's study design (recall that we recommend bringing no more than about 7 levels for each attribute into each respondent's ACBC study design). In this example, further assume that we are using summed pricing, and these 15 brands have associated level-based relative prices: Baldwin Bechstein Bosendorfer Boston Chickering Estonia Kawai Kimball Knabe Petrof Samick Steinway Wurlitzer Yamaha Young-Chang +$6,000 +$10,000 +$18,000 +$12,000 +$4,000 +$5,000 +$8,000 +$4,000 +$2,000 +$6,000 +$2,000 +$25,000 +$4,000 +$8,000 +$0

The actual parent list of brand levels that will feed into the constructed list of brands associated with Attribute 1 in ACBC might be called PianoBrands: Predefined List: PianoBrands Baldwin Bechstein Bosendorfer Boston Chickering Estonia Kawai Kimball Knabe Petrof Samick Steinway Wurlitzer Yamaha Young-Chang If we use screener questions to ask respondents which brands they would consider and which they would never consider, we should show them with their associated relevant prices. We should do this because the way we encode the information for the BYO for utility estimation assumes that respondents considered every brand in the parent list at their relative prices, and they chose their considered brands and rejected the non-considered brands. So, we might ask a select-type screener question named Q1 prior to beginning the actual ACBC questions in which the following list is presented (with relevant reference prices):

Adaptive CBC (ACBC) Help Predefined List: Q1PianoBrands Baldwin (typical price $13,000 for 6-foot piano) Bechstein (typical price $17,000 for 6-foot piano) Bosendorfer (typical price $25,000 for 6-foot piano) Boston (typical price $19,000 for 6-foot piano) Chickering (typical price $11,000 for 6-foot piano) Estonia (typical price $12,000 for 6-foot piano) Kawai (typical price $15,000 for 6-foot piano) Kimball (typical price $11,000 for 6-foot piano) Knabe (typical price $9,000 for 6-foot piano) Petrof (typical price $13,000 for 6-foot piano) Samick (typical price $9,000 for 6-foot piano) Steinway (typical price $32,000 for 6-foot piano) Wurlitzer (typical price $11,000 for 6-foot piano) Yamaha (typical price $15,000 for 6-foot piano) Young-Chang (typical price $7,000 for 6-foot piano) None/No More of the Above

437

However, we'll want to use answers to Q1 (or multiple similar questions) to build a constructed list of the brand names to be used as the level list for Attribute 1 within the ACBC survey. In other words, if respondents pick: Kimball (typical price $11,000 for 6-foot piano) Samick (typical price $9,000 for 6-foot piano) from the list (Q1PianoBrands) we show in Q1, we want to add the following brand items from the original parent list (PianoBrands) to the constructed list to be used in the ACBC study for Attribute 1: Kimball Samick One can use a series of IF-Then statements in unverified perl to accomplish this, building a new list called PianoBrandsForACBC using PianoBrands as the parent list, such as:
begin unverified perl if(VALUE("Q1_1") == 1) { ADD("PianoBrands", 1); } if(VALUE("Q1_2") == 1) { ADD("PianoBrands", 2); } if(VALUE("Q1_3") == 1) { ADD("PianoBrands", 3); } end unverified

Etc., for as many items as need to be added to the list. But, we'll show you a more generic and elegant way to do this in unverified perl. For this example, we create two new constructed lists: ScreenedInPianoBrands PianoBrandsForACBC To build ScreenedInPianoBrands, you select Q1PianoBrands as the parent list, and you use the following list-building logic:
AIC (Q1, 1, 15)

438

SSI Web v7 Create a constructed list named PianoBrandsForACBC (select PianoBrands as the parent list). Insert the following constructed list logic (only two things need to be modified in this sample code, the list names that are underlined):
begin unverified perl #This code will map the results from one list to another one. Both lists need to be in the same order, starting with item 1. #Note: Both parent lists need to be the same length and have corresponding items that match 1 to 1. #The list that you want to "mirror." This is the list that contains the text for items respondents screened in. my $strList1Name = "ScreenedInPianoBrands"; #The parent list of the new list that you are creating. my $strList2Name = "PianoBrands"; #-----------Do not modify code below------------------my $List1 = eval(LISTVALUESARRAY($strList1Name)); my $intItem = 0; foreach $intItem (@{$List1}) { ADD($strList2Name, $intItem); } #----------------------------------------------------end unverified

Adaptive CBC (ACBC) Help

439

3.4.2

Price Adjustments (Within Summed Pricing)

Note: the use of price adjustments requires that respondents have JavaScript enabled on their browsers. You should check whether that is the case using the [%JavaScript()%] instruction with skip logic (to a message stating that respondents should turn their JavaScript on). ACBC provides the ability to assign price premiums or discounts to levels within your study. You do this via "Summed Pricing" by assigning Level Prices on the Pricing tab. However, there may be instances where you want to implement price adjustments to go beyond the simple additive prices offered through Summed Pricing. Consider an example with brands and styles, as might be set up with standard Summed Pricing: Levels: Brand A Brand B Brand C Size 1: Size 2: Size 3: Level Prices: $10 $15 $12 $0 $5 $8

Brand B is a premium brand, and it is possible that in reality the pricing for sizes is quite different for Brand B. Perhaps for Brand B, Size 1 adds $0; Size 2 adds $10, and Size 3 adds $20. To implement this exception for Brand B, we use Price Adjustments. Price adjustments are seen explicitly by respondents within the BYO task (where respondents see the prices per level), but are not explicitly evident to the respondent in the subsequent tasks, where only the total price is shown. Because ACBC estimates continuous functions for Summed Price (and due to the random variation, typically in the range of about +/-30%, given to the summed price amount), Price Adjustments should have little to no practical effect on the final partworth utilities and slope of the price function. But, price adjustments should improve the realism of the design, in terms of displaying concepts that have more realistic prices. When you click Price Adjustments... (from the Pricing tab), a dialog is displayed that contains a grid. Using the grid, price adjustments can be based on a single attribute, or the combination of multiple attributes. In the example below, we do a simple price adjustment just based on the occurrence of attribute 1 level 2 (Brand B).

440

SSI Web v7

In the example, above, if attribute 1 level 2 appears (Brand B) extra amounts are added to the prices for attribute 2. So, if Brand B appears, an additional $5 is added to "Size 2" and an extra $12 is added to "Size 3." Those prices are added above and beyond the generic level-based prices specified on the Pricing tab. One can make price adjustments based on the combination of multiple levels from different attributes, by using the cells further right within the grid.

Adaptive CBC (ACBC) Help

441

3.4.3

Conditional Graphics
Note: conditional graphics requires that respondents have JavaScript enabled on their browsers. You should check whether that is the case using the [%JavaScript()%] instruction with skip logic (to a message stating that respondents should turn their JavaScript on). Conditional display/graphics lets you display a graphic (or text, or video clip) on the screen, based on a level (or levels) included in a product concept. For example, a graphic is shown based on the first four attributes in this study on polyethylene utility tabes (size, leg style, color, and fold-in-half feature):

Conditional display can be shown above or below the concept. Or, conditional display elements can even replace text within concept. One of the challenges of creating conditional graphics based on the simultaneous combination of multiple attributes is that the total number of unique graphics to develop can be very large. In the example above with polyethylene utility tables, the conditional graphic is based on four attributes, each with just 3, 2, 2, and 2 levels, for a total of 3x2x2x2=24 total graphics. This is fairly manageable. But, with a larger number of levels and/or attributes, the total number of graphics could quickly get out of hand.

Adding a Conditional Relationship

To specify conditional display elements, go to the Exercise Settings tab within your ACBC exercise. Select the Conditional Display tab. Add a conditional display element by clicking Add... Then, specify which attributes participate in the relationship (up to as many attributes as are in your study).

442

SSI Web v7 A Conditional Display Table is opened for that relationship, with one row per possible attribute combination. You can directly type text into the fields, or you can cut-and-paste from programs like Word or Excel. To add graphics to the table, use the Pencil icon graphic using the Camera icon . and then browse for the

Attribute Visibility... Although each attribute needs to be displayed in the BYO section, some attributes may seem entirely redundant if they are already included in the graphical representation of the product concept. Therefore, you may wish to make certain attributes invisible after the BYO question has been completed. This button accesses a grid where you can select any attributes to remove from subsequent sections.

Display Controls
For each conditional display relationship, you can specify whether the element to display should occur in BYO and/or in questions after BYO. BYO Display in BYO: Check this to display the conditional element (text or graphics) within the BYO question. Position: Choose whether the conditional text/graphic should appear above, below, to the left, or to the right of the product specification grid. You can also choose to customize the position (see below), which allows you to specify a fixed position on the screen where the graphic is displayed (so it appears to "hover" over the area even though the respondent may scroll up and down within the window). Placeholder Width and Height: Before the levels have been picked in BYO, SSI Web doesn't know what conditional element (such as a graphic) to display. But, SSI Web can reserve an area for the graphic to display so that other text on the screen doesn't suddenly reposition on the screen when the conditional display text/graphic is available to be shown. Specify in pixels the width and height of the placeholder to reserve. Default Text/Graphic: Here, you can specify the text or graphic to display by default, prior to SSI Web learning enough information from the BYO selections to show a conditional element. Questions After BYO Display in Questions After BYO: Check this to display the conditional element (text or graphics) in questions after BYO. Position: Choose whether the conditional text/graphic should appear above or below the product concept, or whether it should replace the text of an attribute within the concept. Attribute Label (Optional): If you are showing attribute labels in the left of the grid, you can specify a label to appear to the left of the conditional text/graphics.

Custom Graphic Display in BYO

If you select Position: Custom for the position of the conditional text/graphic in BYO, then you need to do a little extra work to tell SSI Web how to position the conditional text/graphic within the BYO question.

Adaptive CBC (ACBC) Help An SSI Script function is provided to help with this:
[% BYOCondText(ExerciseName, CondText#) %] [% BYOCondText(ExerciseName, CondText#, XPos, YPos) %]

443

If you place this SSI Script function within the Header 1, Header 2, Footer, Side Panel, Top Panel, or Bottom Panel sections, the conditional display element (such as a graphic) is shown in that position. If you use the additional XPosition and YPosition elements, then the graphic is shown at a fixed coordinate on the screen (and the graphic seems to hover over that coordinate if the respondent scrolls).

444

SSI Web v7

3.4.4

Unverified Perl to Customize ACBC Questions

Sometimes, ACBC users would like to customize their interviews beyond what the standard interface offers. Examples include: Showing a different version of the attribute level text in the BYO question than in the Screeners and Choice Tasks sections. For example, for binary on/off features, displaying text to indicate "Not Included" in the BYO section, but leaving that level "blank" for the remaining questions. Or, showing a slightly longer version of the attribute level text in the BYO question than in subsequent sections. Changing the Header 1 text for subsequent Screener questions, after the first Screener has been seen.

Example 1: Changing the Attribute Level text for the BYO question Put the following into an element of the attribute list for Attribute 3 Level 1 (where "widget" represents the name of your ACBC exercise): [% begin unverified perl if(QUESTIONNAME() eq "widgit_BYO") { return "Here's the level text to show in the BYO question for Attribute 3 Level 1"; } else { return "Text for Attribute 3 Level 1"; } end unverified %]

Example 2: Changing the Header 1 text for subsequent Screener questions, after the first Screener has been seen Place the following in the Header 1 section of the Screener question (where "widget" represents the name of your ACBC exercise): [% begin unverified perl if(QUESTIONNAME() eq "widgit_Screener1") { return "Here are a few widgets you might like."; } else { return "Here are a few more widgets you might like."; } end unverified %]

Adaptive CBC (ACBC) Help

445

3.5
3.5.1

Checklist Prior to Fielding

Checklist Prior to Fielding
1. Is this an appropriate study for ACBC? Appropriate studies typically involve about five or more attributes. Studies with very few attributes (such as Brand-Package-Price studies) are probably more appropriately studied using standard CBC. 2. Have you taken the survey yourself multiple times, ensuring that the software can generate the requested number of product concepts to be screened without producing a warning in the "error log" in the /admin folder (indicating that it had problems generating the requested design)? 3. Have you examined the level representation frequency for individual respondent test records you generate (open the dat.dat file to view the counts array recorded there per respondent)? Does each non-BYO level (taken into the respondent's ACBC questionnaire) occur at a minimum of 2 times (3 times, preferably)? 4. Does each respondent evaluate no more than about 7 levels per attribute? If more than about 7 levels per attribute are being studied, then a constructed list should be used and respondents should evaluate the most preferred/relevant levels. 5. If you are studying a large number of attributes or levels per attribute (such that using constructed lists to discard levels and attributes from consideration within the ACBC survey is necessary), is your sample size sufficient to stabilize the parameters across the full list of attributes and levels? Are you willing to assume that discarded attributes are entirely unimportant to the respondent? 6. When you take the questionnaire, do the Unacceptables and Must-Have questions properly identify (using correct "at least" or "at most" phrasing) the levels you have consistently selected or rejected? (This confirms you set the correct worst to best or best to worst attribute direction). 7. Have you analyzed the data from your practice questionnaires, making sure that the computed partworths reflect the preferences you expressed in the questionnaire? 8. Have you asked colleagues to take the survey? Have you debriefed them regarding their experiences and analyzed their data? Were any sections/instructions confusing? Do the computed part-worth utilities reasonably reflect their individual preferences? 9. Have you fielded the study (a "soft launch") among a few target respondents, examining the same issues as directly above?

446

SSI Web v7

3.6
3.6.1

Analysis of ACBC Data

Counting Analysis for ACBC
Counting different outcomes in ACBC data can provide useful insights. ACBC software lets you count: BYO: How often levels were included Unacceptables: How often levels were unacceptable Must-Haves: How often levels were must-haves Screeners: How many products were screened into the consideration set Choice Tournament: How often levels were included in the "winning" concept

To perform counting analysis, click Analysis + ACBC Counting Analysis... A summary tabulation is shown on the screen. This counts report can be interesting information to supplement the findings of the more traditional utility analysis and market simulations. Also, a .csv file (with labels) named ACBCEXERCISENAME_Counts.csv is written to your study folder that contains the counting information listed above for each respondent. Within that file: For the BYO and Tournament Winner, a column is written to the .csv file for every level in the study. A "1" indicates it was included in the concept; a "0" means it is not included. For the number of Screener questions marked "a possibility" a single value is written to the file. For the Must-Haves information, a column is written to the .csv file for every level in the study. A "1" indicates it is a "Must-Have" level; a "0" means it is not a "Must-Have" level. For the Unacceptables information, a column is written to the .csv file for every level in the study. A "1" indicates it is an "Unacceptable" level; a "0" means it is not an "Unacceptable" level.

How to Treat Missing Levels or Attributes from Constructed Lists

If you used constructed (dynamic) lists to customize the list of attributes and levels brought into the ACBC questions for each respondent, you'll need to specify how to treat missing levels. We'll refer to as levels/attributes as being partially missing or entirely missing attribute. Partially Missing levels result when, say, only a subset of brands are brought forward to the ACBC questions for a respondent. An Entirely Missing Attribute results in all its levels not appearing in the ACBC questions for a respondent (for example, if that attribute was previously determined as entirely unimportant). Click the Settings... button to control how the missing levels should be treated in the report:

Adaptive CBC (ACBC) Help

447

We have established defaults to specify how to treat levels that are either partially missing or from an entirely missing attribute. When tallying the BYO product specifications, we assume you will want any partially-missing levels marked as 0 and any entirely missing attribute levels marked as missing. When tallying the Unacceptables and Must Haves, we assume you will want any partially-missing levels marked as unacceptable and any entirely missing attribute levels marked as missing (not known). When tallying the Tournament Winners, we assume you will want any partially-missing levels marked as 0 and any entirely missing attribute levels marked as missing. You can change these defaults if you wish.

How Unacceptables and Must-Haves Are Determined

Unacceptables: 1. VIA CONSTRUCTED LISTS: If the researcher has specified to treat missing attributes or levels as unacceptable, those are marked as unacceptable for the respondent. 2. INFERRED FROM MUST-HAVE QUESTION: For Attributes with Sequence Order set to "None", if the respondent indicates that one level is must-have, all other levels are marked unacceptable. 3. INFERRED FROM MUST-HAVE QUESTION: For Attributes with Sequence Order, the respondent indicates that a certain level is a "must-have" threshold (e.g. must have "at least" or "at most" this level). Any levels lower (if at least) or higher (if at most) than this level are marked unacceptable. For example, suppose MPG (miles per gallon) is the attribute in question with levels 20MPG, 25MPG, 30MPG, and 35MPG. If the respondent marks that "at least" 30MPG is a must-have, 20MPG and 25MPG are marked unacceptable. As a second example, if Seating Capacity is the attribute in question with levels 4-person, 5-person, 6-person, 7-person, and 8-person. If the respondent marks that "at most" 7-person seating is a

448

SSI Web v7 must-have, 8-person is marked unacceptable. 4. DIRECTLY FROM UNACCEPTABLE QUESTION: For Attributes with Preference Order set to "None", if a respondent indicates a level is unacceptable, it is marked as unacceptable. 5. DIRECTLY FROM UNACCEPTABLE QUESTION: For Attributes with Preference Order, if the respondent indicates that a certain level is unacceptable, that level and any levels less preferred than it are marked unacceptable. Exception in the case of constructed lists: If an attribute is marked as having preference order, if an "unacceptable" level (omitted due to constructed list) is "more preferred" than the first "acceptable" level, it is changed to "acceptable". Must Haves: 1. If an attribute has only one level not marked as unacceptable, it is a "must-have." 2. If an attribute is marked as having preference order, the first "most preferred" acceptable level is marked as "must-have."

Adaptive CBC (ACBC) Help

449

3.6.2

Coding the Information for Parameter Estimation

The information in the core three sections of the ACBC questionnaires can be coded as a sequence of choice tasks and may be estimated using maximum likelihood estimation under the MNL model. By default, non-price attributes are coded as part-worth functions using effects coding (a type of dummy coding procedure).

BYO and Most Likelies

The choice of level for each attribute may be coded as a choice task, where respondents chose 1 of K levels traded off with price. (In the case of BYO question where the price column was suppressed, price is not included in the tradeoff). Thus, if 10 total attributes (other than price) are included in the study, this section will contribute 10 tasks. If level(s) are omitted from a respondent's design (using constructed lists), the appropriate coding depends on whether the attribute involved has rational preference order. For omitted levels for attributes without rational preference order (such as brand, color, etc.), we append additional information to the design matrix in the form of binary tasks and an additional "threshold" parameter. The threshold "level" is "compared" to each of the included levels (at their component prices, if applicable) and "rejected"; then compared to all excluded levels (at their component prices, if applicable) and "chosen." For levels missing from a respondent's design for attributes that have rational order, we do not add any new information to the respondent's data. Rather, we suggest you apply monotonicity constraints for that attribute across the sample.

Screener Section
Each respondent has marked T concepts as "possibility" or "not a possibility." We treat these each as binary choices, where the respondent is assumed to compare the utility of the concept to the utility of a constant threshold prior to making a choice. The constant threshold is included in the design matrix as an additional dummy coded column.

Choice Tasks Tournament

These are coded as per standard CBC practice.

450

SSI Web v7

3.6.3

Modeling the Price Function

With ACBC, we have taken the approach of treating the price attribute as a "summed," continuous variable. With summed price, the survey author assigns component-based (level-based) prices (and optionally, a base price for the product). When we display the total price of the product (in the Screener, Choice Tasks, and Calibration sections), we add the prices for its levels (plus any optional base price), and then we vary the price randomly by a user-specified amount of random variation (e.g. 30% to +30%). This has the benefit of displaying overall prices that are more in line with the general quality of the products (for example, it avoids showing a poor product at relatively high prices). But, the drawback is that if price is not varied randomly (and independently) enough, strong enough correlations between price and product features can result and will lead to relatively imprecise estimates of both. In general, we recommend that overall price be varied by at least -30% to +30% to reduce such multicollinearity and ensure reasonably precise estimates. More guidance related to this can be found in the white paper: "Three Ways to Treat Overall Price in Conjoint Analysis" available in our Technical Papers library at www.sawtoothsoftware.com. Because there are myriad combinations of attribute levels that can make up a product (and each level can have assigned component prices) and prices are varied randomly within a specified range of variation, thousands of unique "displayed" prices can be found across a sample of respondents. To model the effect of price on product choice, we have chosen three effective coding methods: Linear, Log-Linear, and Piecewise. Linear Function: The total price shown to respondents is coded as a single column in the design matrix. This assumes that the impact of price on utility is a linear effect. (However, note that when projecting respondents' utilities to choices in the simulation module, one typically exponentiates the total utility for products, so the final impact of price on choice probabilities is modestly non-linear.) The benefit of linear coding is that only one parameter is estimated, conserving degrees of freedom. But, if the actual impact of price on utility is non-linear, the linear assumption for this model is simplistic and suboptimal. (Advanced Note: We temporarily transform whatever price range you provide to the range -1 to +1 so that HB converges quickly. Then, we re-expand the parameters back to the original scale when reporting the final utilities.) Log-Linear Function: The total price shown to respondents is transformed by the natural log and coded as a single column in the design matrix. This assumes that the impact of price on utility is log-linear, or somewhat curved, as shown below:

Adaptive CBC (ACBC) Help

451

As with the Linear function, only one parameter is estimated, which conserves degrees of freedom. But, if the actual impact of price on utility doesn't conform well to the smooth log-linear curve, this model is suboptimal. Because of the method we use to estimate discrete utilities for price points for communication with the SMRT market simulator, you must specify a number of interior price points in addition to the endpoints for the curved function to be reflected within market simulations. Piecewise Function: This is a more advanced and flexible function--and the approach we recommend for most datasets. The user provides specific breakpoints, and a different slope (linear function) is estimated to characterize the relationship of price to utility between each breakpoint. For example, consider a conjoint study where the total price (after adding the price components and randomly varying the prices by, say, -30% to +30%) results in a range of prices actually shown to respondents from $725 to $5,250. The user supplies "breakpoints" to use in developing the piecewise price function. Let's assume the user hypothesizes that the price function has elbows at $1,500 and $3,000, but is linear between adjacent breakpoints and end points. In other words, the user might hypothesize a relationship between price and utility as is depicted in the chart below:

452

SSI Web v7

Rather than estimate a single slope (parameter) that characterizes a smooth relationship from $725 to $5,250, we are in this case fitting three slopes (1, 2 and 3). The particular function above suggests that this respondent (remember that these betas are computed at the individual level) reacts more strongly to price changes between $725 and $1,500, and then less strongly to prices between $1,500 and $3,000. Then, the reaction is stronger again for prices higher than $3,000. One could specify as many breakpoints as desired along the price continuum, but each additional breakpoint leads to another parameter to be estimated, and overfitting (including reversals) will eventually result. A key decision is where to place the breakpoints. It would be a mistake to place breakpoints such that very few products seen by respondents actually fell within a range bounded by two breakpoints. In the example above, if very few products were evaluated that fell in the range $750 to $1,500, the model would have very little information by which to obtain a stable estimate of 1. We recommend you export the BYO data (using Analysis + ACBC Counts Report) and view the distribution of prices for chosen products to help determine appropriate breakpoints. Or, to get a more complete picture of the prices for products included in the design, we've created a free program (Get_ACBC_Prices.EXE) that you can download that will report a frequency distribution of all prices included in the near-neighbor designs across respondents. The benefit of piecewise functions is that they will provide much better fit to the data than a linear function if the price function is truly non-linear. The drawbacks are that they lead to more parameters to estimate, and importantly that the analyst must determine the appropriate number and position of the breakpoints. We also restrict you from estimating interaction terms involving piecewise Price, as such designs are often deficient. Why is this so? Consider an interaction between brand and price, where some brands never occur in the high tier range. We'll be attempting to estimate betas for interaction effects where there is no information in the design matrix. If the piecewise function is used, we simply carry the utility forward to the market simulator just for the prices corresponding to the end points and breakpoints for each respondent. When simulating prices between adjacent breakpoints, the market simulator applies linear interpolation. Thus, using the piecewise model results in utilities associated with discrete price levels (determined by the analysis) for use in the market simulator--the same process as is used in Sawtooth Software's ACA, CBC, and CVA systems. However, the choice of specific price levels is done post hoc rather than prior to fielding the study.

Adaptive CBC (ACBC) Help

453

(Advanced Note: We temporarily transform whatever price range you provide to the range -x to +x, where x is the number of price parameters to estimate in the model so that HB converges more quickly. Then, we re-expand the parameters back to the original scale when reporting the final utilities.)

Recommendations:
When using piecewise functions for summed price, it may take significantly longer for HB to converge. You may wish to significantly increase the number of initial iterations (prior to assuming convergence). In general, we recommend the piecewise function, using between 1 to 3 breakpoints. One can constrain each of the betas in the piecewise function to be negative, if constrained estimation is desired. Even though we restrict you from estimating interactions involving piecewise price, note that the very nature of creating a piecewise coding should inherently capture some interaction effects with price. For example, if premium brands are associated with a relatively large level-based price, then the beta associated with the upper price range will to a great deal be estimated specific to the occurrence of premium brands.

454

SSI Web v7

3.6.4

HB Estimation
In a previous section entitled Coding the Information for Parameter Estimation we described the strategy for coding respondent answers into choices from sets of available options. We have employed HB effectively to estimate part-worth utilities for ACBC experiments. Details regarding our implementation of HB may be found in the CBC/HB product manual and in the CBC/HB Technical Paper.

Output Files
A summary report is displayed on the screen after estimation is completed, where the part-worth utilities are summarized as normalized zero-centered diffs. Importance scores are also displayed, along with the standard deviations of these measures. The diffs method rescales utilities so that the total sum of the utility differences between the worst and best levels of each attribute across attributes (main effects) is equal to the number of attributes times 100. It removes the often strong differences in the magnitude of "scale" that make it hard to compare raw utilities across respondents or between groups of respondents. The utility run report displayed to the screen has multiple tabs. The first tab includes the summary report as just described. The second tab contains the raw part-worth utilities per respondent (same information as saved to the .csv and .hbu files, described below). The third tab includes the normalized (zero-centered diffs) part-worth utilities, appropriate for subsequent segmentation/cluster analysis. Various output files are written to the study directory: Most Commonly Used Files: .csv Comma-separated values file containing the mean part-worth utilities (one row per respondent). May be opened directly with Excel, and contains a header row containing labels. These are raw utilities, appropriate for market simulations. If using the utilities for segmentation analysis (such as cluster analysis), one should use normalized utilities (e.g. zero-centered diffs) rather than raw utilities. .hbu Contains the mean part-worth utilities (same as written to the .csv file). This file is formatted as text-only, and may be imported into SMRT for market simulations (note: only the utility run from an ACBC study is imported into SMRT. ACBC's .CHO file is incompatible with SMRT and should not be imported into SMRT). See also .hbu file format. .sum Contains final estimates of the population means and covariances. Files Containing Diagnostics to Assess Convergence .log Contains a history of RLH, Avg. Variance, and Parameter RMS, which is useful information to assess convergence. .alp Contains a history of the estimated population mean for part-worths. One row is written per draw saved (file is only written if Save Draws is enabled).

Adaptive CBC (ACBC) Help

455

.gamma If using Otter's Method for estimating task-specific scale factors, this file contains the estimated scale factors for the three sections in ACBC used for part-worth estimation (BYO, Screeners, Choice Tasks). One row is written per draw saved (file is only written if Save Draws is enabled). Files Containing Advanced Information .cov Contains the estimated variance-covariance matrix for the distribution of part-worths across respondents. Only the elements on-or-above the diagonal are saved. One row is written per draw saved (file is only written if Save Draws is enabled). .dra Contains estimated part-worths for each individual saved from each nth iteration, where n is the Skip Factor for Using Random Draws. The file is formatted like the .hbu file, without the header. .std Contains the within-respondent standard deviations among random draws. Only created if you elect not to save random draws.

Otter's Method
One consideration when estimating part-worth utilities from ACBC is the large difference in scale (response error) for the three main sections of the questionnaire. If using generic HB estimation, a key assumption is that the three types of tasks can be combined within the same dataset, even though we have observed that each section has different scale factor. Even though the BYO section has larger scale than the other two sections, we ignore this fact when using generic HB to estimate the parameters across the tasks. We have wondered about the practical effect of the differences in scale for the three sections when using a generic HB model. We have been fortunate to benefit from the expertise of Dr. Thomas Otter, a recognized expert regarding HB application to choice experiments. Otter has built more sophisticated HB models that separately model scale for the three sections in ACBC questionnaires. He has found that the generic HB model performs very nearly as well as the more sophisticated models. You can turn on "Otter's Method" for accounting for differences in scale during HB estimation through the HB interface provided in ACBC. When Otter's method is selected, an additional diagnostic file is written to disk when Save Random Draws is enabled that reports the relative gammas (scale factors). By default, the scale factor for the Screener section is constrained via very strong priors to remain around 1.0. Gammas for BYO, Screeners, and the Choice section are written to the .gamma file (in that order). A gamma of less than one indicates that this section of the survey has smaller scale than the Screener section. The final utility estimates are placed on the scale commensurate with the Screener section. Note: Otter's Method requires substantial sample size to stabilize the estimates of scale for the different sections of the survey. We recommend it be used when you have about 300 respondents or more. Also, we recommend a greater number of used iterations than default, as the parameters seem to "wander" in larger amplitude cycles (than standard HB) due to the additional estimation of the scale parameters. For more information, please see: Otter, Thomas (2007), "HB Analysis for Multi-Format Adaptive CBC," Sawtooth Software Conference

456

SSI Web v7 Proceedings, Sequim, WA.

Adaptive CBC (ACBC) Help

457

3.6.5

Interaction Effects
By default, interaction effects are not estimated in ACBC's HB routines. But, you can include selected first-order interactions (interactions between two attributes) if you wish. Most conjoint models assume a simple additive model. This implies that the effect (utility) of each attribute can be estimated and considered independent of all other attributes. Indeed, main effects tend to account for the majority of the explainable variance in the data. However, there are instances in which significant interaction effects exist and market simulators can be improved if those effects are included in the model. Including interaction effects in CBC models is especially critical if pooled (aggregate) analysis is employed. However, it has been demonstrated that the need to model interaction effects is significantly reduced when using individual-level estimation (as is the norm for ACBC). But, that doesn't mean that one can simply ignore interaction effects if using HB. We recommend you be on the lookout for interaction effects and that you investigate including them in ACBC models. ACBC's designs appear to be as effective as standard CBC designs with respect to accommodating interaction effects. Standard CBC gives two useful tools for identifying interaction effects that may provide improved fit to the model. CBC's Counts program reports a interaction Chi-Square test for all attributes taken two-ata-time. This can help you identify which effects (at least when considered in a pooled model) may be significant and useful. In the CBC documentation, we have encouraged users also to use aggregate logit and the "2 Log-Likelihood" test to test the strength and significance of first-order interactions. With Adaptive CBC designs, standard counting analysis is not appropriate (since the designs are not level balanced). But, one can perform a series of pooled logit models (using our Latent Class software with a one-group solution) to investigate potential interaction effects using the "2 Log-Likelihood" test. It is interesting to note, however, that a strong lift in likelihood due to the inclusion of an interaction effect for a pooled model may not necessarily translate into a strong lift for an individual-level model. Many of the interactions detected with aggregate modeling in CBC over the years have been due to respondent heterogeneity. For example, if the same people who prefer convertibles also prefer the color red for automobiles, then the "convertible + red" interaction effect may be mostly accounted for simply by estimating individual-level models (with main effects). But, interaction effects may more pervasive within each individual's psyche, and in those cases interaction effects can improve individuallevel model performance. A recommended practice is to investigate all potential first-order interaction effects using aggregate logit and the "2 Log-Likelihood" test. Those interaction effects that are strongest and most critical to the aims of the study (such as brand and price, for a brand equity/pricing study) could also be investigated within HB modeling. Useful interactions can be confirmed by examining the model fit to holdouts, or by seeing if there appears to be a positive lift in RLH as reported by HB. As noted earlier, it is quite possible that including what appears to be a strong interaction effect (from an aggregate modeling perspective) within individual-level HB models may offer no improved fit. This would suggest that the interaction is mostly due to heterogeneity rather than a more pervasive (intrarespondent) interaction effect. Unless there is compelling evidence for including an interaction in individual-level ACBC models, we recommend omitting it to avoid overfitting.

458

SSI Web v7

3.6.6

Utility Constraints, Including Customized Constraints

The HB and Monotone Regression estimation routines included in ACBC software allow you to impose monotonicity constraints (utility constraints). For example, if you know that the relationship between price and preference should be negative for all respondents, you can constrain all respondents' partworths to reflect a negative slope for price. Specify utility constraints by clicking the Use Constraints box from the Attribute Information tab (within the HB or Monotone estimation dialogs). Then, click the Edit... button that is enabled. A drop-down control appears that lets you specify constraints for attributes in your study. Constraints have the benefit of eliminating "reversals" in partworths (out-of-order relationships among levels with known rational preference order). Constrained utilities generally will show improved individual-level hit rates of holdout tasks. But, constraints introduce some bias in the parameters, and often result in models that have slightly lower aggregate predictive accuracy (such as for shares of preference). As another potential problem, if constraints are applied to some attributes, but not to others, this can change the sensitivity of the simulation models in favor of the constrained attributes. Researchers have generally found within-attribute constraints (such as ratings of levels within an attribute) to be more robust than between-attribute constraints (such as those based on stated attribute importance ratings). We only implement within-attribute constraints in ACBC. If the main goal of the research is individual-level predictive accuracy, we recommend applying constraints for all attributes: global constraints for attributes with known a priori order, and customized constraints for other attributes based on responses to additional questions in the survey. If the main goal of the research is share prediction accuracy for segments or for the population, we generally recommend avoiding utility constraints. However, if using ACBC with very small sample sizes (especially n<30), constraints across all attributes may be a good idea both for individual-level prediction and aggregate share prediction accuracy.

Global (Universal) Constraints Global constraints may be applied when one can safely assume that all respondents agree regarding the relative preference order of levels within an attribute. When imposing constraints in ACBC, a grid appears for each attribute, such as: Speed (Pages Per Minute): Level 5 PPM 7 PPM 10 PPM 12 PPM To impose global utility constraints, you can specify relative preference values in the right-hand column of the grid. Larger values indicate higher preference. For example: Relative Preference

Adaptive CBC (ACBC) Help Speed: Level 5 PPM 7 PPM 10 PPM 12 PPM Relative Preference 1 2 3 4

459

We could easily have used values of [10, 20, 30, 40] or [1, 3, 10, 1000] instead of [1, 2, 3, 4], and the results would be the same. The values found in the grid for Relative Preference are only referenced to constrain the relative preference order of the final utilities. The utility for 12 PPM will be constrained to be higher than the utility for 10 PPM (because a "4" is greater than a "3"), etc. When fields within the Relative Preference column are all either missing (blank) or equal, the attribute's utilities are not constrained.

Customized Constraints ACBC can apply customized (idiosyncratic) constraints in addition to global constraints. Attributes like speed and price may have rational preference order across all respondents (global). But, attributes such as brand, color, and style do not have the same preference order for every respondent. ACBC allows you to specify customized constraints for such attributes, using information collected in another part of the survey. Customized constraints only involve within-attribute utility constraints, not betweenattribute (importance) constraints, such as have been implemented in Sawtooth Software's ACA system. For example, you might ask respondents to rate different brands (in a question named RateBrand) on a scale from 1 to 5. The answers to this question can be used as customized ordinal constraints (you must ensure that higher values correspond with higher preference for any questions you reference as constraints). For example: Brand: Level Brand A Brand B Brand C Brand D Relative Preference RateBrand_1 RateBrand_2 RateBrand_3 RateBrand_4

The values stored within the RateBrand question are used to determine the applicable constraints for each respondent. For example, if a respondent rates the brands as follows: Brand Brand A Brand B Brand C Brand D Rating 5 3 3 2

Then, utilities for this respondent are constrained according to the following ordinal relationships:

460

SSI Web v7 Brand A > Brand B Brand A > Brand C Brand A > Brand D Brand B > Brand D Brand C > Brand D Notice that brands B and C are not constrained with respect to one another (neither are they tied); their utility values with respect to one another are left to be determined by the data. Note: If using summed pricing and associating price premiums with levels, any ratings applied as constraints should refer to the relative utility of the level independent of any level-based pricing. Respondents should be asked to rate the levels assuming all else equal, including price.

Adaptive CBC (ACBC) Help

461

3.6.7

Monotone Regression (Pure Individual Analysis)

Recommendations:
Monotone Regression can estimate partworth utilities uniquely for each respondent, without the benefit of borrowing information across a sample of respondents. With tiny samples of a few respondents, monotone is probably preferred to HB. In our experience, even with as few as 10 or 20 respondents, HB may be preferred. With hundreds of respondents, we've seen consistent evidence that HB works better than monotone regression. If using monotone regression, we recommend that your ACBC questionnaires be of substantial length, such that all non-BYO levels occur in each respondent's design at least 4x. If using monotone regression, we expect you are working with tiny sample sizes, and we encourage you to impose utility constraints on all attributes. You can specify global constraints for attributes with known logical order, especially price, and idiosyncratic constraints for attributes like brand, style, and color. As a word of caution, partworth utilities are given arbitrary scale under monotone regression, unlike HB where the scaling of the partworths directly maps to probabilities of choice via the logit rule. Thus, you should adjust the Exponent (Scale Factor) appropriately in market simulators that employ partworths developed with monotone regression. Otherwise, the overall sensitivity of the model may be significantly lower or higher than appropriate for predicting buyer choices.

Methodology:
Monotone regression uses a method similar to that described by Richard M. Johnson in "A Simple Method of Pairwise Monotone Regression", Psychometrika, 1975, pp 163-168. Monotone regression may be appropriate for estimation problems where the model is intended to fit the data only in a rank order sense. An example of such a situation would be full profile conjoint analysis, with data consisting only of rank orders of preference for a collection of product concepts. More recently, monotone regression has been adapted to choice modeling, where the data consist of only ones and zeros, with ones indicating chosen concepts and zeros indicating those not chosen. In this context, monotone regression seeks a set of partworths which imply higher utility for each chosen concept than for non-chosen concepts in the same choice set. The method is iterative, starting with random values and finding successive approximations which fit the data increasingly well. Two measures of goodness of fit are reported: Theta and Tau. Tau Suppose a choice task presented four concepts, P, Q, R, and S, and the respondent chose concept P. Also, suppose that at some intermediate stage in the computation, utilities for these concepts are estimated as follows: Concept Utility Chosen P 4.5 1 Q 5.6 0 R 1.2 0 S -2.3 0 We want to measure "how close" the utilities are to the rank orders of preference. One way we could measure would be to consider all of the possible pairs of concepts, and to ask for each pair whether the member with the more favorable rank also has the higher utility. But since our observed data consist only of a one and three zeros, we can focus on the differences in utility between the chosen concept and the three non-chosen concepts.

462

SSI Web v7 In this example the preferred concept (P) has a higher utility than either R or S, but it does not have a higher utility than Q. Kendall's Tau is a way of expressing the amount of agreement between the preferences and the estimated utilities. It is obtained by subtracting the number of "wrong" pairs (1) from the number of "right" pairs (2), and then dividing this difference by the total number of pairs. In this case, Tau = (2 - 1) /3 = .333 A Tau value of 1.000 would indicate perfect agreement in a rank order sense. A Tau of 0 would indicate complete lack of correspondence, and a Tau of -1.000 would indicate a perfect reverse relationship. (Although this example involves only one choice set, in a real application the number of correct and incorrect pairs would be summed over all choice sets in the questionnaire.) Tau is a convenient way to express the amount of agreement between rank orders of two sets of numbers, such as choices and utilities for concepts. However, it is not useful as a measure on which to base an optimization algorithm, because of its discontinuous nature. As a solution is modified to fit increasingly well, the Tau value will remain constant and then suddenly jump to a higher value. Some other measure is required which, while similar to Tau, is a continuous function of the utility values. Theta For this purpose Johnson described a statistic "Theta." In his original article, Johnson measured differences in utility by squaring each difference. For this example we have: Preference P minus Q P minus R P minus S Utility Squared Difference Difference -1.1 1.21 3.3 10.89 6.8 46.24 ---Total 58.34

Theta is obtained from the squared utility differences in the last column of the table above. We sum the squares of those utility differences that are in the "wrong order," divide by the total of all the squared utility differences, and then take the square root of the quotient. Since there is only one difference in the wrong direction, Theta = square root(1.21/ 58.34) = .144 Theta can be regarded as the percentage of information in the utility differences that is incorrect, given the data. The best possible value of Theta is zero, and the worst possible value is 1.000. Now that we have defined Theta, we can describe the nature of the computation. The process is iterative. It starts with random values as estimates of the partworths. In each iteration a direction of change (a gradient vector) is found, for which a small change in partworths will produce the greatest improvement in Theta. Small changes are made in that direction, which continue as long as Theta improves. Each iteration has these steps: 1. Obtain the value of Theta for the current estimates of partworths and a direction (gradient) in which the solution should be modified to decrease Theta most rapidly. 2. Try a small change of the partworths in the indicated direction, which is done by subtracting the gradient vector from the partworth vector and renormalizing the partworth estimates so as to have a sum of zero within each attribute and a total sum of squares equal to unity. Each successive estimate of partworths is constrained as indicated by the a priori settings or additional utility constraints. 3. Re-evaluate Theta. If Theta is smaller than before, the step was successful, so we accept the improved estimates and try to obtain further improvement using the same

Adaptive CBC (ACBC) Help

463

procedure again, by returning to step (2). If Theta is larger than before, we have gone too far, so we revert to the previous estimate of partworths and begin a new iteration by returning to step (1). If any iteration fails to improve Theta from the previous iteration, or if Theta becomes as small as 1e10, the algorithm terminates. In theory, the iterations could continue almost indefinitely with a long series of very small improvements in Theta. For this reason it is useful to place a limit on the number of iterations. A maximum of 50 iterations are permitted, and within any iteration a maximum of 50 attempts at improvement are permitted. To avoid the possibility of stumbling into a bad solution due to a poor starting point, the process is repeated 5 separate times from different starting points. For each respondent, the weighted average of the five resulting vectors of partworths is computed (weighted by Tau, where any negative Tau is set to an arbitrarily small positive number). A weighted Tau is also reported with this final estimate of partworths. Although Theta was originally defined as a ratio of sums of squared differences, more recently we have experimented with a modified measure which uses absolute values of differences. The current software permits the user to choose which of the two measures to use. We would really like to find a way to optimize Tau, which is a "pure" measure of fit to observations which consist solely of zeros and ones. However, because Tau does not vary continuously with small changes in partworths, it was necessary to find some other statistic which was "like" Tau but which varied continuously with small changes in the partworths, and that is the role played by Theta. The rationale for the quadratic definition of Theta is one of historical precedent. The principle of least squares has been used widely and for a long time in statistics. However, when using squared differences, the value of Theta is affected much more by larger differences than by smaller ones, so the algorithm may tolerate many small errors to reduce a few large errors. The rationale for the linear definition of Theta is that when based on absolute values of errors, Theta is more similar to Tau, in the sense that the algorithm will tolerate a few large errors in order to avoid many smaller errors. Our experience with the linear definition of Theta has been limited to a few recent data sets. However, the linear version seems to result not only in better values of Tau (as one would predict) but also in slightly better hit rates for holdout choice sets, and better (more consistently negative) estimates of partworth coefficients for price. We leave it to our users to experiment with the two versions of Theta, and we will be interested to learn whether the linear version continues to outperform the quadratic version.

Limitations No interaction effects: With purely individual-level analysis, there generally isn't enough information to estimate interaction effects. No piecewise regression: This restriction avoids the common occurrence of having little (or no) information to estimate betas for regions of the price demand curve that are less relevant to the respondent. For example, a respondent who chooses a BYO product in the very expensive range may see very few or no products in the very inexpensive region for which we are attempting to estimate a beta coefficient. Note: The restriction of using only linear/log-linear price functions might seem limiting. We can imagine advanced researchers building their own market simulators that include individual-level threshold price cutoffs. For example, one could use the linear price coefficient estimated via monotone regression, but provide a strong utility penalty should the price exceed the respondent's expressed "unacceptable"

464

SSI Web v7 price threshold (this information can be exported per respondent using the Counts function).

Adaptive CBC (ACBC) Help

465

3.6.8

Analysis with Tiny Sample Sizes

Generally, sample sizes in the multiple hundreds are used for conjoint-analysis surveys. But, there are instances in which the market consists of just a few customers, and the researcher wants to gain insights regarding these few people. We have heard of entire conjoint studies designed for the purposes of understanding the tradeoffs of a single customer. One of the advantages of ACBC is the ability to stabilize part-worth utility estimates with smaller sample sizes than typically are used with traditional CBC. With ACBC, more information is collected at the individual-level, and respondents seem willing to spend even greater time in the adaptive CBC survey than the non-adaptive CBC. Respondents have reported equal or greater engagement/interest/satisfaction after taking ACBC interviews than with much shorter CBC interviews. Our research suggests that purely individual-level analysis (via monotone regression) is feasible for ACBC, though when hundreds of respondents are available in the dataset, HB estimation consistently provides more accurate results. (Purely individual-level analysis means that no information is "borrowed" from other respondents in the sample, so results could be computed even for n=1.) Despite the general superiority of HB analysis over monotone regression with ACBC data, we've wondered at what sample size one might start to favor monotone regression over HB. Is a sample size of n=30 sufficient for HB, such that the population estimates of means and covariances can be leveraged to produce more accurate part-worth utilities than monotone regression? What about n=20? Certainly, answers to these questions hinge upon how many attributes and levels are in the study, and whether the questionnaire has been made relatively short or long (as the researcher has flexibility in defining how many total concepts are generated in the near-neighbor pool for the Screening section). Elsewhere in this documentation, we have suggested that you review the designs generated for a few test respondents to ensure that each (non-BYO selected) level taken forward to the ACBC questionnaire appears at least two or three times if using HB, and potentially 4 or more times if purely individual-level estimation is required due to very small sample sizes. We re-analyzed an existing dataset (the "laptop" dataset that is described in our published ACBC papers) to investigate the use of HB with even tiny sample sizes. The dataset contained 286 respondents. In addition to the ACBC interview, each respondent completed four holdout choice tasks that were formatted like CBC tasks. We can use those to compute hit rates (how well the part-worth utilities for each person can predict which product concept the respondent would choose in the holdout sets). Using purely individual-level analysis (monotone regression), we obtained a 52.8% hit rate. Pooling all respondents, we achieved 56.5% hit rate under HB. This confirms other findings, where HB has proven more successful than monotone regression when analyzing hundreds of respondents in ACBC. Default settings were used for both HB and monotone regression, with the exception that price was constrained negative. Next, we split the sample into random partitions, estimating HB separately within each subset of respondents. We re-computed the total hit rate (across all 286 respondents) under each condition. For example, 286 respondents can be divided equally into two groups of 143. But, dividing the sample into four groups results in four subsamples of 72, 72, 71, and 71 respondents. In the table below, when the number of respondents in the sub-sample didn't divide evenly into 286, we report the largest sample size represented (for example, 72, when sub-samples of 71 and 72 were used).

466

SSI Web v7 Sample Sizes of Random Partitions n=286 n=143 n=72 n=36 n=18 n=9 Hit Rate (HB) 56.5% 55.4% 55.0% 55.3% 53.9% 53.5%

As expected, the hit rate for HB estimation declines as less information is available to "borrow" from respondents within the same dataset to estimate part-worth utilities. Even with sample sizes as small as n=9, HB's hit rate is 53.5% compared to 52.8% for monotone regression. So, it appears that HB can be effective with ACBC data for even very tiny sample sizes as low as n=9. We should caution that the absolute scale of the parameters increases as smaller sample sizes are used with HB. The Parameter Root Mean Squared (typical size of the point estimates of the part-worth utilities) was 2.13 when all 286 respondents were included in a single HB run, but was 2.95 when the respondents were estimated in sub-samples of n=9. This leads us to reemphasize that one should tune the market simulator via the Exponent to achieve proper scaling of simulated shares, as the scaling resulting from an HB estimation of ACBC data is affected by sample size and often needs tuning down (below 1.0) to best predict market behavior.

Adaptive CBC (ACBC) Help

467

3.6.9

None Threshold & Calibration to Purchase Likelihood

One of the benefits of traditional CBC modeling is the ability to include a "None" option. However, interpretation of the strength of the None parameter as well as appropriate application within market simulations have been difficult issues in CBC research. Adaptive CBC can estimate a None threshold using two approaches: None parameter estimated from Screening Tasks None parameter inferred from purchase intent scales via Calibration Concepts

None Parameter Estimated From Screening Tasks

The Screening Tasks present product concepts to the respondent and ask if each is a possibility or not. These tasks are coded for utility estimation as binary choices of the product concept versus a threshold of acceptability. The parameter estimate for that threshold of acceptability may be used as a surrogate for the None parameter within choice modeling. However, recognize that this None parameter has some characteristics that depart from the traditional accepted practice of computing a None threshold for CBC projects: It is scaled based on binary choices (associated with labels such as "a possibility" and "not a possibility") rather than in the context of a purchase/choice among multiple available options in a marketplace. The None parameter estimated from ACBC's binary tasks will generally be stronger (higher) than from traditional CBC tasks. The product concepts presented in ACBC surveys are generally of higher utility and closer to the respondent's ideal than typical CBC questionnaires. When respondents become conditioned to having the opportunity to choose among generally more relevant and desirable concepts, they react more negatively to concepts of moderate utility that might have seemed acceptable to them within the context of a standard CBC questionnaire.

None Parameter Inferred from Calibration Concepts

Some users may want to calibrate the None parameter (threshold) using 5-point purchase likelihood questions. This was suggested by Jon Pinnell in a paper entitled, "Alternative Methods to Measure Changes in Market Size in Choice Based Conjoint" presented at the 2006 SKIM/Sawtooth Software European Conference Event. In justifying the 5-point purchase likelihood intent scale, Pinnell reports: "While the 2-point buy/no-buy is consistent with the historical use of a none alternative in choice modeling, we prefer a 5 point follow-up question in that it provides more information and is also consistent with most methods outside discrete choice modeling to gauge market size and product appeal. In addition, many marketers have developed internal normative databases reflecting the five point purchase intent scale and in-market performance and have used those to develop action standards based on the five point purchase intent scale." Our use of the rating scale is in the tradition of concept testing with a five-point Likert scale and departs from choice theory and multinomial logit assumptions. Given the problems in obtaining proper scale for the None parameter under sound choice theory, many will consider this departure to be reasonable and defensible--especially if the firm has prior data regarding how Likert scales map to eventual product acceptance. Unlike ACA, our calibration procedure does not rescale part-worth utilities by multiplying by a slope and adding an intercept. The only purpose of the calibration concept data within ACBC is to estimate a new threshold None utility value (replacing the None threshold computed from the Screening concepts). It is entirely optional to include these calibration concept questions in the ACBC survey (as one can simply retain the original None parameter).

468

SSI Web v7 To estimate a new None utility using the Calibration Concept data, click Analysis + Estimate ACBC None Threshold... Select an existing .hbu file as the Source File (this file contains your part-worth utilities). Specify a new file with .hbu extension as the Destination File (this file will be created using the original utilities plus the newly estimated None utility). A key assumption when estimating a new value for the None utility is the point on the Likert scale that should represent the buy/no buy threshold for respondents. Obviously, this could differ by respondent. However, we generalize that the buy/no buy threshold on the scale is constant across respondents and further assume that you as the forecaster can identify that appropriate scale point (easy, right?). The scale points on the Likert scale are typically labeled as follows: 5 4 3 2 1 Definitely Will Buy Probably Will Buy Might or Might not Buy Probably Will not Buy Definitely Will not Buy

If, for example, you indicate that the None threshold is the product utility corresponding to a 3.2 on the scale, we estimate that utility (for each respondent) by referring to the already-estimated part-worth utilities and the calibration concept data. The calibration ratings are regressed (at the individual level) on the final utilities that resulted from HB or Monotone Regression. We'd recommend that respondents be shown five or six calibration concepts, to get reasonably stable estimates from the regression. Using the respondent-specific estimated slope (Beta1) and intercept (C), we can compute the utility corresponding to a 3.2 on the Likert scale using the equation (3.2 - C) / Beta1. The estimation of the None threshold from the purchase likelihood scale is a post hoc procedure that does not involve re-estimating the part-worths. As it uses OLS, the computation is nearly instantaneous, and the analyst can quickly compute new None threshold parameters using different assumptions about the point on the rating scale that corresponds to the threshold point for buy/no buy. When you calibrate the None parameter, you create a new .hbu file. Only two fields are modified in that file: the None parameter value and the fit statistic. The calculation of the None parameter was described directly above. The fit statistic is the correlation resulting from the calibration regression step (the square root of the R-squared). When this None utility threshold is used within the market simulator, a new "product" with utility equal to the None threshold value is introduced in competition with the other product concepts within that market simulation. The share captured by the "None" product reflects the percent of the market projected not to buy a product within this scenario. The default setting for the point on the rating scale corresponding to the buy/no buy threshold is 3.0. We have no evidence that this is proper for your project needs, and recommend you decide for yourself the appropriate setting. Using a recent dataset shared with us, we found that using a 2.8 on the 5-point purchase likelihood scale led to a None parameter from the OLS calibration that on average was about the same magnitude as estimated via HB using the binary judgments of the Screener section (where the binary choices were labeled "A possibility" or "Won't work for me"). Thus, using the default 3.0 as the buy/no buy threshold resulted in a slightly stronger None parameter utility than was initially estimated using the Screener section information.

Exception Handling:
Given respondent error, regressions at the individual level based on so few data points may sometimes lead to irrational outcomes, such as negative slopes (suggesting a negative relationship between estimated utilities and scores on the Likert scale). Also, if the respondent answers the calibration concepts using the same scale point, there is no variation to model. Therefore, we must handle these

Adaptive CBC (ACBC) Help exceptions.

469

If the estimated slope is negative, the None threshold is set equal to the average utility for products exceeding the threshold point on the Likert scale minus 1. If the slope is negative and no products are above the threshold, we set the none utility equal to the largest utility of any product plus 1. If the slope is negative and all products are above the threshold, we set the none utility equal to smallest utility of any product minus 1. If there is no variation in the Likert scale responses, we do the following: if all products are rated equally at a point below the threshold point on the scale, we set the none utility equal to the largest utility of any product plus 1; if all products are rated equally on or above the threshold, we set the None utility equal to the smallest utility of any product minus 1. With all these exceptions, a fit statistic of 0 is written to the new utility file. Because the None utility is estimated using the equation (UserSpecifiedScalePoint ConstantFromRegression)/Beta, a small positive Beta near zero can cause the None utility to become extremely large in absolute magnitude. To guard against the most extreme cases, if the estimated beta is between 0 and 0.05, we set it to 0.05. For a recent dataset shared with us, 88% of the sample had calibration data that were well behaved and did not require exception handling (there was variation in the dependent variable and the estimated slopes were positive and larger than 0.05). Of the 12% that required exception handling, 5% had tied ratings (across four calibration concepts), 6% had negative slopes, and 1% had slopes between 0 and 0.05 that we trimmed to 0.05.

470

SSI Web v7

3.6.10

Exporting ACBC Data to .CHO File

Some researchers may wish to export the data to a .CHO file for analysis using other software programs, such as our CBC/HB system or Latent Class software. (Notes: SMRT cannot import a .CHO file from an ACBC study correctly, as the new .CHO file includes additional incompatible information. If your design uses constructed lists that drop entire attributes from consideration, please see the warning further below.) Select File + Export Data + Prepare ACBC Data Files (*.cho and *.att)... When you export a .cho file, ACBC reports how many respondents have been exported to the file. It also informs you regarding which attribute numbers in the .cho file are coded as "user-specified." This simply means that these columns in the .cho file should be used in the estimation software "as-is": they require no further effects-coding or dummy-coding. When you open the .cho file within our CBC/HB or Latent Class software, you should change the coding method for these attributes to "user-specified." The "user-specified" columns within ACBC studies reflect the Summed Price attribute, the None parameter, and any threshold parameters for attributes using constructed lists (where selected levels were dropped from consideration). Under the Settings dialog, you can select which respondents to export as well as the method to use for respondent numbering. You can also select which tasks to include in the .CHO file: BYO (by attribute), Screener Section, Choice Task Tournament.

Scaling of Summed Price

The default prior settings in CBC/HB software work well as long as the data are scaled similarly within the design matrix. Generally, attributes are effects-coded, which transforms the level indices to values of 1, 0 or -1. But, if your summed prices are in the hundreds, thousands or millions, it can pose serious problems for proper convergence in CBC/HB. Thus, the software allows you to rescale your price units by supplying a price multiplier on the Settings dialog. We recommend a price multiplier that brings your summed prices into units scaled in the 1s or 10s. For example, a price range of 1 to 5 or 10 to 50 seems to work well. But, prices scaled from 100 to 500 or especially 100,000 to 500,000 would not work well. Please note that if using summed prices, your results when using CBC/HB to estimate utilities will not exactly match the results from running HB within the ACBC interface. That is due to the way we have chosen to rescale price in ACBC to ensure proper convergence under unpredictable price scaling conditions. Within the ACBC interface, HB automatically transforms prices to a -1 to +1 range within the design matrix. Then, before writing the results out to the utility file, it scales the utilities back to the original price scaling. Using a .cho export, however, will result in prices that are not zero-centered. This will especially affect the constant (the None parameter) in the model.

How to Treat Missing Levels

If you used constructed lists for some attributes, we recognize that it is possible that respondents have some levels deleted prior to the ACBC interview. For example, you may have deleted certain brands that were unacceptable or not available to the respondent. You must specify how to treat these levels in the coding of the .cho file. Missing levels can be treated as: Inferior to included levels Unavailable

Adaptive CBC (ACBC) Help Missing at random

471

This parallels the options that are available within the built-in estimation interface for HB within the ACBC system. If you specify that missing levels are to be treated as inferior to included levels, additional choice tasks are written to the file for each respondent, informing utility estimation that included levels are superior to (chosen over) inferior levels. If you specify that missing levels are either Unavailable or Missing at random, these additional tasks are not written. The result is the same for the .cho export whether you select these latter two options.

Warning Regarding Using .CHO File and Missing Attributes

If you used constructed lists to wholly drop certain attributes from the ACBC questionnaire that were deemed entirely unimportant (using questions prior to the ACBC sections), then you should be careful about using CBC/HB, Latent Class, Aggregate Logit or another routine outside ACBC software for estimation. Unless you take special steps to inform utility estimation that certain attributes have been dropped and should have no importance for certain respondents, your utility estimation will overstate the importance of such attributes that were dropped from consideration. ACBC's built-in CBC/HB and Monotone Regression routines inform utility estimation regarding missing (and entirely unimportant attributes) by inserting utilities of zero for the levels of such attributes for respondents who have dropped the attribute from consideration in ACBC. Therefore, if you use another routine, you should take steps such as post-processing the utilities to set the part-worths (at the individual level) to zero for any attributes that have been dropped from consideration by those respondents.

472

SSI Web v7

3.7

ACBC: How Well Does It Work?

To this point, we have conducted three fairly ambitious methodological studies comparing ACBC to standard CBC. Because we have documented the results of those experiments in two white papers published on our website, we refer the reader to those articles and just cover the highlights here. To read more about the details of our studies, we recommend you download and read the following white papers, available from our Technical Papers Library at www.sawtoothsoftware.com: "A New Approach to Adaptive CBC" "Testing Adaptive CBC: Shorter Questionnaires and BYO vs. 'Most Likelies' " ACBC passed through an eight-month beta test program. Over fifty beta testers conducted over 40 studies during that period. We received very positive feedback from our beta testers. Some of their experiences are described on our website, in the ACBC area.

Judging Success
The critical design issue when comparing one conjoint method to another is the criteria for success. Ideally, the researcher would have access to sales data (or subsequent actual choices of respondents) and could compare predicted choices with actual choices. In the absence of data on real-world choices, many researchers turn to holdout choice tasks included within stated preference surveys. Traditionally, these have looked exactly like standard CBC questions: typically 3 to 5 product concepts. One of the key things we and other researchers have learned about standard CBC tasks is that respondents often key on just a few levels to make decisions. They often establish a few cutoff rules for excluding or including concepts within their consideration sets. And, if one uses standard CBC tasks that employ "minimal overlap" (where each level is typically available no more than once per task), often only one product concept can satisfy respondents. Choices to such tasks often reflect simplified (superficial) behavior, and other choice tasks designed in the same way not surprisingly are quite successful in predicting answers to those holdouts. We have found that ACBC does about as well as CBC (sometimes better, sometimes worse) in predicting these kinds of holdout CBC tasks. And, that parity doesn't concern us, as we'll further explain. We have wondered whether traditional holdout tasks really do a good job in mimicking actual purchase behavior, or whether they reflect simplified processing heuristics that are a byproduct of respondents (especially internet sample) completing marketing research surveys with less motivation than they would apply to real-world purchases. When people make real decisions, they often narrow down the choices to an acceptable consideration set (with respect to must-have and must-avoid features) and then make a final choice within the consideration set. To better mimic that process in the holdouts, our first three methodological studies used a customized type of holdout CBC task that involved comparing winning concepts chosen from previous CBC tasks. For example, the respondent might be shown four standard (fixed) CBC holdout tasks, but the fifth holdout included the four winning concepts from the first four tasks. Such customized tasks lead to questions that can probe beyond just the first few critical levels of inclusion and reflect more challenging tradeoffs. We have found that ACBC consistently predicts these choices more accurately than traditional CBC. Some researchers (such as Elrod 2001) have stressed that if we validate to holdout tasks within survey questionnaires, not only should the holdout tasks be excluded from the calibration tasks used for estimating the utilities, but a sample of holdout respondents should be held out as well. For example, we should be using calibration tasks from respondent group A to predict holdout tasks from respondent group B (and the actual tasks used in group B should not have been used in the experimental design for group A). When this is done, one does not compute individual-level hit rates (since it is no longer a within-respondent analysis), but one uses market simulations and measures share prediction

Adaptive CBC (ACBC) Help

473

accuracy. Simulations of market choices for group A are used to predict choices to fixed holdouts for group B. We were fortunate enough to have a robust enough sample in our first ACBC methodological study to follow Elrod's recommendations. In addition to the six hundred respondents in the sample used to estimate part-worth utilities, we were able to collect a separate sample of 900 respondents who all received a single-version (fixed) set 12 CBC tasks. We found that ACBC predicted shares of choice for these scenarios just as well as CBC. But, when we split the 900 respondents into three equal-sized groups based on interview time, we found that ACBC performed worse than CBC in predicting the choices for respondents who were the quickest but better than CBC for respondents that took the most time (and presumably were providing more in-depth and careful answers). We mention these conclusions and findings to provide direction to researchers who may want to test the merits of ACBC relative to more traditional methods like CBC. If standard holdout CBC tasks are used as the criterion for success, there is methods bias that strongly favors CBC in predicting those tasks. If respondents are using the same (probably unrealistic) simplification heuristics to answer calibration CBC tasks, they will employ the same heuristics in answering holdouts. One needs to consider carefully the appropriate criteria for success. You may find that ACBC does not consistently beat CBC in predicting static minimal-overlap holdout choice tasks, for the reasons outlined above. It occurs to us, however, that if static choice tasks include a greater number of concepts than are traditionally used, more overlap will be introduced, increasing the likelihood that respondents will need to make a more careful decision regarding multiple acceptable concepts per task prior to making a decision. In those cases (which are probably more representative of real choice situations), ACBC may very well consistently beat CBC. We suspect that ACBC will be more predictive of real-world buying behavior than CBC, but we need practitioners and firms with access to the appropriate sales data to verify this. This is an exciting new area of research, and we are optimistic that ACBC will indeed prove better than traditional CBC for complex conjoint problems involving about five or more attributes.

Previous Results and Conclusions

To date, we have completed three substantial methodological studies comparing ACBC to CBC. The study names, sample sizes, and numbers of attributes are as follows: Laptop PC Study (n~600 calibration respondents; n~900 holdout respondents; 10 attributes) Recreational Equipment Study (n~900; 8 attributes) Home Purchases study (n~1200; 10 attributes) The experiments all used web-based interviewing. In the case of the Laptop and Home Purchase studies, we used Western Wats panel. For the Recreational Equipment study, customer sample was provided by the client. For the Laptop and Home Purchase studies, we implemented a carefully controlled random split-sample design, where respondents were randomly assigned to receive ACBC or CBC questionnaires, or assigned to be holdout respondents. With the Recreational Equipment study, because of conditions related to working within the client's time line, we weren't able to conduct a perfectly controlled random assignment of respondents to different conditions, and the sample pull receiving the ACBC survey had potentially different characteristics from those receiving CBC. This weakens our ability to draw firm conclusions from that study. Interview Time The ACBC questionnaires took longer than the CBC questionnaires to complete (50% longer in the Housing survey, 100% longer in the Laptop study, and 200% longer in the Recreational Equipment study). We suspect that the Recreational Equipment study was relatively longer because it interviewed respondents from a customer list, who may have been even more enthusiastic and engaged in the

474

SSI Web v7 process. The Housing survey was relatively shorter, because we investigated whether we could shorten the survey length without sacrificing much in terms of predictive validity (see results below, which show that even the shortened ACBC surveys were successful). A key finding is that even though ACBC respondents were asked to do a longer task, they rated the survey as more interesting and engaging than respondents taking CBC surveys rated the CBC surveys. Our thought is that when collecting conjoint data, speed alone shouldn't be the goal, as we cannot ignore quality. And, if we can improve the survey experience for the respondent even though we are asking them to spend more time, then the additional investment to acquire better data is well worth the effort. Interest/Satisfaction with the Survey For the Laptop and Home Purchases studies (which provide the strongest results due to the controlled experiments), the ACBC respondents reported higher satisfaction and interest in the survey (than CBC respondents). They reported that the survey was more realistic, less monotonous/boring, and that it made them more likely to want to slow down and make careful choices (than the CBC respondents). All these differences were significant at p<0.05. Hit Rates In the two studies involving random assignment to either ACBC or CBC cells, the ACBC respondents had higher hit rates (for the customized CBC holdout task) than CBC respondents, with differences significant at p<0.05: Hit Rates, Custom Holdout Task ACBC Laptop Study: Houses Study: 61 44 CBC 50 37

For the Recreational Equipment study (which lacked a completely random split-sample control), ACBC had a 2-point edge in hit rate prediction over CBC, but the result was not statistically significant. All these figures reflect the generic HB estimation method. Some researchers have wondered whether the edge in predictive ability demonstrated by ACBC over CBC might be accounted for by the longer questionnaire time for ACBC. Maybe ACBC's success is due to more precise estimates because we've asked more questions of each respondent. While this seems plausible, we doubt that this can entirely explain the results, because it has been demonstrated that hit rates for CBC respondents gain very little beyond about 10 to 15 choice tasks (and the CBC surveys for the Laptop and Houses studies both included 18 tasks). We come to this conclusion based on a meta-study of 35 commercial data sets by Hoogerbrugge and van der Wagt of SKIM Analytical as presented at the 2006 Sawtooth Software Conference. They randomly threw out one task at a time and estimated the loss in hit rate precision depending on questionnaire length. Based on their research, in the range of 10 to 15 choice tasks, the gains in hit rate rapidly decreased and in some cases leveled off through up to 19 tasks. The net pattern and absolute magnitude of the gains (observed across 35 data sets) by extending the questionnaire would not seem to be able to eventually make up the gaps of 11 and 7 absolute hit rate points in favor of ACBC as seen in our two experiments. In the future, it would be good to see research that time-equalized the CBC and ACBC questionnaires (though respondents receiving the CBC questionnaire will face a monotonous challenge!).

Adaptive CBC (ACBC) Help Share Predictions

475

In the Laptop study, we were able to estimate shares for holdout CBC tasks completed by a new sample of 900 respondents. The 900 new respondents completed 12 fixed CBC tasks. The partworths generated from ACBC performed just as well as those from CBC in predicting these holdout shares, when including all respondents. But, we split the 900 respondents into three equal samples, based on time taken in the interview. After segmenting based on time, we found that CBC predicted the fastest respondents better than ACBC, but ACBC predicted the middle and slower respondent groups better than CBC (with the best prediction occurring for the holdouts completed by the slowest respondents). In each case, we tuned the simulations for scale, so the differences cannot be explained by the random error components (which were significantly different for the three segments of respondents). Also, a Swait-Louviere test (which also controls for scale) found differences in the parameters across the groups of holdout respondents. This finding suggests that ACBC can do a better job at predicting choices made by respondents who take the most time and presumably are being more careful and considerate in their choices. For the Houses Study, we did not collect holdout respondents. However, we were able to use the partworths from the ACBC and CBC respondents to predict choices made by the two groups of respondents combined (a total of 1200 respondents in the validation sample). Thus, ACBC partworths were being used to predict holdout responses made by both ACBC and CBC respondents; and CBC respondents the same. Based on predictions of four fixed holdout tasks (each with four product alternatives), the ACBC predictions were better than those from CBC (the mean absolute errors in prediction were 3.00 and 5.47 share points for ACBC and CBC, respectively). Price Sensitivity In the three methodological studies we conducted, the price sensitivity derived from ACBC estimates was quite similar to that from CBC. Our beta testers have reported some instances where ACBC has produced significantly greater price sensitivity than CBC. In those instances, the ACBC result was viewed as potentially more realistic. We hope to see further research conducted in this area.

476

SSI Web v7

3.7.1

References
Elrod, Terry. (2001), "Recommendations for Validation of Choice Models," Sawtooth Software Conference Proceedings, Sequim, WA. Gilbride, Timothy and Greg M. Allenby (2004), "A Choice Model with Conjunctive, Disjunctive, and Compensatory Screening Rules," Marketing Science, 23, 3, (Summer) 391-406. Hauser, John R., Ely Dahan, Michael Yee, and James Orlin (2006) "'Must Have' Aspects vs. Tradeoff Aspects in Models of Customer Decisions," Sawtooth Software Conference Proceedings, 169-181. Hoogerbrugge, M. and K. van der Wagt (2006), "How Many Choice Tasks Should We Ask?" Sawtooth Software Conference Proceedings, Sequim, WA. Johnson, Rich and Orme, Bryan (2007), "A New Approach to Adaptive CBC," Sawtooth Software Conference Proceedings, Sequim, WA. Orme, Bryan and Rich Johnson (2008), "Testing Adaptive CBC: Shorter Questionnaires and BYO vs. 'Most Likelies'," Technical Paper Available at www.sawtoothsoftware.com.

MaxDiff Help

477

4
4.1
4.1.1

MaxDiff Help
Getting Started with MaxDiff
Introduction
Have you ever been asked to measure respondents' preferences for things such as brands, product features, job-related benefits, or product packaging? Have you ever been asked to prioritize a list of performance attributes or gauge the potential impact of different advertising claims? If so, you should consider the class of trade-off related techniques available within MaxDiff, a component within the SSI Web system. MaxDiff is an approach for obtaining preference/importance scores for multiple items (brand preferences, brand images, product features, advertising claims, etc.) using marketing or social survey research. MaxDiff is a component within Sawtooth Software's SSI Web platform for Web- and CAPIbased interviewing. It may be used for designing, fielding, and analyzing: MaxDiff (best-worst scaling) experiments Method of Paired Comparisons (MPC) experiments (choices from pairs) Choices from subsets of three items, four items, etc. Projects may be conducted over the Internet, using computers not connected to the internet (CAPI interviewing), or via paper-and-pencil questionnaires (using the included MaxDiff paper-and-pencil utility and the separately purchased CBC/HB system). MaxDiff makes it easy for researchers with only minimal exposure to statistics to conduct sophisticated research for the scaling of multiple items. The trade-off techniques used in MaxDiff are robust and easy to apply (for example, much easier than the related conjoint analysis). The resulting item scores are also easy to interpret, as they are placed on a 0 to 100 point scale.

478

SSI Web v7

4.1.2

Motivation for MaxDiff

Researchers across many disciplines face the common task of measuring the preference or importance of multiple items, such as brands, product features, employee benefits, advertising claims, etc. The most common (and easiest) scaling approaches have used rating, ranking, or chip allocation (i.e. constant sum tasks). Example Rating, Ranking, and Allocation questions are shown below (this software does not employ any of these approaches, but we repeat them here for exposition): Example Rating Task: Please rate the following in terms of importance to you when eating at a fast food restaurant. Use a 10-point scale, where "0" means "not important at all" and "10" means "extremely important" ___ ___ ___ ___ ___ ___ ___ Clean bathrooms Healthy food choices Good taste Reasonable prices Has a play area Restaurant is socially responsible Courteous staff

Example Ranking Task: Please rank (from most important to least important) the following in terms of importance to you when eating at a fast food restaurant. Put a "1" next to the most important item, a "2" next to the next most important item, etc. ___ ___ ___ ___ ___ ___ ___ Clean bathrooms Healthy food choices Good taste Reasonable prices Has a play area Restaurant is socially responsible Courteous staff

Example Allocation Task: Please tell us how important the following are you when eating at a fast food restaurant. Show the importance by assigning points to each attribute. The more important the attribute, the more points you should give it. You can use up to 100 total points. Your answers must sum to 100. ___ ___ ___ ___ ___ ___ ___ Clean bathrooms Healthy food choices Good taste Reasonable prices Has a play area Restaurant is socially responsible Courteous staff

Total: ______ There are a variety of ways these questions may be asked, including the use of grid-style layout with

MaxDiff Help

479

radio buttons and drag-and-drop for ranking. And, it is commonplace to measure many more items than the seven shown here. (These examples are not intended to represent the best possible wording, but are meant to be representative of what is typically used in practice.) The common approaches (rating, ranking, and allocation tasks) have weaknesses. Rating tasks assume that respondents can communicate their true affinity for an item using a numeric rating scale. Rating data often are negatively affected by lack of discrimination among items and scale use bias (the tendency for respondents to use the scale in different ways, such as mainly using the top or bottom of the scale, or tending to use more or fewer available scale points.) Ranking tasks become difficult to manage when there are more than about seven items, and the resulting data are on an ordinal scale only. Allocation tasks are also challenging for respondents when there are many items. Even with a manageable number of items, some respondents may have difficulty distributing values that sum to a target value. The mechanical task of making the allocated points sum to the target amount may interfere with respondents revealing their true preferences. Researchers seek scaling approaches that feature: Ease of use for respondents from a variety of educational and cultural backgrounds Strong discrimination among the items Robust scaling properties (ratio-scaled data preferred) Reduction or elimination of scale use bias A very old approach, the Method of Paired Comparisons (MPC), seems to perform well on all these requirements. With MPC, respondents are shown items, two at a time, and are asked which of these two they prefer most (or which is most important, etc.). Example Paired Comparison Task:

The respondent is not permitted to state that all items are equally preferred or important. Each respondent is typically asked to evaluate multiple pairs, where the pairs are selected using an experimental plan, so that all items have been evaluated by each respondent across the pairs (though typically not all possible pairings will have been seen) and that each item appears about an equal number of times. It should be noted that MPC can be extended to choices from triples (three items at a time), quads (four items at a time) or choices from even larger sets. There would seem to be benefits from asking respondents to evaluate three or more items at a time (up to some reasonable set size). One research paper we are aware of (Rounds et al. 1978) suggests that asking respondents to complete sets of from 3 to 5 items produces similar results as pairs (in terms of parameter estimates), and respondents may prefer completing fewer sets with more items rather than more sets with just two items. A much newer technique, MaxDiff, may perform even better than MPC, especially in terms of overall

480

SSI Web v7 question efficiency. MaxDiff questionnaires ask respondents to indicate both the most and least preferred/important items within each set. Example MaxDiff Task:

Interest in MaxDiff is increasing in the last few years, and papers on MaxDiff have won "best presentation" awards at recent ESOMAR and Sawtooth Software conferences.

MaxDiff Help

481

4.1.3

What is MaxDiff?
MaxDiff is a technique invented by Jordan Louviere in 1987 while on the faculty at the University of Alberta. With MaxDiff, respondents are shown a set (subset) of the possible items in the study, and are asked to indicate (among this subset with a minimum of three items) the best and worst items (or most and least important, etc.). Below is an example, involving a set of four items. Example MaxDiff Task:

MaxDiff may be thought of as a more sophisticated extension of the Method of Paired Comparisons (MPC). Consider a set in which a respondent evaluates four items, A, B, C and D. If the respondent says that A is best and D is worst, these two "clicks" (responses) inform us on five of six possible implied paired comparisons: A>B, A>C, A>D, B>D, C>D where ">" means "is more important/preferred than." The only paired comparison that we cannot infer is B vs. C. In a choice among five items, MaxDiff questioning informs on seven of ten implied paired comparisons. MaxDiff questionnaires are relatively easy for most respondents to understand. Furthermore, humans are better at judging items at extremes rather than in discriminating among items of middling importance or preference. And since respondents make choices rather than expressing strength of preference, there is no opportunity for scale use bias. This is an extremely valuable property for crosscultural research studies.

482

SSI Web v7

4.1.4

MaxDiff Tutorial and Example

Introduction
This tutorial leads you step-by-step through a simple MaxDiff study, from thinking about a research problem and planning a questionnaire to entering a list of MaxDiff items in SSI Web, generating and testing the questionnaire, and analyzing the results. SSI Web is a powerful program, and this tutorial only touches on the basics for running a MaxDiff experiment. You'll become aware of different or more sophisticated approaches by reading other sections in the on-line help and by experimenting with SSI Web. (Note: this example is based on a presentation delivered at the 2003 Sawtooth Software Conference by Amanda Kraus, Diana Lien, and Bryan Orme. Though many of the items we use in this example are identical to the ones used in the conference presentation, the data we have developed for this example are purely fictional. The suggested steps represent one approach to this type of research problem, but may not necessarily reflect the single "best" way to conduct such a research study using the SSI Web system.)

The Research Problem

As the economy has improved, the Navy has faced declining reenlistment rates among its sailors. The Navy is looking for economic ways to improve the reenlistment rate. Previous research has shown that reenlistment terms, quality of life issues, and the work environment are all significant factors when sailors face the reenlistment decision. Pure economic aspects matter as well, such as the amount of the reenlistment bonus. You have been brought on as a consultant to help the military design and execute a research study to prioritize a list of possible job and quality of life enhancements as well as reenlistment terms. The Navy is keenly interested in how reenlistment bonuses may affect sailors' desire to enlist relative to other non-monetary issues. Those improvements which are most likely to increase the reenlistment rates at the lowest expense to the military should be implemented first. They would like to implement this study over the Web, and expect that between 1,000 and 2,000 sailors might complete the study. At first, you consider whether a conjoint method such as CBC would be appropriate. You ask your military client, Jane, if the resulting model needs to be able to project the likely reenlistment rate when combinations of job enhancements are taken together. Jane assures you that although that type of analysis would be of interest to military planners, the primary consideration is the ability to prioritize the items from most to least effective in terms of improving reenlistment rates. The planners need to be able to directly compare the effect of each job enhancement or monetary bonus. After pondering the problem, you recognize that a strength of MaxDiff is its ability to place multiple items on the same scale using a trade-off methodology. With conjoint analysis (such as CBC), the client would not be able to directly compare the preference for levels of one job-related item to another. MaxDiff doesn't formally support adding the preference scores across items and predicting the overall benefit of the job package or likelihood of reenlistment given a specific job package (as would CBC), but that is not a requirement for the study. Therefore, you decide to use MaxDiff.

Developing a List of Items

You meet with Jane to discuss the elements (items) to be studied in the MaxDiff exercise. Jane explains that a series of focus groups had been conducted over the last four months wherein the reasons for reenlistment versus leaving the Navy were discussed. Hundreds of ideas had been presented, but there seemed to be some common themes. Furthermore, Jane had discussed the myriad ideas with military planners. Many of the ideas were clearly off-the-table (such as "king-sized

MaxDiff Help

483

beds on submarines," "more parties," and "supermodels as supervisors"). However, there were a number of areas of overlap: job conditions that the military was in the position to change and those that also were on the minds of the sailors. The list included the following: Length of reenlistment obligation Location or duty assignment guarantees for next assignment Faster promotion Living conditions when in port (2-person barracks vs. 4-person barracks) Increase in sea pay Percent of on-job time using your skills and training Time off each week allowed for on-line college coursework Online career counseling services Personal counseling services Reenlistment bonus

This looked like a good start, but it seemed obvious to both you and Jane that some further refinement and quantification of benefits was needful. The items needed to carry specific meaning to the sailors and the results needed to be actionable for the military planners. Most of the items seemed to require multiple levels, whereas some seemed to stand on their own. After thinking about the reasonable items (and some back-and-forth with Jane's internal clients, the military planners) a final list was drawn up: Factors Affecting Reenlistment Decision: 3-year reenlistment obligation 5-year reenlistment obligation Location guarantee for next assignment Duty guarantee for next assignment Both location and duty guarantees for next assignment Get promoted 6-months sooner than expected Live in 2-person barracks when in port Live in 4-person barracks when in port Choice between living in 2-person barracks or 4-person barracks when in port $50 per month pay increase when out at sea $75 per month pay increase when out at sea Half of on-job time using your skills and training Three-quarters of on-job time using your skills and training Time off each week allowed for on-line college coursework Receive online career counseling services Receive in-person personal counseling services Receive phone-based personal counseling services Current reenlistment bonus ($2,500) Increased reenlistment bonus ($5,000) Increased reenlistment bonus ($10,000) Next, you worked with Jane to decide which items would need to be prohibited from displaying with each other versus which could freely combine with all other items within the same set. For example, it would not make sense to ask sailors to choose between a $50 pay increase versus a $100 pay increase when out at sea. Every rational sailor would choose higher pay over lower, so this would be a wasted comparison. Other items such as living in a 2-person barracks versus a 4-person barracks didn't seem to have a rational preference order (though having a choice between 2-person barracks or 4-person barracks would seem to be preferred to no choice). Perhaps some sailors would prefer to live with more people, whereas others would prefer just one roommate. You organized the list into groupings of items (listed as groups A through F), where the items within

484

SSI Web v7 groups A through E would require at least some prohibitions (restrictions that they could not be displayed together within the same choice set). A) Prohibit these two items: 3-year reenlistment obligation 5-year reenlistment obligation B) Prohibit third item with appearing with either item 1 or item 2: Location guarantee for next assignment Duty guarantee for next assignment Both location and duty guarantees for next assignment C) Prohibit third item with appearing with either item 1 or item 2: Live in 2-person barracks when in port Live in 4-person barracks when in port Choice between living in 2-person barracks or 4-person barracks when in port D) Prohibit these two items: $50 per month pay increase when out at sea $75 per month pay increase when out at sea E) Prohibit these items from appearing together: Current reenlistment bonus ($2,500) Increased reenlistment bonus ($5,000) Increased reenlistment bonus ($7,500) F) No prohibitions with other items: Half of on-job time using your skills and training Three-quarters of on-job time using your skills and training Get promoted 6-months sooner than expected Time off each week allowed for on-line college coursework Receive online career counseling services Receive in-person personal counseling services Receive phone-based personal counseling services

Scripting the Questionnaire

A real research project would involve other questions about each sailor, including gender, current rank or job specification, current location, inclination to reenlist, etc. Standard survey questions (such as select questions with radio buttons or check-boxes) can be developed within the SSI Web platform (of which MaxDiff is a subcomponent). Rather than take time within this tutorial to add such classification questions, we'll focus on the MaxDiff section of the survey. Interested users can look to the CiW Tutorial within this same online help and manual for further help with standard survey questions. You decide to include the following introductory text leading into the MaxDiff questions:

MaxDiff Help Start: Thank you for taking time to help the Navy consider ways to improve your working environment and career development. Our goal is to make the Navy a satisfying career for you and to improve your desire to reenlist when the decision arises. We would like you to imagine that you are facing the decision whether to reenlist or not--today. Please think about the issues involved, including future career planning, family/relationship considerations, job location, and compensation. Click the Next button below to continue... IntroMaxDiff: If you were making the decision to reenlist or not today, there would certainly be many factors that affect your decision to reenlist or not. Real life decisions are not always easy. We want to gain some insight into how you make this decision by asking you to evaluate a series of tradeoff scenarios. In each scenario, we'll show you four possible factors affecting your career in the Navy and the reenlistment decision. We'll ask which factor (among this set of four) would make you most want to reenlist, and which would make you least want to reenlist. We need to ask you repeated tradeoff scenarios (involving different factors each time) so that we can learn what is truly important to you when deciding whether to reenlist.

485

Layout for MaxDiff Questions Next, you need to decide the layout of the MaxDiff questions. Research has suggested that showing more than about five items at a time in MaxDiff questions may not be very beneficial. Some of the items in your study are a bit on the wordy side, so you decide to show just four items at a time. The MaxDiff documentation suggests showing each item between 3 to 5 times on average per respondent across the sets. With 20 items in the study, the number of questions to display so that each item is shown on average 3 times per respondent is equal to: 3(K/k) where K is the total number items in the exercise, and k is the number of items per set Therefore, you would need to ask each respondent a minimum of 3(20/4) = 15 sets. The "nice to have" number of sets would be 5(20/4) = 25. After discussing with the client, you decide that respondents should be able to complete 20 sets with little problem (each item displayed on average 4 times to each respondent). You estimate that each set will take about 20 seconds to complete (prior research presented at the Sawtooth Software Conference reports that figure), for a total average interviewing time of between 6 to 7 minutes for the MaxDiff section. To try to keep respondents engaged and working productively through the MaxDiff exercise, you plan to show respondents a progress bar and also provide a count-down message after every five MaxDiff questions. Here's an example of the MaxDiff question layout you plan to use:

486

SSI Web v7

After every five MaxDiff questions, you plan to show some text that provides encouragement and a countdown, such as: ProgressCounter: Good work so far. We know that some of these tradeoffs are challenging. We need to ask you to evaluate 15 more sets so that we can better determine how you make your reenlistment decision.

And at the end, you want to display a screen that thanks respondents, and then immediately sends them to a Navy site www.navy.mil. Finish: Thank you very much for your help. Your data will be used to help Navy planners as they work to make the Navy an even better career choice. Click the Next button to finish submitting your data and to return to the Navy's official website.

MaxDiff Help

487

Using SSI Web to Write the Questionnaire

Start by opening the Sawtooth Software SSI Web program. If you have a standard installation, you start it by clicking Start | Programs | Sawtooth Software | Sawtooth Software SSI Web. "SSI" stands for "Sawtooth Software Inc." and "Web" conveys the idea that this software is for developing Web-based surveys (although SSI Web can also be used for standalone computer interviewing with its CAPI interviewing module.) MaxDiff is one component within that software program. (Although not emphasized here, MaxDiff may be used for conducting paper-based MaxDiff projects.) After starting SSI Web, choose File | New Study. The New SSI Web Study dialog appears, with your cursor active in the Name field.

The Location field lists the folder (directory) that SSI Web currently is using to store studies. You can use any folder you like for this tutorial project. You can browse to and create new folders by clicking the Browse... button to the right of the Location field, and then (after browsing to the folder in which you want to create a new sub-folder to contain this study) by clicking the "new folder" icon. Each studyname in SSI Web has a maximum number of twenty-four characters (either letters or numbers), and SSI Web requires that the study be located within its own folder. The folder that SSI Web automatically creates for your study also carries the same name as the study. For this tutorial, you might create a name such as navyproject. From the New SSI Web Study dialog, specify navyproject as the studyname. Click the OK button. You are returned to the main menu, and a new "Study Navigator" window is displayed. As you work with this study, items appear in the Study Navigator window, listing the functional areas you have been using or that are now available to you. This list of items provides a Shortcut link to access those parts of SSI Web. Alternatively, you can access those same areas by clicking icon buttons on the toolbar or by using the pull-down menus.

Adding the Introduction Screens to the Survey

(If you are already familiar with the Start (passwords) question, Text/HTML Filler and Select questions within SSI Web surveys, you may choose to skip forward to the section entitled "Adding MaxDiff Questions to the Survey"). Let's enter the introductory text using the Write Questionnaire dialog. You can access the Write Questionnaire dialog by selecting Compose | Write Questionnaire or by clicking the "pencil" icon

488

SSI Web v7

on the toolbar. The introductory text and survey questions would most likely be initially developed within a word processing document. Assuming you really had such a document, you might press Ctrl-C to copy, and Ctrl-V to paste the information into SSI Web. We suggest you simply copy-and-paste the text within this document into your SSI Web questions rather than re-type the text (if viewing this document with Acrobat Reader, you can use the "text select" icon from the Acrobat Reader toolbar.) The first question to compose is the introductory screen: Start: Thank you for taking time to help the Navy consider ways to improve your working environment and career development. Our goal is to make the Navy a satisfying career for you and to improve your desire to reenlist when the decision arises. We would like you to imagine that you are facing the decision whether to reenlist or not--today. Please think about the issues involved, including future career planning, family/relationship considerations, job location, and compensation. Click the Next button below to continue... We'll insert this introductory screen in the Start question that is automatically the first question of any SSI Web survey. The Start question is also the place where you would ask respondents to type their passwords to access the survey. We'll not use passwords for this sample survey. From the Write Questionnaire dialog open the Start question by highlighting it in the list of questions and clicking Edit.... Alternatively, you can double-click the Start question, and it automatically opens. The Questionnaire Access and Passwords dialog is displayed::

MaxDiff Help

489

Most question types in SSI Web have "Header 1", "Header 2," "Body" and "Footer" sections. (The Start question has all but the "Body" section.) These are "text" areas in which you can insert any text (including HTML). When the question is viewed with a web browser, the sections are organized roughly as follows:

For this introductory screen (that includes three paragraphs of information), it seems to make sense to place the first paragraph in the "Header 1" area, the second paragraph in the "Header 2" area, and the

490

SSI Web v7 third paragraph in the "Footer" area. Type (or cut and paste) the following text for the Start question into the text areas in the Start question. Put the first paragraph in Header 1, the second paragraph in Header 2, and the third paragraph in Footer. Thank you for taking time to help the Navy consider ways to improve your working environment and career development. Our goal is to make the Navy a satisfying career for you and to improve your desire to reenlist when the decision arises. We would like you to imagine that you are facing the decision whether to reenlist or not--today. Please think about the issues involved, including future career planning, family/relationship considerations, job location, and compensation. Click the Next button below to continue... The main dialog only shows a portion of the text, but you can click the "pencil" icon each text area to expand the text editing window. adjacent to

To preview how the question will look when it runs on the respondent's computer using Microsoft Internet Explorer, click the Preview button. The question is displayed in "Preview" mode (no data are saved, and no special "error checking" or "validation" is performed for the question).

SSI Web automatically places paragraph breaks (extra blank lines) between the Header 1, Header 2, and Footer sections. If you put all the text in a single section, you may see that when the web browser interprets the text, it all runs together without any blank lines between the paragraphs (unless you insert some HTML instructions to force blank lines between paragraphs). We'll talk about using HTML within your text to take greater control over the layout, font, and style later in this tutorial. Click the OK button or the "X" icon in the upper-right hand portion of the Preview window to close that window and return to the previous SSI Web dialog. After viewing the text in preview mode, you might decide that it is really too small, or that you want to make the text bold. You can modify the font and sizes of the font using the Paintbrush icon. Or, you can select a different Style (selecting styles was described in the 45-minute tutorial for SSI Web). After changing any setting, click Preview again. After you are happy with the layout of the introductory screen, click OK to return to the Write Questionnaire dialog.

MaxDiff Help

491

Remember that when you add a new question to the List of Questions, it is inserted directly below the question (or page break) that was highlighted when you clicked the Add... button. If you need to move a question once it has been added to the List of Questions, simply highlight the question to be moved and click the or buttons to move the questions within the list (you can also highlight a question and click Ctrl-X or Ctrl-C to cut or copy and Ctrl-V to paste questions within the current questionnaire, or even to another SSI Web study.) Now, add a the second introductory screen on a separate page. The text for that screen is as follows: If you were making the decision to reenlist or not today, there would certainly be many factors that affect your decision to reenlist or not. Real life decisions are not always easy. We want to gain some insight into how you make this decision by asking you to evaluate a series of tradeoff scenarios. In each scenario, we'll show you five possible factors affecting your career in the Navy and the reenlistment decision. We'll ask which factor (among this set of five) would make you most want to reenlist, and which would make you least want to reenlist. We need to ask you repeated tradeoff scenarios (involving different factors each time) so that we can learn what is truly important to you when deciding whether to reenlist. Note that there already is a page break following the Start question. Highlight the page break and click Add.... Select Text/HTML Filler as the question type with IntroMaxDiff as the question name. Click OK to add and then edit the question by double-clicking it from the list of questions. This introductory screen has four paragraphs and it also includes two words that need to be underlined (most and least). Therefore, we'll show you how to how to format text in HTML. While we're showing you how to add underlining to text, we will show you how to insert paragraph breaks as well. Copy and paste the full text for the IntroMaxDiff question from this document into the Header 1 field. Next, click the "pencil" icon adjacent to the Header 1 text area to expand the text editing window.

Highlight the first full paragraph with the mouse and click the "paragraph break" icon which inserts the HTML text <p> at the beginning of the paragraph and </p> at the end of the paragraph. Repeat the same steps for all four paragraphs. Next, we need to underline the words most and least. Highlight "most" and click the "underline" icon . Repeat the same for the word "least". When you are finished, the dialog should look like:

492

SSI Web v7

Click OK to close the text editor dialog and Preview to see how it is displayed in the browser window. At this point, we are ready to add the MaxDiff questions to the questionnaire.

Adding MaxDiff Questions to the Survey In this section, we'll add the MaxDiff questions to the survey, directly following the two introductory screens we previously added. From the Write Questionnaire dialog, insert a page break beneath the IntroMaxDiff question. While the page break beneath IntroMaxDiff is highlighted, click Add... and select MaxDiff as the Exercise type. Specify a name for this exercise, such as Reenlist. (Note: you can have more than one MaxDiff exercise within the same SSI Web questionnaire. In this tutorial, we'll only insert a single exercise.) Click OK and the MaxDiff Exercise dialog is shown. The MaxDiff Exercise dialog has six tabs: Question Text (here you type the text to display directly above and below the question) Label Text (here you specify the text to display directly next to the item text) Items (here you specify the list of items in the exercise) Format (you may format your questionnaire, including widths and colors from this tab) Design (the number of items to display per set and number of sets is specified here) Skip Logic (this dialog is inactive and only present for consistency with other dialogs in the SSI Web system) We'll begin by specifying the 20 items used in this exercise within the Items tab. Click the Items tab. A list box is displayed which allows you to specify a new list of items to be used in this SSI Web study. Or, you can select from an existing list of items. Since we have not yet specified a list within this study, we'll add a new one. By default, the list name is provided as ReenlistList. Let's go ahead and use this default name.

MaxDiff Help

493

(Note that other options such as making items on the list "Other Specify" items or "None of the Above" options are available. These are for consistency with other dialogs in the SSI Web System, but are of no use in MaxDiff exercises.) From this dialog, we could click the Add... button and type the items individually, but it is easier to copy-and-paste the list of items from an existing document. Copy-and-Paste the list of items by highlighting the items below... 3-year reenlistment obligation 5-year reenlistment obligation Location guarantee for next assignment Duty guarantee for next assignment Both location and duty guarantees for next assignment Get promoted 6-months sooner than expected Live in 2-person barracks when in port Live in 4-person barracks when in port Choice between living in 2-person barracks or 4-person barracks when in port $50 per month pay increase when out at sea $75 per month pay increase when out at sea Half of on-job time using your skills and training Three-quarters of on-job time using your skills and training Time off each week allowed for on-line college coursework Receive online career counseling services Receive in-person personal counseling services Receive phone-based personal counseling services Current reenlistment bonus ($2,500) Increased reenlistment bonus ($5,000) Increased reenlistment bonus ($10,000) ...and pressing Ctrl-C to copy them into the clipboard, and then switching to the Items tab within the SSI Web System and clicking the "Paste from Word" inserted within the list. icon. The 20 items are all automatically

Next, we'll specify the text to display above the MaxDiff sets into the Question Text tab. Click the Question Text tab. Some default text is available within this question, but we'll replace it with wording more appropriate for our study. Paste the following text into the Header 1 field: Imagine you were making the decision to reenlist in the Navy today. Which of the following four factors would make you most and least likely to want to reenlist? Insert paragraph breaks and underlining as we showed you earlier in this tutorial. Delete the wording in Header 2 and the Footer. We are almost finished formatting the basics for our MaxDiff exercise. Click the Label Text tab, and type the labels to display over the radio buttons respondents click to indicate best and worst items. The text to use is as follows: Best label: Makes me <u>Most</u> want to reenlist Worst label: Makes me <u>Least</u> want to reenlist Next, we need to tell SSI Web how many MaxDiff questions (sets) to display in the questionnaire. Click the Design tab. Basic design settings for your MaxDiff exercise are displayed:

494

SSI Web v7

You may recall that we previously decided that each set would present 4 items at a time and that the respondent would receive 20 questions (so each item was shown four times on average). Specify 4 in the Number of Items per Set (Question) field. Specify 20 in the Number of Sets (Questions) per Respondent field. At this point, we might preview the MaxDiff question to see how we are doing so far. Click the Preview button, and you may see something like this:

Note that by default the preview displays the first four items in our list therefore some of the items that we wanted to prohibit are being displayed together. We can add the prohibited combinations under this same Design tab. Inserting prohibitions is a capability provided within the Advanced Settings on this dialog. (After retiring the Preview dialog) click Show Advanced Settings to reveal additional controls, including the Prohibitions... button. Click the Prohibitions... button. We need to prohibit the following pairs of items for our exercise:

MaxDiff Help Prohibit this item... 3-year reenlistment obligation Location guarantee for next assignment Duty guarantee for next assignment Live in 2-person barracks when in port Live in 4-person barracks when in port $50 per month pay increase when out at sea Current reenlistment bonus ($2,500) Current reenlistment bonus ($2,500) Increased reenlistment bonus ($5,000) ...from appearing with this item. 5-year reenlistment obligation

495

Both location and duty guarantees for next assignment Both location and duty guarantees for next assignment Choice between living in 2-person barracks or 4person barracks when in port Choice between living in 2-person barracks or 4person barracks when in port $75 per month pay increase when out at sea Increased reenlistment bonus ($5,000) Increased reenlistment bonus ($10,000) Increased reenlistment bonus ($10,000)

Click an item to prohibit from the list on the left, then click the item to prohibit from the list on the right. Continue picking items until all prohibitions have been specified. Click the OK button when finished. Now, we are ready to "generate the design." The design is all the combinations of items that respondents will see across their 20 choice sets (questions). (Note: it may take a few minutes for the computation to finish.) The sets need to be chosen carefully, so each item appears about an equal number of times, and that each item is shown about an equal number of times with every other item (non-prohibited item, that is). Click Generate Design, and the software tries 1000 separate attempts to create a balanced experiment (using a different random number start each time). The best attempt is selected, and the results are shown to you in a design report. When the computation finishes, you are shown a report (which you have the opportunity to save to a text file if you wish). We'll explain the report, one step at a time. First, design specifications are reported:
Number of Items (Attributes): 20 Number of Items per Set: 4 Number of Sets per Respondent: 20 Number of Versions: 300 Number of Iterations: 1000 Random Number Seed: 1 Number of Prohibitions: 9 Iteration 612 was chosen

Above, we see that the software made 1000 total attempts (iterations) to generate a design plan, and the 612th attempt was chosen as "best." In total, the software created 300 versions of the questionnaire (where each version features different item combinations within the sets) to display to respondents. Multiple versions of the questionnaire are generated to improve the stability of the item scores and also to reduce psychological order and context effects that might occur if every respondent received the same combinations of items. Although you plan to interview 1000 or more respondents, it would probably be overkill to generate even more unique questionnaire versions. (Some might argue that 300 versions is already overkill!) Additional versions would provide imperceptibly little measurable benefit, though the software lets you specify up to 999 versions. Each respondent will receive one of these versions of the questionnaire, and many of the questionnaires will therefore be repeated. Below, the one-way balance is reported. This shows how many times each item is displayed across all 300 versions of the questionnaire plan, where each version had 20 sets displaying 4 items at a time. If the design was perfectly balanced, each item would be displayed 1200 times. As can be seen, the

496

SSI Web v7 design is nearly perfect with respect to one-way frequencies:

One Way Frequencies: Item Times Used ---------------1 1200 2 1200 3 1200 4 1200 5 1200 6 1200 7 1200 8 1200 9 1200 10 1200 11 1201 12 1200 13 1200 14 1200 15 1200 16 1200 17 1200 18 1199 19 1200 20 1200 Mean = 1200.000000 Std Dev. = 0.316228

The quality of the design is summarized as the standard deviation across the 20 reported frequencies, or 0.316228. This number is used to classify the quality of the design for multiple attempts (iterations) the software makes. Attempts that demonstrate a lower standard deviation across the frequency counts are deemed better. Next, the report summarizes how many times each item appeared together with every other item within sets. Ideally, each item would appear an equal number of times with every other item. You may recall that we prohibited nine combinations from occurring, so there cannot be perfect balance. These prohibited combinations are shown with 0 frequency in the table below (for example, item 1 shown with item 2), highlighted in red.

Although the balance in the non-prohibited cells is not nearly as good as the one-way counts (the nine prohibitions certainly inhibited the MaxDiff designer from finding a balanced design), the results are still

MaxDiff Help

497

quite acceptable in practice. In the two-way table, each non-prohibited cell is represented a minimum of 162 and a maximum of 217 times. How would you know if you were in deep trouble due to prohibitions? If you have too many prohibitions, you would receive an error message reporting that it is impossible for the designer to ensure connectivity across items in the study. If you have to prohibit a single item from appearing with more than half of the other items in the study, then this may be excessive. Again, we'd like to stress that it is much more important that the one-way frequencies be nearly balanced than to achieve approximate balance in the two-way frequency table. (Note for advanced users: We were interested in the potential loss in design efficiency resulting from these nine prohibitions, so we performed a simple test. We assumed 250 respondents receiving this study as just designed versus the same study in which there were no prohibitions. The design with no prohibitions had perfect one-way balance, and nearly perfect two-way balance (low of 189 and high of 190 counts per two way cells). Upon estimating MNL parameters and studying the standard errors, we determined that the difference in overall design efficiency between these two designs was only 1%. For those familiar with conjoint analysis, the important point is that prohibitions are much less detrimental to MaxDiff designs than conjoint designs. That is because prohibitions in conjoint are within-concept whereas prohibitions in MaxDiff are between-concept. And, the nine prohibitions in MaxDiff eliminate a much smaller proportion of the full-factorial combinations than would nine prohibitions in a typical conjoint plan where each attribute has three to five levels.) Finally, the report shows how often each item appears in the four positions within sets (top, bottom, etc.). This type of balance has no bearing on statistical design efficiency. But, balance here helps reduce psychological order effects (e.g. we wouldn't want a given item to always appear in the top position each time). The positional balance for our design is nearly perfect:
Positional Frequencies: | Pos. 1 2 3 4 ----------+------------------------------------Item 1| 300 300 300 300 2| 300 300 299 301 3| 299 301 300 300 4| 300 299 301 300 5| 300 300 301 299 6| 300 301 300 299 7| 300 300 300 300 8| 300 300 300 300 9| 300 300 300 300 10| 300 299 301 300 11| 300 300 301 300 12| 300 300 300 300 13| 300 300 299 301 14| 300 301 299 300 15| 300 300 300 300 16| 300 300 300 300 17| 300 300 299 301 18| 301 299 300 299 19| 300 300 300 300 20| 300 300 300 300 Mean = 300.000000 Std Dev. = 0.524404

At this point, we have the basics programmed for this MaxDiff study. You can click OK to return to the Write Questionnaire dialog, where you'll see the twenty MaxDiff questions automatically inserted within your project:

498

SSI Web v7

Inserting the "Progress" and "Finish" Questions

When respondents have completed five MaxDiff questions, you had wanted to display a screen that says: ProgressCounter:

MaxDiff Help

499

Good work so far. We know that some of these tradeoffs are challenging. We need to ask you to evaluate 15 more sets so that we can better determine how you make your reenlistment decision.

Insert a new Text/HTML Filler question by highlighting the Page Break directly beneath the Reenlist_5 question, and then clicking Add.... Select Text/HTML Filler as the question type and name the question ProgressCounter. Copy-and-paste the text for this question from this document and format the text into paragraphs. Preview the question to make sure it looks as expected. Click OK to return to the Write Questionnaire dialog, and insert another page break directly following this ProgressCounter question. If you would like, you can insert similar "progress screen" reminders after the 10th and 15th MaxDiff questions. Last, we need to insert the final screen in which we tell respondents that they have completed the survey and link them to the Navy's official site. Finish: Thank you very much for your help. Your data will be used to help Navy planners as they work to make the Navy an even better career choice. Click the Next button to finish submitting your data and to return to the Navy's official website. To do this, scroll to the bottom of the list of questions on the Write Questionnaire dialog. Highlight the final question in the list (Reenlist_20) and insert a page break. Then, click the Add... button and specify a Text/HTML Filler question named Finish. Copy-and-paste the text for this question from this document, implement any formatting you desire. Click OK to return to the Write Questionnaire dialog. Next, insert a page break directly following the Finish question. Then, insert a new Terminate/Link question directly following that entitled FinishLink. No text should be shown on this page. This page simply redirects respondents immediately to the Navy's website. While editing the FinishLink question, click the Settings tab. Check the Terminate Respondent box indicating that respondents seeing this page should be terminated and marked as completed records. Then, check the second Link to a Questionnaire or Website box and type within the provided field so that the respondent is sent to http://www.navy.mil. When you are finished, that dialog should look like:

500

SSI Web v7

Page Layout and Study Settings

It may be helpful at this point to review how SSI Web breaks the survey into separate pages, and some basic global settings that affect the look and functionality of your SSI Web questionnaire. We also want to add a progress bar to our survey. The Write Questionnaire dialog also displays the page breaks that are currently set for your questionnaire. Return to the Write Questionnaire dialog. This dialog shows how the various questions we've specified (or that SSI Web has automatically added to the list) are arranged across different pages. Page breaks are indicated by "---------------<Page>".

From the Write Questionnaire dialog, click the Survey Settings icon (alternatively, click Compose | Survey Settings...). This dialog controls global aspects of the questionnaire (meaning that the settings here control the general behavior of the questionnaire rather than page- or questionspecific aspects.) Examples include the fonts and colors used in the questionnaire, the wording for the "Next" button, and the global header and footer that can appear at the top or bottom of each page of the survey. The details of this dialog are beyond the scope of this unit, but you can learn more about the items on the dialog by pressing the F1 key. However, we do wish to insert a progress bar. This is easily done by clicking the Progress Bar tab, and checking the Automatically show the progress bar box with the position selected as Page Footer. Save your changes and exit the Write Questionnaire dialog.

Adding Polish and Style

You have probably noted that the survey we've created is pretty bland looking. We can add some polish and style by clicking the Survey Settings icon from the Write Questionnaire dialog (or by clicking Compose | Survey Settings from the main menu). First, let's select a style from the Styles tab. Select a style you wish to use, and click Use Selected at the bottom of the dialog to implement the style. We'll select Panama as the style, but you might want to experiment with other styles.

MaxDiff Help

501

On the General Settings tab, you can select to use graphical Next, Previous, and Select/Checkbox buttons. A library of buttons is provided, available in the C:\Program Files\Sawtooth Software\SSI Web\graphics folder. On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Reenlistment Questionnaire as the header text. The Progress Bar tab lets you add a progress bar to your survey. We'll click the check box to add a progress bar to the page footer. When you go back and test run the survey, your survey should have a bit more polish. Under Panama style (and by selecting graphical Previous and Next buttons), our survey now looks like:

Test Running Your Survey Using "Local Test Server"

Although the Preview Survey function is nice, you generally will want to test your survey in runtime mode, with any skip patterns, response verification, and randomizations in place. You could upload your files and the Perl scripts to the Web Server, but SSI Web provides a more convenient way to test run your survey locally on your PC or laptop. When SSI Web was installed on your computer, web server software called "Apache" was also installed. Apache lets your computer run the questionnaire just like a remote Web Server would.

502

SSI Web v7

Local Test Server

From the Write Questionnaire dialog, Click the Test Survey button Locally.

and select to Test Survey

Your browser opens up the first survey page. This is your survey, as it would appear to a respondent over the Internet (there are slight visual differences from browser to browser). To close the survey, simply close the browser window (using the X in the upper-right hand corner). To run the survey again, click the Test Survey button .

After you are convinced that the survey is functioning properly, you should also pretest your survey among your colleagues. They can give you feedback regarding the usability of the survey, and you can examine the resulting Counts data or perhaps even compute individual-level scores using the Analysis | MaxDiff Scores dialog (assuming you have enough data) to make sure the data at least have face validity. At this point, you would set up the study on your server (or use the hosting services provided by Sawtooth Software). Server setup is beyond the scope of this tutorial, but is covered in this SSI Web manual. You would invite the respondents to complete the survey (probably using an email message with a clickable link).

Data Management
SSI Web includes an Admin Module so that you can monitor or access your project from any computer connected to the Web. The Admin Module is password protected, and your passwords for access are specified on the Server Settings tab of the Survey Settings dialog. SSI Web generates random administrative access passwords whenever you create a new study, but you can modify them to suit your needs. To download the data for our study, you would browse to the administrative module for your study on the web site (again, we are speaking hypothetically, as for this tutorial study there is no such site set up). Once at the administrative module, you would download your data (by clicking Download Data from the main menu), making sure to save your data (named STUDYNAMEdat.dat, where STUDYNAME is your study name) to the same folder on your hard drive in which you developed the SSI Web project. Alternatively, from the SSI Web interface, you can simply click Field | Download Data... to download your data without logging into the Admin Module. Once you have downloaded the data, you are ready to analyze the data by Counting analysis (Analysis | MaxDiff Counts) or through a more sophisticated methods called hierarchical Bayes analysis through clicking Analysis | MaxDiff Scores.

Analysis: Counts and Computed Scores

At this point, we would like you to open the Tutorial study which contains hypothetical MaxDiff data for you to analyze. Click Help | Sample Studies | Sample MaxDiff. We'll assume that you have collected data over the web and have downloaded the data to the folder on your hard drive containing the study files (as described in the previous section).

MaxDiff Help

503

A quick summary method called "counts" is often used for summarizing respondents' preferences. Counting analysis simply looks at how many times an item was available within sets, and counts how many times it was chosen either best or worst. The percent of times an item was chosen as best or worst (when it was available) is termed the "count proportion." Run counting analysis for the tutorial data set by clicking Analysis | MaxDiff Counts.... The MaxDiff Counts dialog is displayed. It has various settings, including the ability to select which MaxDiff exercise to use (we only have one in our study) and to select which sets to include in the analysis (we'll use all sets). Click Compute Counts..., and after a few moments of computation, a counts report is displayed:
MaxDiff Counts Analysis MaxDiff Exercise Sets Included Respondents Included Total Number of Respondents Total Number of Sets Reenlist All Completes Only 250 5000 Number Times Best Times of Times Selected Count Selected Shown Best Proportion Worst 999 1000 999 1000 1000 1000 1000 1000 1000 1000 1000 1001 1001 1000 1000 1001 1000 1000 999 1000 310 72 191 202 361 183 252 208 294 230 246 185 199 190 234 231 195 276 386 555 0.310 0.072 0.191 0.202 0.361 0.183 0.252 0.208 0.294 0.230 0.246 0.185 0.199 0.190 0.234 0.231 0.195 0.276 0.386 0.555 177 561 286 283 135 307 231 276 186 271 225 311 251 309 268 238 279 189 110 107 Worst Count Proportion 0.177 0.561 0.286 0.283 0.135 0.307 0.231 0.276 0.186 0.271 0.225 0.311 0.251 0.309 0.268 0.238 0.279 0.189 0.110 0.107

Item Number 3-year reenlistment obligation 5-year reenlistment obligation Location guarantee for next assi Duty guarantee for next assignme Both location and duty guarantees Get promoted 6-months sooner than Live in 2-person barracks when in Live in 4-person barracks when in Choice between living in 2-person $50 per month pay increase when o $75 per month pay increase when o Half of on-job time using your sk Three-quarters of on-job time usi Time off each week allowed for on Receive online career counseling Receive in-person personal counse Receive phone-based personal coun Current reenlistment bonus ($2,50 Increased reenlistment bonus ($5, Increased reenlistment bonus ($10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

The report shows results for 250 respondents and 5000 sets (remember, each respondent received 20 choice sets; 250 x 20 = 5000). You may recall that we had originally generated 300 questionnaire versions for respondents to potentially complete. We only interviewed 250 respondents, so not all questionnaire versions were used. However, we see from the "Number of Times Shown" column that each item was shown almost exactly the same number of times (subsets of questionnaire versions are also carefully balanced). In the "Times Selected Best" column, we see how many times each item was selected best (across all sets and respondents). Dividing the "Times Selected Best" by the "Number of Times Shown" results in the "Best Count Proportion." The higher the proportion, the more impact this item has on reenlistment. Counts proportions are a good way to get a quick (top-line) look at the data. This method of analysis is very intuitive. But, we don't suggest you use counts results in final reporting. Because of prohibitions in your design or slight imbalances, the data are not as accurate as when using the more timeconsuming HB estimation routine, automatically provided under Analysis | MaxDiff Scores.... Also, the counts probabilities tend to be artificially "smoothed" relative to the final estimated scores where we see larger differences between items. The item selected best most often was "Increased reenlistment bonus ($10,000)." Thus, we can see that these sailors (at least our hypothetical ones) pay most attention to monetary rewards. Even the $5,000 reenlistment bonus tends to trump every other issue. If you are interested, you can also examine the proportion of respondents choosing each item as worst. This data set was generated with

504

SSI Web v7 artificial symmetric properties, so there is no difference between bests and worsts other than random noise. However, this isn't always the case with real data sets, where respondents may not use a consistent scoring approach when judging bests and worsts. Let's turn our attention to the more powerful and accurate method of analysis offered in the MaxDiff System: hierarchical Bayes (HB). A typical data set may take about 10 to 15 minutes to run using HB (large data sets may take hours!). HB estimates a set of scores for each respondent. These scores may be saved to an Excel-compatible file. How HB actually works is beyond the scope of this tutorial, and we refer you to technical papers on HB on our website, www.sawtoothsoftware.com. Click Analysis | MaxDiff Scores... to run the HB estimation routine and estimate scores for each respondent. The MaxDiff Scores dialog is shown. As with the MaxDiff Counts dialog, you can select which MaxDiff exercise to use and which sets to include. Click Prepare for Scores Computation.... First, the software reads the data set and prepares it in a more compact way for more efficient processing. As part of this step, the software reports how many respondents it has found and how many times items were chosen in each position of the question.
** Data Summary ** Total respondents = 250 Total 'Best' choices = 5000 Total 'Worst' choices = 5000 Average sets per respondent = 20.00 'Best' responses by position: (1) 24.38% (2) 26.34% (3) 25.08% (4) 24.20% 'Worst' responses by position: (1) 24.76% (2) 25.28% (3) 24.78% (4) 25.18%

Click Compute Scores, and the estimation of scores using hierarchical Bayes (HB) begins. You do not need to have any special training in statistics to consistently achieve good results with this analysis. HB uses an iterative method that requires many thousands of iterations to stabilize and complete the run. We have chosen defaults to govern the HB run that should work well for most every situation. Advanced users can modify the settings if they desire. Partway through the estimation, the MaxDiff Scores dialog may look something like this:

MaxDiff Help

505

This display shows how well the estimated scores are stabilizing over the planned thousands of iterations. The graphic plots the estimated scores at each iteration of the process. The estimates first all start at 0, and then trend toward their final values (which may be either negative or positive). Once the process has "converged" the estimates will tend to randomly "wobble" up and down, but there should be no noticeable trend remaining. The graphic is divided into two regions: a gray region (iterations prior to assumed convergence) and a white region (iterations after assumed convergence). Once the iterations cross into the white region, you should look for no noticeable trend in the data. The plotted scores should just continue to wobble randomly up and down, with no strong and sustained move in any specific direction. If there is noticeable trend after the plotted values are in the white region of the graphic, then you should repeat the estimation using more iterations (under the Advanced button). (Note that for sake of brevity we didn't include "constraints" in this estimation to force the scores for some items to be higher than others, such as $10,000 bonus higher than $5,000 bonus. You do not need to enforce such constraints during estimation, though some researchers choose to do so.) When the estimation finishes, the graphical plot of the estimation remains on the screen. Click the button stating: The estimation is complete. Click here to close. When you click the "close" button, a report is given which resembles a spreadsheet, with multiple "sheets" that can be accessed by clicking the tabs at the bottom of the dialog labeled "Summary," "Rescaled," and "Raw." The first (Summary) sheet contains a summary of the final scores, 95% confidence intervals,and other information regarding the settings for your run:

506

SSI Web v7

MaxDiff Help

507

The second (Rescaled Scores) and third (Raw Scores) sheets contain the scores for each respondent, including a fit statistic. Most researchers will pay most attention to the Rescaled scores as they are easiest to interpret and present to others. These individual-level item scores are positive values ranging from 0 to 100 that sum to 100. This approach has the valuable property of ratio-scaling. That is to say, an item with a score of 20 is twice as important (or preferred) as an item with a score of 10. Respondents who answer randomly will generally have low fit statistics. We have provided guidelines (fit statistic thresholds) for identifying "bad" respondents elsewhere in this documentation. You can use the suggested cutoff rules to discard respondents that are most likely answering randomly. Click the Save As... button to save this report. Two formats are offered: Microsoft Excel format --an Excel Workbook .XLS file is saved with multiple sheets (one for each of the three sheets displayed in this dialog) .CSV (Comma Delimited) --only the current displayed sheet is saved to the file. To save all information from the three sheets to .CSV format, select each sheet separately and save each to a unique .CSV file.

508

SSI Web v7

4.2

Creating a MaxDiff Experiment

Each SSI Web questionnaire may include one or more MaxDiff experiments (a series of questions designed in a careful manner to lead to the efficient estimation of item scores). (Remember, even if you are not asking the "worst" half of the MaxDiff question or if you are doing a method of paired comparisons study, we still refer to the experiment in the software as "MaxDiff.") Each MaxDiff experiment uses a list of items. This list may be a pre-defined (fixed) list, or it can be built dynamically for each respondent (a constructed list). If a constructed list, it must have a prespecified number of items. To add a MaxDiff experiment with a list of items to measure, from the Write Questionnaire dialog, click Add | MaxDiff Exercise. Specify a name for the exercise (starting with a letter and having no more than 25 total numbers or letters). Click OK and the MaxDiff Exercise dialog is shown. The MaxDiff Exercise dialog has six tabs: Question Text: Here, you specify the text that appears above and below the MaxDiff questions. Label Text: Here, you specify the labels that appear above the columns associated with "best" and "worst" (or "best" only, if you are not asking respondents to identify the "worst" item in the set). Items: This tab lets you add or delete items from the list used in the MaxDiff exercise. Please note that you can use any predefined (fixed) list, or you can select a constructed (dynamic) list for use in MaxDiff experiments. Format: The layout style and colors are specified on this dialog. Design: Here, you specify how many items to show per question, how many questions to ask, any level by level prohibitions, and other controls that govern the experimental plan. Skip Logic: This tab is not active, and is available only for consistency with other parts of the SSI Web software.

MaxDiff Help

509

4.2.1

Question and Label Text

Preparing Respondents for the Repetitive Task
Prior to asking respondents to complete a MaxDiff (or Method of Paired Comparisons exercise), it is helpful to prepare them for what they are about to do, and why. For many respondents, the repeated sets may seem annoying or redundant. If they don't grasp the reason for completing a dozen or more sets and especially if they perceive no end to the questions, they may abandon the questionnaire or provide data of questionable quality. Thus, we recommend preparing respondents for the repetitive nature of the task. You might use text such as: "We think you'll find this next section both interesting and perhaps a bit challenging. We are going to display a number of tradeoff scenarios. In each scenario, we'll show you five features of restaurants. We'll ask which feature (among this set of five) is the most important and which is least important to you when selecting a restaurant. We need to ask you repeated tradeoff scenarios (involving different features) so that we can learn what is truly important to you in a restaurant." After the fifth set, you might insert a Text/HTML Filler question that thanks the respondent for the work they've done so far, and encourages them to continue. You might use text such as: "Thank you for your effort so far. We know some of these tradeoff questions don't have easy answers. We need to ask you some more questions like these so we can further learn how you evaluate restaurants." After the tenth set, you might state that there are 10 more trade-off questions to go, and then the respondent will be finished with this section. After 15 sets, you might again state that there are just 5 more questions to go. Again, the overall principle is to prepare the respondents for the task, explain why you need to ask what may appear to be repetitive questions, and give them some indication of progress as they move through the questionnaire.

MaxDiff Question Text

MaxDiff lets you control the text that appears within the question. The text positions within a MaxDiff question are as follows:

510

SSI Web v7

Default text for the MaxDiff style question is provided. However, you may wish to ask only the first half of the MaxDiff question (the "best" selection) without asking respondents to identify the "worst" item. Also, MaxDiff supports Method of Paired Comparison (MPC) experiments, where respondents choose between items shown two at a time. Edit the text so that it is appropriate for your situation. This same question text is used for all MaxDiff questions in this exercise. Here are some suggested texts, where you substitute appropriate wording for the red "Xs":

Questionnaires Focused on "Importance"

MaxDiff Questionnaire focused on "Importance": Header 1: Please consider how important different features are when selecting a XXXXXXX to purchase. Header 2: Considering only these X features, which is the <u>Most Important</u> and which is the <u>Least Important</u>? Footer: Click the "Next" button to continue... Best Label: Most<br>Important Worst Label: Least<br>Important MaxDiff or MPC Questionnaire focused on "Importance" (without asking respondent to indicate the "worst"): Header 1: Please consider how important different features are when selecting a XXXXXXX to purchase. Header 2: Considering only these X features, which is the <u>Most Important</u>? Footer: Click the "Next" button to continue... Best Label: <Empty> Worst Label: <Empty>

Questionnaires Focused on "Preference"

MaxDiff Questionnaire focused on "Preference":

MaxDiff Help

511

Header 1: Please consider different features of XXXXXXX that you might purchase. Header 2: Considering only these X features, which is the <u>Most Desirable</u> and which is the <u>Least Desirable</u>? Footer: Click the "Next" button to continue... Best Label: Most<br>Desirable Worst Label: Least<br>Desirable MaxDiff or MPC Questionnaire focused on "Preference" (without asking respondent to indicate the "worst"): Header 1: Please consider different features of XXXXXXX that you might purchase. Header 2: Considering only these X features, which is the <u>Most Desirable</u>? Footer: Click the "Next" button to continue... Best Label: <Empty> Worst Label: <Empty>

To Repeat or Not Repeat the Header Text

Some researchers may find that repeating the Header text each time seems overly redundant. A more concise layout for the MaxDiff tasks can be presented if the Header text doesn't appear each time. If you want to present the Header text initially, but delete it from subsequent questions, you might consider leaving the Header text blank, but then add Text/HTML Filler questions (containing the text you wish to display directly above the MaxDiff questions) directly above the first few MaxDiff tasks.

512

SSI Web v7

4.2.2

Question Format
The Format tab on the MaxDiff Exercise dialog provides many options for formatting your MaxDiff or Method of Paired Comparison (MPC) questions. We suggest you experiment with the settings and click the Preview button to view the effects. There has not been much research conducted on the best layout for MaxDiff questions. We provide different layouts here for your experimentation and use. If these options do not give you the exact look you need for your question, you can customize your own MaxDiff layout using Free Format and your own scripted HTML. One significant methodological question is whether to ask for both "best" and "worst" (when displaying three items or more per set) or whether to omit the "worst" question.

Should We Ask for "Worsts"?

MaxDiff allows researchers to ask for "best" and "worst" choices within subsets of items (set size >=3), or to ask only for "bests." Collecting both bests and worsts contributes more information. However, it has been shown that the scores resulting from best choices may differ (statistically significant differences) from those developed only using worst choices. However, the results tend to be quite similar between bests and worsts. There is some debate among leading academics regarding the statistical properties of "worsts" and whether including both bests and worsts is appropriate. We at Sawtooth Software do not know which approach is best (to ask for worsts or not). Asking only for "bests" is theoretically more sound, but asking for "worsts" seems to offer practical value. We hope that offering flexibility in this software will lead to more experimentation in this area.

MaxDiff Help

513

4.2.3

Developing a List of Items

Choice studies such as those supported by the MaxDiff software, which include, MaxDiff Method of Paired Comparisons (MPC) Choices from sets of three (triples), sets of four (quads), etc. usually involve at least eight items (i.e. features, or attributes). With seven items or fewer, a designed choice experiment may risk overkill. When dealing with so few items and with reasonably sophisticated respondents, perhaps an allocation-based task would be sufficient. Even so, we could imagine conducting a paired comparison exercise with as few as four or five items, if the demands of the project required a simple respondent task (appropriate for people of all educational and cultural backgrounds) and avoidance of the response-style bias typically seen with traditional ratings scales. On the high end, we at Sawtooth Software have been involved in a paired comparison experiment that involved 160 employment-related benefits/conditions. Sample size was in the thousands of respondents, and we pooled information across respondents for estimation. We're certain other researchers have conducted even larger studies than that.

Specifying a List of Items

On the Items tab of the MaxDiff Exercise dialog, you can select an existing list to use in the MaxDiff exercise, type a new list by clicking the Add... button, or copy-and-paste a list of items from another program by clicking the "Paste from Word" button . Depending on your license, you can specify up to 500 items. Items can involve either text, graphics, or a combination of text and graphics.

Recommendations for Item Descriptions:

Item text should be clear and succinct. Anything you can do to help respondents process information more quickly and accurately is helpful. When possible, items should be as specific and actionable as possible. For example, it would probably be more useful to describe a level as "Fuel efficiency increased from 5 liters/hour to 4.5 liters/hour" instead of "Improved fuel efficiency." That said, we realize that such quantitative certainty doesn't make sense for many studies and items. For example, in an image study, we may very well ask respondents to select which picture of an automobile most "Makes me feel successful when I drive it." How one defines "successful" in a concrete or quantitative way is probably not useful to the goals of the study. Items can be made to be multi-level and mutually exclusive. For example, in a study of fastfood restaurant features, rather than ask about "Fast service" generically, you might create three separate items that probe specific levels of fast service: Order and receive food within 3 minutes Order and receive food within 6 minutes Order and receive food within 10 minutes When considering a fast-food restaurant, it would seem rational that all respondents would prefer faster service to slower service. Therefore, it makes no sense to include more than one of these levels within the same set. You should specify prohibitions between these levels (from the Design

514

SSI Web v7 tab) so that they never appear compared with one another. It is also possible during estimation to require that faster levels of service receive a higher score than slower levels of service (by specifying monotonicity constraints). Another example of multi-level, mutually-exclusive levels was shown in the award-winning paper at the 2004 Sawtooth Software Conference by researcher Keith Chrzan. Using a MaxDiff methodology, he studied price sensitivity for options for automobiles. Each option was available at multiple prices, so he could plot the resulting scores as relative demand curves. (Of course, the same option never appeared within the same set at different prices.) You may mix multi-level items with single-level items within the same study. However, we should note that the increased frequency of multi-level attributes in the design might bias respondents to pay more attention to that dimension. But, we are not aware of any research yet in MaxDiff or MPC that substantiates that concern. Graphics are possible! Remember, SSI Web lets you use graphics as list items. It is sometimes helpful to create a "reference" level with monetary meaning. Some researchers have found it helpful to associate one of the levels with a specific monetary equivalent. For example, in studying job-related conditions and benefits, it might be useful to include a level that says: "Receive an immediate $500 bonus." Or, if studying improvements to a product, we might include a level that says: "Receive a $20 off coupon." In both cases, we can associate a specific item score with a specific and immediate monetary gain. The scores for other levels may then be compared to this monetary-based reference level. Also note that it is possible to make the monetary reference point multi-leveled with mutually exclusive levels, such as "$5 off coupon," "$10 off coupon," "$15 off coupon," "$20 off coupon." This provides multiple monetarygrounded reference points for comparing the scores of other items in the study. In other studies, such as those studying potential modifications to existing products, services, or (to be more specific) employment conditions, it might make sense to include a reference level reflecting no change; for example, "No change to current work environment." That way, item scores that might have a negative affect (relative to "no change") can be identified.

Using Constructed Lists in MaxDiff Experiments

Warning: This is an advanced feature that if not used carefully can result in errors in data collection and incorrect model results. Please test your survey thoroughly prior to fielding the study. After collecting data, analyze the data via Counts to ensure that all items from the parent list have been shown a reasonable number of times to the sample of respondents. If some items were never shown to any respondents, they will be marked as having been shown zero times (with an asterisk) and estimation using HB will be incorrect unless these items are omitted from the analysis. SSI Web software can build constructed (dynamic) lists based on answers to previous questions or some other rule you define (such as randomization or always forcing a client's brand onto a list). You may use the constructed list in later questions or even in MaxDiff exercises. When at the items tab on the MaxDiff Exercise dialog, you may select a constructed list rather than the typical pre-defined list used in MaxDiff experiments. This powerful option even allows for the inclusion of respondentsupplied other-specify text as an item in the MaxDiff exercise. The key limitation for using constructed lists is that all constructed lists for respondents must have the same length. SSI Web requires you to use SETLISTLENGTH as the last instruction in the constructed list logic used in a MaxDiff list. The Number of Items per Set specified on the Design tab must match the number of items specified in the SETLISTLENGTH instruction.

MaxDiff Help

515

Warning: the SETLISTLENGTH instruction ensures that the constructed list has a fixed size or length. If the SETLISTLENGTH instruction needs to add members from the parent list it will not add "Other Specify" or "None of the above" type members. This can cause a problem if there are no other members to add. Please thoroughly test the constructed list in the MaxDiff exercise prior to fielding the study. SSI Web uses the design generated from the Design tab, but draws items from the constructed list according to the design by position on the constructed list. The values stored for analysis (both for the design shown to respondents and for the answer given) correspond to the original pre-defined list. Warning: the design report displays frequencies for a design having the same number of items as specified in the SETLISTLENGTH instruction and in a fixed order. However, each respondent's actual items may vary, and therefore the true frequency of occurrences for each parent item and co-occurrences of items may differ significantly from that shown in the design report. When estimating item scores, MaxDiff estimates (via HB) a full set of item scores for all items in the pre-defined list, even if each respondent received only a customized subset of the items. These scores for items not seen take on values near population averages. Depending on your reason for using a constructed list that is a subset of the pre-defined list, this may or may not be a desired outcome.

516

SSI Web v7

4.2.4

Designing the Study

The Design Tab presents a number of options to control your study design: Number of Items (Attributes) In this field, you see the number of items (i.e. levels or features) to include in the exercise. This field is not editable, but is computed by examining the current list you have specified on the Items tab Depending on your license, you can specify up to 500 items on the Items tab. Number of Items per Set (Question) Generally, we recommend displaying either four or five items at a time (per set or question) in MaxDiff questionnaires. However, we do not recommend displaying more than half as many items as there are items in your study. Therefore, if your study has just eight total items, we would not recommend displaying more than four items per set. If you would like to use the Method of Paired Comparisons approach, then specify just two items per set. Research using synthetic data suggests that asking respondents to evaluate more than about five items at a time within each set may not be very useful in MaxDiff studies. The gains in precision of the estimates are minimal when using more than five items at a time per set. The small statistical gains from showing even more items may be offset by respondent fatigue or confusion. Number of Sets (Questions) per Respondent We generally recommend asking as many sets (questions) per respondent such that each item has the opportunity to appear from three to five times per respondent. For example, consider a study with 20 items where we are displaying four items per set. With 20 total sets, we know that each item will be displayed 4 times (assuming the design is perfectly balanced). This leads to the following decision rule and formula: For best results (under the default HB estimation used in MaxDiff), the suggested number of sets is at least: 3K/k where K is the total number of items in the study, and k is the number of items displayed per set. The software will warn you if the number of sets you have requested leads to each item being displayed fewer than 2 times per respondent. These recommendations assume HB estimation (the default method in MaxDiff). Advanced researchers may elect to export the data (File | Export Data | Prepare MaxDiff Data Files (*.cho, etc.)...) for analysis using Sawtooth Software's Latent Class system, or another analytical program of their choice. When pooling data across respondents using Latent Class or aggregate logit, you may be able to achieve reasonable population-level scores using even fewer sets per respondent than these recommendations. If you use relatively few sets per respondent (relative to the number of parameters to estimate), aggregate modeling may be the better option to stabilize and summarize parameters across subgroups or the population.

MaxDiff Help

517

Generate Design
When you are pleased with your settings, click Generate Design to produce a design (which is stored internally). (You can export the design to a text file if you wish using the Export Design... button described at the end of this section.) The design algorithm follows these guidelines: 1. Create a design that features item frequency balance. A balanced design is one in which the one-way frequencies are nearly equivalent (how many times each level appears across the entire design) and two-way frequencies are also nearly equivalent (how many times each pair of items appears within the same set across the entire design). When one-way and two-way frequencies are balanced, this is termed orthogonality. Ensure that each version of the design (respondent questionnaire) features connectivity (meaning that all items are linked either directly or indirectly), unless the Allow Individual Designs Lacking Connectivity option is checked. Without connectivity it becomes very difficult for the default estimation procedure in MaxDiff (HB) to scale the items properly relative to one another for each respondent. After the designs have been generated based on the guidelines in 1 and 2, swap the order of the items within each set so that each item appears approximately an equal number of times in each position. Positional balance reduces or virtually eliminates order bias. Finally, randomize the order of the tasks within each version.

2.

3.

The design process is repeated 1000 separate times, and the replication that demonstrates the best one-way balance is selected. If multiple designs have the same degree of one-way balance, then we select among those designs based on the best two-way balance. If multiple designs have the same degree of one-way and two-way balance, then we select among those designs based on the best positional balance. With small to moderate sized designs and no prohibitions, this usually happens within a few seconds.

Test Design
If you have already generated a design, you may at a later point want to review the properties of the design in terms of: Number of versions used Design seed used One-way and two-way frequencies Positional balance

Click Test Design to produce a report showing the characteristics of the design currently being used for your MaxDiff exercise.

Advanced Settings
Note: it is not necessary to modify the default settings in this section to achieve robust results. The default recommendations and guidelines for MaxDiff should work well for most situations. However, researchers wanting to fine-tune their models or employ a different method for estimating item scores may find the following advanced settings useful.

518

SSI Web v7 Number of Versions Though it is possible to estimate scores relatively efficiently using just a single questionnaire version for all respondents, there is practical benefit to using multiple versions (sometimes called "blocks") of the questionnaire. With multiple versions of the questionnaire, different respondents see different series of questions. Across respondents, this dramatically increases the variation in the way items are combined within sets, which can reduce potential context biases (which are usually modest). The default is 300 questionnaire versions, which ensures a very large overall design and a great deal of combinatorial variation in the sets. Using 300 versions is more than adequate, since beyond a handful of questionnaire versions the further practical benefits of increased variation are barely detectable. Since using SSI Web makes it nearly automatic to produce and use multiple versions of the questionnaire, it makes sense to use a large number. The software allows you to request up to 999 questionnaire versions. The two main take-aways are, 1) it makes sense to use many questionnaire versions, 2) it is not necessary for each respondent to receive a unique questionnaire version. If using the separate MaxDiff designer for conducting paper-and-pencil studies, we recommend using at least four different versions of the questionnaire. You need to find a good balance between difficulty of study administration and design quality. Number of Iterations By default, the MaxDiff designer repeats its algorithm 1000 times and returns the best design found across those passes. The quality of the design is based on the one-way, two-way, and positional balance (in that order of precedence). For many designs, 1000 iterations can be done within a few seconds. We encourage you to increase the number of passes, as it may enable you to find a slightly better design. Up to 999,999 iterations may be specified. Design Seed Specify a value from 1 to 999,999,999. This integer (seed) is used for determining the starting point for the design algorithm. Different seeds will yield different designs, all having approximately the same overall design efficiency. You can try different starting seeds to see if you obtain slightly better designs. Favor Two-Way Balance The MaxDiff designer by default puts primary emphasis on balancing the number of times each item is included (one-way balance), with secondary emphasis on two-way balance (how often each paired combination of levels appears together). If you have specified prohibitions, these two goals are at odds with one another. The software puts more emphasis toward balancing the two-way frequencies if you check this box. Recognize, however, that with prohibitions it may be impossible to achieve near-perfect balance in either the one-way or two-way frequencies. We strongly suggest checking this box when displaying two items at a time within sets (pairwise presentation). The design algorithm will almost always produce one-way frequencies that differ very little from when one-way balance is favored. But, the two-way frequency balance will usually be far superior. Allow Individual Designs Lacking Connectivity Connectivity is the property that the items cannot be divided into two sets wherein no comparison is made between any item in one set and another item from the other. Even if creating choice sets randomly, if respondents receive enough sets, it would be difficult for the items to lack connectivity for any given respondent. However, if you select a design with relatively many items and relatively

MaxDiff Help

519

few tasks per respondent, many individual versions of the questionnaire could lack connectivity. The default for the software is to reject any version (questionnaire for one respondent) that lacks connectivity. But in the case of many items and few sets per respondent, it may become impossible to satisfy connectivity. If you would like to permit versions lacking connectivity (which may be the case if you ask few questions of any one respondent, probably leading to aggregate analysis), then you can check this box. Make sure to specify many questionnaire versions. To ensure stable estimation, advanced users may wish to generate dummy (random) response data, export the data for estimation under aggregate logit, and examine the size of the standard errors to ensure reasonable precision. Prohibitions Occasionally, your study might include multi-level items characterizing degrees or amounts of some feature. For example, in studying features of fast food restaurants, you may wish to include the following three items in your study list: Order and receive food within 3 minutes Order and receive food within 6 minutes Order and receive food within 10 minutes When considering a fast-food restaurant, it would seem rational that all respondents would prefer faster service to slower service. Therefore, it makes no sense to include more than one of these levels within the same set. You should specify prohibitions between these levels by clicking the Prohibitions... button so that they never appear compared with one another. For those familiar with conjoint analysis, it may be heartening to learn that prohibitions are not as damaging in MaxDiff experiments as with conjoint analysis. For example, in a conjoint study where there are 4 prohibitions specified between two 4-level attributes, 4 of the possible 16 combinations (or 25%) are prohibited. Depending on the pattern of prohibited combinations, this could be very detrimental. However, in a typical 20-item study, there are 1/2(20*19) = 190 possible combinations of items (taken two at a time). Prohibiting 4 of these represents only about 2% of the possible combinations. This will have very little impact on the overall results. Please see Robustness of MaxDiff Designs to Prohibitions for more details. When you compute the Generate Design button, if the software returns warnings that the questionnaire versions lack connectivity, this is clear sign that your prohibitions are excessive (after ruling out the possibility that you are not displaying enough sets per respondent).

Importing a Design from an External File

Some advanced researchers may wish to import their own design(s) from an external text file using the Import Design... button. To demonstrate the required file format, consider a MaxDiff exercise with the following specifications: 20 items in the exercise 4 items per set 20 sets per respondent 2 versions of the questionnaire

The design must have the following comma-delimited format (the header row with labels is optional): Version,Set,Item1,Item2,Item3,Item4 1,1,10,1,19,7 1,2,19,17,15,3 1,3,17,10,16,6

520

SSI Web v7 1,4,6,20,12,13 1,5,13,2,9,5 1,6,2,9,13,11 1,7,15,18,1,14 1,8,7,3,20,8 1,9,5,6,18,9 1,10,3,12,2,16 1,11,8,3,10,4 1,12,15,7,5,2 1,13,11,16,14,19 1,14,4,19,11,12 1,15,14,11,17,8 1,16,20,15,13,4 1,17,9,1,17,18 1,18,18,8,16,6 1,19,1,5,12,20 1,20,14,4,7,10 2,1,4,11,6,15 2,2,10,13,14,18 2,3,20,10,9,1 2,4,11,20,8,5 2,5,16,13,5,7 2,6,12,19,7,17 2,7,12,14,8,17 2,8,13,18,3,2 2,9,18,7,4,11 2,10,17,4,15,16 2,11,5,14,10,15 2,12,19,2,3,8 2,13,2,15,17,20 2,14,16,12,5,9 2,15,4,2,19,10 2,16,6,11,14,3 2,17,8,7,4,6 2,18,3,1,11,19 2,19,1,9,16,20 2,20,12,9,2,14 When you import a design from an external file, the designs are stored within the system as if the MaxDiff designer had generated them. You can click Test Design from the Design tab to compute one-way, two-way, and positional frequencies for the design you supplied through the import process. During data collection, each new respondent entering the survey is assigned the next questionnaire version in the design. Once all questionnaire versions have been assigned to respondents, the next respondent receives the first design, etc.

Exporting the Current Design

Some users may wish to examine their designs or use the design details to create paper-andpencil questionnaires. You can export the current design to a comma-delimited format with the layout as described in the previous section by clicking Export Design....

MaxDiff Help

521

4.3

Analyzing MaxDiff Data

MaxDiff data involve choices: respondents are shown items and asked to choose among them. This kind of data is very useful in marketing and social research applications. The analysis involves observing the probabilities of choice for the items. More detailed explanation than given here is available within the "MaxDiff Technical Paper," available on our website, www.sawtoothsoftware.com. Two main types of analysis are offered in the MaxDiff System: Counting Analysis Individual-Level Score Estimation

522

SSI Web v7

4.3.1

Counting Analysis
Through the Admin Module
When your survey is running live on the Web (or over an intranet), you can log into the Admin Module to check the progress of your study, view data, run simple tabulations, download data, and other operations. When you log into the Data Management Module and click Marginals, you can perform a simple counting analysis for your MaxDiff exercises. This often gives you a good "top line" view of the general preferences of the respondents, but generally should not be used for final reporting. This counting procedure simply counts how many times each item was picked best or worst, and reports this as a raw count and as a percentage of total choices. Here is an example:

If your experiment is perfectly balanced (each item displayed an equal number of times, and each pair of items shown together an equal number of times, etc.) then these percentages reflect a fair and accurate picture of these items' relative importances or preferences. However, if the experiment is unbalanced (for example, due to prohibitions), then these scores may not be quite accurate. You'll want to rely on the more sophisticated method of analyzing item scores, found under Analysis | MaxDiff Scores....

Counting Analysis
We previously described a method of simply counting how many times an item was chosen best or worst through the Admin Module. That approach works well for a top-line view of the data, as long as

MaxDiff Help

523

each item is displayed about an equal number of times in a balanced experiment. If some items appeared significantly fewer or more times than others, then these items would have had an artificial advantage and a greater opportunity to be chosen. A more accurate method of counting analysis takes into account how often each item was available for choice. We divide the number of times each item was chosen by the number of times it was displayed to the respondents (and available for choice). Intuitively, we are asking: "When an item is available within a set, what is the probability that respondents select it as best or worst?" To obtain this analysis, you first must have downloaded the data from your web server. Then, from the SSI Web system, click Analysis | MaxDiff Counts.... Here is an example of the output:

These "counts proportions" do not add to 100% (since all items were not displayed in each set). But if you would like, you may copy-and-paste the values into Excel and re-scale them to sum to 100. The interpretation is the same irrespective of the scale. If your experiment is perfectly balanced (each item displayed an equal number of times, and each pair of items shown together an equal number of times, etc.) then the results should closely resemble (but with "flatter" scaling) what is produced by the most sophisticated analysis method provided by MaxDiff, described in the next section.

524

SSI Web v7

4.3.2

Individual-Level Score Estimation

MaxDiff uses a sophisticated estimation technique to produce scores for each respondent on each item. You do not need to have any special training in statistics to consistently achieve good results. Item scores for each respondent may be saved to an Excel-compatible file. These scores are very easy to interpret and present to others. The statistical technique used to estimate scores for each respondent is called HB (hierarchical Bayes estimation). This is a well-documented and trusted advanced statistical technique in the market research industry. It is beyond the scope of this documentation to explain HB, but information is available in technical papers on Sawtooth Software's website at www.sawtoothsoftware.com. MaxDiff questionnaires provide typically too little information to stabilize scores for any one individual taken by itself. The HB estimation routine is able to stabilize the estimates for each individual by "borrowing" information from the body of respondents in the same data set. Therefore, by adding new respondents to the data (and re-computing results), the resulting scores for existing respondents will change a little. And, you cannot obtain stable results unless a large enough pool of respondents is available for HB estimation. Typical sample sizes start at one hundred or more respondents, though most projects will probably involve from 300 to 1000 respondents. If using HB analysis, it is also important that each respondent have provided enough data (relative to the number of items in your study) to stabilize the score estimates and to avoid overfitting. For recommendations regarding proper questionnaire length, see the section entitled Designing the Study. Warning: Because the information from MaxDiff questionnaires is relatively "sparse," it requires a reasonable number of respondents to obtain stable results. If you run the MaxDiff Scores procedure with tiny sample sizes (such as a twenty or fewer respondents), you may see very unstable (and extreme) scores for individuals. Note: Individual-level score estimation in Max/Diff Web does not perform all the types of analysis that some analysts may desire. As an example, some analysts may take the rescaled or raw scores into another statistical package and perform additional analysis such as histograms to view the distribution of scores across individuals.

Settings and Options

When you click Analysis | MaxDiff Scores, a simple dialog is displayed. The default settings in MaxDiff Scores estimation should work well for almost every application. If you click Settings..., you can review (or modify) the settings that will be used in your score computation. Click the Show Advanced Settings>> button to reveal additional advanced settings within this same dialog.

MaxDiff Help

525

You should choose a Respondent Numbering method consistent with other data exports you may conduct under File | Export Data (so you can later align/merge your MaxDiff scores for respondents with other segmentation data). Number of Iterations before Using Results The estimation routine that MaxDiff employs (HB) uses many thousands of iterations to find stable scores prior to actually "using" (saving) the results for each respondent. 20,000 iterations is the default, and should give you a healthy margin of safety, in the sense that the scores should have every opportunity to stabilize (assuming appropriate design and sample size). You may change the number of iterations if you wish. Constraints Sometimes you know beforehand that some levels rationally should be preferred to others. For example, perhaps you included three items in your list related to end-of-year bonus: Receive a $500 end-of-year bonus Receive a $750 end-of-year bonus Receive a $1,000 end-of-year bonus Every rational respondent should prefer higher bonus to lower bonus, and you can inform the estimation routine of this preference. Click the Edit button under the Constraints section if you would like to specify known preference orders and constrain the estimated scores to conform to these expectations. The method that MaxDiff uses to constrain the scores is called "simultaneous tying" and it is described in the CBC/HB Technical Paper on our website.

526

SSI Web v7

The Computation Run

When you click Analysis | MaxDiff Scores, a simple dialog is displayed. You can click the Prepare for Scores Computation button to prepare the data file and compute the scores using your current settings. The data are prepared and you are shown a summary report of the number of respondents included in the run:
** Data Summary ** Total respondents = 300 Total 'Best' choices = 4500 Total 'Worst' choices = 4500 Average sets per respondent = 15.0 'Best' (1) (2) (3) (4) responses by position: 25.64% 25.35% 24.89% 24.12%

'Worst' responses by position: (1) 24.34% (2) 24.68% (3) 25.25% (4) 25.73%

Click the Compute Scores button to proceed with HB estimation. Depending on the size of your data set, it can take between fifteen minutes to a few hours or more to run! HB uses an iterative method that requires many thousands of iterations to stabilize and complete the run. We have chosen defaults to govern the HB run that should work well for most every situation. Advanced users can modify the settings if they desire. While HB runs, a graphic is shown that characterizes how well the estimated scores are stabilizing over the planned thousands of iterations. The graphic plots the estimated scores at each iteration of the process. The estimates first all start at 0, and then trend toward their final values. Once the process has "converged" the estimates will tend to randomly "wobble" up and down, but there should be no noticeable trend remaining. Below is an example:

MaxDiff Help

527

The graphic is divided into two regions: a gray region (iterations prior to assumed convergence) and a white region (iterations after assumed convergence). Once the iterations cross into the white region, you should look for no noticeable trend in the data. The plotted scores should just continue to wobble randomly up and down, with no strong and sustained move in any specific direction. The smaller the sample size or the more items included, the larger the wobble. If there is noticeable trend after the plotted values are in the white region of the graphic, then you can repeat the analysis using more iterations. When the estimation finishes, the graphical plot of the estimation remains on the screen. Click the button stating: The estimation is complete. Click here to close. When you click the "close" button, a report is given which resembles a spreadsheet, with multiple "sheets" that can be accessed by clicking the tabs at the bottom of the dialog labeled "Summary," "Rescaled Scores," and "Raw Scores." The first (Summary) sheet contains a summary of the final scores, upper and lower 95% confidence bounds, as well as other information regarding the settings for your run. The second (Rescaled Scores) and third (Raw Scores) sheets contain the scores for each respondent (presented in two different formats), including a fit statistic. Rescaled (Probability) Scores These individual-level item scores are positive values summing to 100 that reflect the likelihood of items being chosen within the questionnaire. Most researchers will probably use this scaling procedure, as it is easiest to interpret and present to others. This approach has the valuable property of ratio-scaling. That is to say, an item with a score of 20 is twice as important (or preferred) as an item with a score of 10. Click here for more details regarding the rescaling procedure. Raw Scores These are weights that directly follow from the MNL (multinomial logit) procedure employed within the HB engine. The items can have positive or negative weights and are zerocentered (the "average" item has a weight of 0). These weights are on an interval scale, which does not support ratio operations. In other words, you cannot state that an item with a score of 2.0 is twice as important (or preferred) as an item with a score of 1.0. Advanced analysts may choose to use these raw scores. Click the Save As... button to save this report. Two formats are offered: Microsoft Excel format --an Excel Workbook .XLS file is saved with multiple sheets (one for each of the three sheets displayed in this dialog) .CSV (Comma Delimited) --only the current displayed sheet is saved to the file. To save all information from the three sheets to .CSV format, select each sheet separately and save each to a unique .CSV file. (Note: the rank-order position of items based on average scores can differ between the raw and rescaled reports. However, the changes in rank-order position are seen generally when there is little absolute difference in scores between adjacent items. The overall interpretation of the results and main conclusions should not differ whether using the raw or rescaled reports.)

Advanced Settings
Some advanced users may want to click Show Advanced Settings to access other settings that govern the HB estimation. However, we stress that most users will not need to investigate this area. (For more information about the controls used in this area, please refer to the CBC/HB Technical Paper, available for downloading from our website in the technical papers library).

528

SSI Web v7

Number of Draws to Be Used for Each Respondent Each iteration that HB completes results in an estimate of each respondent's scores called a "draw." From one draw to the next, the scores can vary by a random perturbation. One could plot the distribution over many draws and we'd generally find that the draws are distributed normally. MaxDiff averages these draws for each respondent across as many draws as indicated in this field, and reports that average as the final raw score for each item. Theoretically, the more draws used per respondent, the more precise the estimates. However, you will find that 10,000 draws already provides a high degree of precision. Prior Degrees of Freedom This value is the additional degrees of freedom for the prior covariance matrix (not including the # parameters to be estimated), and can be set from 2 to 100000. The higher the value, the greater the influence of the prior variance and more data are needed to change that prior. The scaling for degrees of freedom is relative to the sample size. If you use 50 and you only have 100 subjects, then the prior will have a big impact on the results. If you have 1000 subjects, you will get about the same result if you use a prior of 5 or 50. As an example of an extreme case, with 100 respondents and a prior variance of 0.1 with prior degrees of freedom set to the number of parameters estimated plus 50, each respondent's resulting scores will vary relatively little from the population means. We urge users to be careful when setting the prior degrees of freedom, as large values (relative to sample size) can make the prior exert considerable influence on the results. Prior Variance The default is 1 for the prior variance for each raw score, but users can modify this value. You can specify any value from 0.1 to 100. Increasing the prior variance tends to place more weight on fitting each individual's data, and places less emphasis on "borrowing" information from the population parameters. The resulting posterior estimates are relatively insensitive to the prior variance, except 1) when there is very little information available within the unit of analysis relative to the number of estimated parameters, and 2) the prior degrees of freedom for the covariance matrix (described above) is relatively large.

MaxDiff Help

529

Random Starting Seed Any computer algorithm that uses random numbers needs a starting "seed" value. You may set this as an integer from 0 to 32000. When using different random seeds, the final scores will vary, but insignificantly, assuming convergence has been reached and many draws have been used. If you specify a "0," this uses the system's clock as the starting seed. If you want to be able to repeat results and achieve the same answer, then you should use a specific integer greater than 0. Sets to Include Clicking Filter under the Sets to Include area accesses a dialog in which you can specify which sets to include. Most users will utilize all sets within analysis. However, there may be instances in which advanced users wish to use only a subset of the full data. Respondents to Include Clicking Filter under the Respondents to Include area allows you to choose whether to include All Respondents, Qualified/Completes Only, or Disqualified and Incompletes Only in the analysis. Responses to Include If you asked MaxDiff questions that included both a selection of "best" and "worst," you can select which response types to use in analysis. Researchers have noted that there can be statistically significant differences in scores developed from bests versus worsts. This control lets you investigate these issues if you'd like, including the ability to use answers from bests only in computing item scores.

530

SSI Web v7

4.3.3

Identifying "Bad" Respondents

Fit Statistic and Identifying Random Responders
MaxDiff displays an average "fit statistic" to the screen during HB estimation and also writes to file the individualized fit statistic along with each respondent's scores on the items. This fit statistic ranges from a low of 0 to a high of 1000. It characterizes the internal consistency for each respondent. In technical terms, the fit statistic is the Root Likelihood (RLH) times 1000. It is natural to ask what minimum fit statistic respondents should have to be able to recognize more thoughtful responses from purely random data. For example, some "professional" respondents complete multiple surveys each day, sometimes even using automated computer programs that fill in random answers! The fit statistic can help you determine if a respondent has provided purely random answers and should be discarded. The fit statistic largely depends on how many items were shown in each set. If four items are shown in each set, then any random set of scores should be able to predict respondents' answers correctly with 25% likelihood (fit statistic=250). If two items are shown per set (paired comparison approach), then the likelihood given random data is 50% (fit statistic=500). Thus we can generalize that the fit statistic should be at a minimum 1/c x 1000, where c is the number of items displayed per set. We should hope that respondents don't provide random answers and that the estimated scores should perform considerably better than chance. However, the score estimation algorithm used in MaxDiff (HB) attempts to fit the respondent's choices, even if they are just random. Thus, the actual fit we observe even from random data is mostly above the chance rate. The table below displays a suggested minimum fit statistic assuming we want to achieve 95% correct classification of random responders. In developing this table, we assumed a 20-item study wherein respondents are shown each item either two or four times across all sets. Suggested Minimum Fit Statistic to Identify Random Responders with 95% Correct Classification Items per Set Suggested Minimum Fit. Each Item Shown Four Times to Each Respondent 574 371 282 227 191 Suggested Minimum Fit. Each Item Shown Twice to Each Respondent 634 402 305 247 208

2 3 4 5 6

Technical Notes: We simulated 1000 respondents answering randomly to each questionnaire, then estimated the scores using HB: prior variance=2, d.f.=5. Both "best" and "worst" answers were simulated for each respondent (with the exception of the 2 items case). For each simulated data set, we sorted the respondents' fit statistics from highest to lowest and recorded the 95% percentile fit. If asking only "bests," because the amount of information for each respondent is reduced, the Suggested Minimum Fit would be higher.

MaxDiff Help

531

4.4

Customized MaxDiff Questions with "Free Format"

Warning: This is an advanced feature that if not executed properly can result in errors in design, data collection, and analysis. It requires careful, manual checking of Free Format HTML syntax. Some researchers may encounter situations in which they need to customize the look and layout of their MaxDiff or MPC exercises beyond that which is provided by the MaxDiff templates within the SSI Web System. The instructions provided here will help you customize the look of your MaxDiff exercise. To implement your own question format, you'll use the Free Format question type. You'll insert a Free Format question directly above each standard MaxDiff question. Within these Free Format questions, you'll be able to display the items to be asked in each MaxDiff question (according to the experimental plan) in most any way you desire. Then, you'll use a special programming instruction to suppress the standard MaxDiff questions so they do not display. The steps are as follows: 1) Add a Free Format question directly above its corresponding MaxDiff question. Assuming your standard MaxDiff exercises were named "MaxDiff" and your Free Format variables were named FFMD1, FFMD2, etc. you would specify: FFMD1 MaxDiff_1 <page break-----------------> FFMD2 MaxDiff_2 <page break-----------------> etc. Please refer to the documentation on Free Format if you are not familiar with adding Free Format questions supporting radio buttons within your survey. If you are applying Styles to your survey, you may wish to refer to the styles documentation to see how to make your Free Format MaxDiff questions use the style settings that are applied to standard MaxDiff questions. 2) In each Free Format question, name your radio button variables "_b" for best and "_w" for worst. For example: FFMD1_b FFMD1_w FFMD2_b FFMD2_w etc. If you are only asking for a single response within your MaxDiff questionnaire (not asking for "worst"), then only add a variable capturing the single response as "_b". 3) In your Free Format HTML use the system functions described below: To return the label (the text or graphic you specified) for the item to display: MaxDiffDesignLabel(ExerciseName_MaxDiffQuestionName, Position#) For example:

532

SSI Web v7 MaxDiffDesignLabel(Myexercise_MaxDiff_1, 3) returns the text for the item label for the item in the 3rd position of the question MaxDiff_1 (e.g. "Red") To return the value for the item displayed: MaxDiffDesignValue(ExerciseName_MaxDiffQuestionName, Position#) For example: MaxDiffDesignValue(Myexercise_MaxDiff_1, 3) returns the item value for the item shown in the 3rd position within question MaxDiff_1 (e.g. "5" where red is the 5th item in the parent list) 4) When specifying your Free Format radio text and value, you'll need to ensure that you associate the correct values and labels with the correct radio buttons. For example, the instructions below display a radio button before the first item text for the third MaxDiff question in an exercise. The first radio button collects the "best" answer, which is the item value selected best (you must store the item value for analysis to function properly). The second radio collects the "worst" answer, which is the item value selected worst. (Note: we have aligned the text in this example to make it easier to check that the function calls are indexed properly with the correct variable names.) <input name="FFMD3_b" type="radio" value="[%MaxDiffDesignValue(ABC_MaxDiff_3, 1)%]"> [%MaxDiffDesignLabel(ABC_MaxDiff_3, 1)%] <input name="FFMD3_w" type="radio" value="[%MaxDiffDesignValue(ABC_MaxDiff_3, 1)%]"> To display the second item for this question and capture the appropriate answers for best and worst questions, you'd specify: <input name="FFMD3_b" type="radio" value="[%MaxDiffDesignValue(ABC_MaxDiff_3, 2)%]"> [%MaxDiffDesignLabel(ABC_MaxDiff_3, 2)%] <input name="FFMD3_w" type="radio" value="[%MaxDiffDesignValue(ABC_MaxDiff_3, 2)%]"> etc. for as many items as are displayed in the question. 5) It is critical that you very carefully check that you have incremented your variables and function calls properly: that the correct information is displaying on the screen and the correct information is being saved when the respondent clicks any button on the screen. It is useful to view the original MaxDiff question and your custom Free Format question together via the Local Test Server to make sure that the displayed text matches the design for each question. Next, you must verify that the correct item numbers are being saved within the Free Format variables corresponding with the buttons you choose. You can do this by checking your original HTML coding, or by viewing what is stored in the respondent record through the Admin Module. The value saved must correspond to the label shown. We suggest you just create a single Free Format question and test its behavior first prior to copying and indexing the HTML for each MaxDiff question. 6) To remove the MaxDiff question so that it does not display to respondents, enter the following HTML comment into the Header1 of the MaxDiff question:

MaxDiff Help

533

<!-SSI_Comment: CUSTOM_MAXDIFF_DESIGN Free_Format_Base_Name: FFMD Verification: ON --> This comment suppresses the original MaxDiff question so it does not display to the screen. It also instructs SSI Web to save the data from the Free Format question, specified by Free_Format_Base_Name, in place of the data for the original MaxDiff question. Make sure that you have the original MaxDiff questions and your corresponding custom Free Format questions on the same page. If your Free_Format_Base_Name: is "FFMD" then your Free Format questions should be named: FFMD1 FFMD2 etc. By placing "Verification: ON" within the HTML comment, it instructs SSI Web to use the original MaxDiff verification for your Free Format question (ensuring that respondents do not mark the same item as both "best" and "worst"). To turn off verification use the following: Verification: OFF

534

SSI Web v7

4.5

Computing Scores On-The-Fly

If you want to compute individual-level MaxDiff scores on-the-fly (realtime during the interview), you can use the following functions: MaxDiffRankAttLabel(ExerciseName, Rank#) MaxDiffRankAttValue(ExerciseName, Rank#) MaxDiffScore(ExerciseName, ItemIndex) These functions allow you to return (e.g. show to the screen, or use in calculations or skip patterns) the label and score for each of the items in your MaxDiff exercise. You can also return the label and/or score for the top, second, third, etc. item in terms of importance/preference. You can use the functions above together with list-building instructions (e.g. ADD) to force the top n items onto another list. If the list used in ExerciseName is a constructed list, then the ItemIndex refers to items on the original parent list. If you want ItemIndex to refer to the item position in the constructed list, then you can substitute ListValue(listname, 1) in place of ItemIndex. For example, if you want to reference the score for the first item on your constructed list (named q1list), you would specify MaxDiffScore(ExerciseName, ListValue(q1list, 1)). These functions use the individual-level logit method to compute positive scores summing to 100. The scores developed in this way only rely on data from the current respondent (the scores cannot benefit from information of other respondents, as final score computation using HB does). As a result, the scores will have less stability and precision than if computed across a sample of respondents under HB (hierarchical Bayes). We recommend that each item be shown at least 4 times per respondent for reasonable individuallevel score estimation using these functions. For more detail on the individual-logit method and comparisons to HB estimation for MaxDiff, see the white paper entitled "MaxDiff Analysis: Simple Counting, Individual-Level Logit, and HB" available for downloading from our Technical Papers library at www.sawtoothsoftware.com. Technical Details: Because individual-level logit includes sparse data that may not converge under logit, we break out if the log-likelihood fails to improve by 0.03 in an iteration. We also break out if the range in the logit scores is greater than 15. For all practical purposes, these break-out rules lead to ample precision in the estimated scores, and keep the scores reasonably scaled. The computation is nearly instantaneous, and should not burden servers nor result in a wait time for respondents. The scores returned using the functions above are normalized, positive scores that sum to 100. The normalization is done as follows: 1. The raw logit scores for all items are first zero-centered by subtracting off their mean. 2. The scores from step 1 are transformed by e^u/(1+e^u); where e is the exponential constant. 3. The scores from step 2 are normalized to sum to 100.

MaxDiff Help

535

4.6

Paper-and-Pencil Studies
Overview
SSI Web lets you run your MaxDiff studies via Web, over PCs (CAPI installation), or as paper-andpencil questionnaires. The process of running paper-based MaxDiff studies is very similar to computer-based studies. You should set up your levels and compose your MaxDiff tasks in the same manner as if you were going to run a computerized study. However, you should probably choose plain black (text) on white (background) for questionnaire colors, unless you plan to print the questionnaires in color. For MaxDiff, you will not have so many different versions of the questionnaire as with computerized interviewing, because that becomes difficult to manage on paper. Rather than having so many possible versions of the questionnaire , you will create just a few versions (design blocks) of the questionnaire and assign respondents (randomly) to take one of the different questionnaire versions. With paper-based experiments, one typically uses just a few versions of the questionnaire since each additional questionnaire version adds more administrative hassle. Researchers often employ between three to eight unique versions of the questionnaire when conducting paper-and-pencil experiments. One sacrifices a little in terms of design efficiency and control for context and order biases by using just a few versions (relative to the 300 or more versions often employed in computerized interviewing), but the practical impact on quality of results is usually relatively small. It is not necessary that the same number of respondents complete each version of the questionnaire, though it is helpful to achieve approximate balance.

Steps for Paper-Based Studies

Generate the Questionnaire: After you have created the questionnaire and generated the design, save the results to a file using Field | Create Paper & Pencil Interviews. Your questionnaire is saved in HTML format to a STUDYNAME_EXERCISENAME_V#.htm file, where V# is the version number. That file is placed in a /Paper-and-Pencil/ExerciseName folder within your study folder. If you would prefer to deal with the item #s in a .csv data file rather than formatted text within an HTML document, you can export the design by clicking the Export Design... button from the Design tab of our MaxDiff Exercise dialog. A file named STUDYNAME_EXERCISENAME_Design.csv is saved to your study folder. Collect the Data: Make sure when fielding the questionnaire that you are able to match the different questionnaire versions correctly with respondent answers. You should also try to randomly distribute the questionnaire versions among respondents, so that roughly equal numbers of respondents complete each version. Prepare the Answers: After you have collected the data, you prepare a .csv file containing the respondent answers. This file follows a required .csv (comma separate values) format, with specific labels in the first row of the file. To see that format, prepare a template for yourself by clicking Field | Create Accumulated Data Template File.... A file named Accumulated Data.csv is saved to your study folder. Use this template to arrange your data, and make sure to use the prescribed header row

536

SSI Web v7 within your data file. The layout is: Column 1: Respondent# Column 2: Version# Column 3: MaxDiff answer #1, etc. The MaxDiff answers can be recorded either as position numbers in the questionnaire or item numbers selected. When you accumulate data, you are asked to specify which type of coding you are using. Any missing data are coded as blank (meaning, if you viewed the data file using Excel, the cell would be empty; if you viewed the file with a text editor, the missing value would look like two consecutive commas with no text in between). Accumulate Respondent Answers: Click Field | Accumulate Paper & Pencil Data... and browse to the .csv file containing your respondent data.

MaxDiff Help

537

4.7

MaxDiff Analysis Using CBC/HB v5.x

In this section, we assume you own the CBC/HB v5.x software and have prepared a .CHO file containing respondent data. You obtained the .CHO file by clicking File | Export Data | Prepare MaxDiff Data Files (*.cho, etc.) from the SSI Web interface. When you prepare the .CHO file, a number of other files are created that (although not required) are useful for setting up your CBC/HB project: ExerciseName.ATT ExerciseName.EFF ExerciseName.VAL ExerciseName.MTRX (Attribute file) (Effects file) (Values file) (Default Prior Covariance Matrix)

Open the CBC/HB v5.x software. Browse to and open the .CHO file. CBC/HB v5 will recognize the presence of the .ATT, EFF, .VAL, and .MTRX files when placed in the same folder as the .CHO file. It will prompt you to import these files as well. Once you have imported the data (along with the optional files), note that there will be k-1 total attributes in the study (to avoid linear dependency, meaning that one of the columns can be perfectly predicted by a linear combination of other columns), representing your k total items in the MaxDiff design. The final item will be constrained to have a utility of zero, and the remaining items are estimated with respect to the final item's score. We should note that when using HB to estimate parameters for many items under dummy coding, the estimates of the parameters (relative to the reference "0" level) can sometimes be distorted downward, often quite severely when there are many items in the study and when the number of questions asked of any one respondent is relatively few. (This distortion of course makes it appear as if the "omitted" item's estimate is "too high" relative to the others.) To avoid this difficulty, we automatically export the ExerciseName.MTRX file containing an appropriate prior covariance matrix. By default, we assume a prior variance of 2. The proper prior covariance matrix for use with MaxDiff exercises is a (k-1) x (k-1) matrix of values, where k is equal to the total items in your MaxDiff study. Make sure the custom prior covariances file (the information from .mtrx) is actually being used by checking that the Use a Custom Prior Covariance Matrix box is check on the Advanced tab. If you examine the .MTRX file (with a text editor), you will find a matrix composed of "2"s across the main diagonal, and "1"s in the off-diagonal positions (assuming you are using the default prior variance specification of 1.0), such as: 2 1 1 1 . . . 1 2 1 1 . . . 1 1 2 1 . . . 1 1 1 2 . . . . . . . . . . . . . . . . . . . . . . . .

If you had requested a different prior variance, the values in this matrix are multiplied by the desired variance constant. When you have finished these steps, click Estimate Parameters Now... from the Home tab. Weights for the scores are written to the .HBU and .CSV files. Remember, the weight of the reference value (the last item in your design that was omitted) for each respondent is 0 (you'll need to fill this in when done). The other items are estimated with respect to that reference item. We strongly encourage you to consider using covariates in your HB run, employing questions asked elsewhere in the questionnaire that are predictive of differences in item scores between respondent

538

SSI Web v7 segments.

Transforming Weights to 0-100 Scale To convert the raw weights to the 0-100 point scale, first zero-center the weights by subtracting the mean weight for each respondent from each respondent's weights. Then, perform the following transformation for each zero-centered item score for each respondent:

Where: Ui = zero-centered raw logit weight for item i eUi is equivalent to taking the antilog of Ui. In Excel, use the formula =EXP(Ui) a = Number of items shown per set Finally, as a convenience, we rescale the transformed item scores by a constant multiplier so that they sum to 100. The logic behind this transformation is as follows: We are interested in transforming raw scores (developed under the logit rule) to probabilities true to the original data generation process (the counts). If respondents saw 4 items at a time in each MaxDiff set, then the raw logit weights are developed consistent with the logit rule and the data generation process. Stated another way, the scaling of the weights will be consistent within the context (and assumed error level) of choices from quads. Therefore, if an item has a raw weight of 2.0, then we expect that the likelihood of it being picked within the context of a representative choice set involving 4 items is (according to the logit rule):

Since we are using zero-centered raw utilities, the expected utility for the competing three items within the set would each be 0. Since e0 = 1, the appropriate constant to add to the denominator of the rescaling equation above is the number of alternatives minus 1. Note: Prior to SSI Web v6.2, we used a different rescaling procedure that assumed that the scaling of the logit weights was consistent with a data generation process of respondents picking one item from t items, where t was the total number of items in the experiment. However, upon further review, this assumption provided scaling that was too extreme when considering the actual data generation process (where items are being chosen from subsets of items much smaller than t). The benefit of the new scaling procedure is that you will note fewer inconsistencies between the summary raw scores and probability-scaled scores for the population. The new probability-scaled estimates for the population will be more stable.

MaxDiff Help

539

4.8

MaxDiff Analysis Using Latent Class v4

In this section, we assume you own the standalone Latent Class v4.0 (or v2.5) software and have prepared a .CHO file containing respondent data. You might have obtained the .CHO file by clicking File | Export Data | Prepare MaxDiff Data Files (*.cho, etc.) from the SSI Web interface. Or, if using the paper-and-pencil data collection procedure, you would have generated a .CHO file using the MaxDiff Experiment Designer software. When you export a .CHO file from the SSI Web system, the following additional files are created for use in Latent Class v4: ExerciseName.ATT ExerciseName.EFF ExerciseName.VAL (Attribute file) (Effects file) (Values file)

These files are all described in the Latent Class software documentationbut they have been formatted in special ways to accommodate MaxDiff analysis. (Note, Latent Class v4 is required for MaxDiff estimation. If you are licensed with Latent Class v3.x, you can obtain a free copy of v4 from Sawtooth Software) For your information, the formats for the files automatically created by SSI Web are described below: ExerciseName.ATT is also a text-only file, containing labels for the items (attributes) in your MaxDiff study (remember, if you have 8 items in total, you'll be using n-1, or 7 labels in the .ATT file). To conform to the rule in standard Latent Class that each attribute have at least two levels, we represent each MaxDiff item as a two-level attribute. For example, your ExerciseName.ATT file should contain: Red (use a single blank line here to separate the levels) Red (use two blank lines here to separate attributes, etc.) Green Green Blue Blue ExerciseName.EFF is a text-only file, containing the attribute numbers to be used in analysis. If you have 8 items in total in your MaxDiff study, you need to follow a dummy-coding procedure in which you estimate n-1 or 7 parameters (the utility of the omitted parameter is set to zero, and all other utilities are scaled relative to the omitted item). Assuming 8 items, specify the following on the first line of your ExerciseName.EFF file: #1 #2 #3 #4 #5 #6 #7 The "pound notation" is specialized for use when the analyst wishes to override the typical effectscoding procedure that Latent Class v2 uses for conjoint data. ExerciseName.VAL is also required whenever using "pound notation," and is a text-only file with two lines per item in your study. Again, assuming an 8 item study, you are only estimating 7 parameters, and your ExerciseName.VAL file should have 14 total lines (2 for each of the 7 parameters). The first six lines look like:

540

SSI Web v7 "Attribute 1 Label" 1 2 "Attribute 2 Label" 1 2 "Attribute 3 Label" 1 2 (etc.) Although we are only estimating a single parameter for each attribute, we list the level values as "1" and "2." This is just to satisfy the Latent Class software (which was designed for conjoint experiments, where every attribute has at least two levels), and to ensure that the scaling of data in the text report that we produce is proper.

Running Latent Class Start Latent Class v4 by clicking Start + Programs + Sawtooth Software + Sawtooth Software Latent Class. Click File + Open. Browse to the .CHO file created by the MaxDiff Designer software. On the Attribute Information tab, modify all attributes to have "User-specified coding." (Note that there will be k-1 total attributes in the study, representing your k total items in the MaxDiff design.) Click Estimate Parameters Now... from the Home tab. Utility values are written to the .LCU and .CSV files. Remember, the utility of the omitted value for each group is 0, and the other items are measured with respect to that omitted item.

Transforming Weights to 0-100 Scale To convert the raw weights to the 0-100 point scale, first zero-center the weights by subtracting the mean weight for each segment from each segment's weights. Then, perform the following transformation for each zero-centered item score for each segment:

Where: Ui = zero-centered raw logit weight for item i eUi is equivalent to taking the antilog of Ui. In Excel, use the formula =EXP(Ui) a = Number of items shown per set Finally, as a convenience, we rescale the transformed item scores by a constant multiplier so that they sum to 100. The logic behind this transformation is as follows: We are interested in transforming raw scores (developed under the logit rule) to probabilities true to the original data generation process (the counts). If respondents saw 4 items at a time in each MaxDiff set, then the raw logit weights are developed consistent with the logit rule and the data generation process. Stated another way, the scaling of the weights will be consistent within the context (and assumed error level) of choices from quads. Therefore, if an item has a raw weight of 2.0, then we expect that the likelihood of it being picked within the context of a representative choice set involving 4 items is (according to the logit rule):

MaxDiff Help

541

Since we are using zero-centered raw utilities, the expected utility for the competing three items within the set would each be 0.0. Since e0 = 1, the appropriate constant to add to the denominator of the rescaling equation above is the number of alternatives minus 1. Note: Prior to SSI Web v6.2, we used a different rescaling procedure that assumed that the scaling of the logit weights was consistent with a data generation process of respondents picking one item from t items, where t was the total number of items in the experiment. However, upon further review, this assumption provided scaling that was too extreme when considering the actual data generation process (where items are being chosen from subsets of items much smaller than t). The benefit of the new scaling procedure is that you will note fewer inconsistencies between the summary raw scores and probability-scaled scores for the population. The new probability-scaled estimates for the population will be more stable.

542

SSI Web v7

4.9

Robustness of MaxDiff Designs to Prohibitions

This section investigates the robustness of MaxDiff designs to prohibitions using Sawtooth Software's MaxDiff System and simulated (synthetic) respondent data.

The Case for Prohibitions

MaxDiff questionnaires present respondents with typically four to six items at a time and ask which item is "best" and which is "worst." In practice, enough sets are shown to the respondent such that each item occurs about two to five times. (In some applications, the "worst" question may be skipped altogether.) Sawtooth Software's MaxDiff component within the SSI Web system helps researchers design, field, and analyze MaxDiff studies very rapidly. One advanced option in the software is the ability to specify that certain levels should never be paired within the same set. For example, in a fast-food restaurant study, we might wish to study the importance of the length of time to order and receive food. We might use three items in our MaxDiff design related to this: Order and receive food within 3 minutes Order and receive food within 5 minutes Order and receive food within 10 minutes When people go to fast-food restaurants, we presume they want to receive their food as quickly as possible. Therefore, it wouldn't make sense to show respondents a MaxDiff task in which any two of these items are paired. Respondents would always prefer the faster speed to slower, so this would be a wasted comparison.

MaxDiff Help

543

Prohibitions and Design Efficiency

Those researchers familiar with conjoint analysis know that prohibiting certain level combinations from occurring in the conjoint questionnaire can often be very damaging to design efficiency (the ability of the questionnaire to estimate preference weights with precision). However, it turns out that prohibitions are generally not very harmful in MaxDiff questionnaires. We demonstrate that point below using simulated data sets with differing numbers of prohibitions. Why are prohibitions more damaging in conjoint analysis than for MaxDiff? Prohibitions in conjoint analysis are "within-concept" and lead to correlations between attributes in the independent variable matrix. For MaxDiff, the prohibitions are "between-concept" and are less damaging to orthogonality. The reduction in design efficiency in MaxDiff studies from prohibitions is due in part from deviations from orthogonality. Other contributing factors are the imbalance that is sometimes introduced in the frequency of items and the reduction in the degree of interconnectivity among items. In optimal MaxDiff studies, each item is shown an equal number of times and is also compared an equal number of times with every other item. And, for any one respondent, we want to minimize the total number of steps that it takes to directly or indirectly compare all items. For example, if A is compared to B in question one and B is compared to C in question two, then A is "linked" to C through two steps.

"Gold Standard" Design with No Prohibitions

In a MaxDiff study involving 20 items, there are t(t-1) possible two-way combinations of items that can be prohibited. The (20)(19)=190 possible paired levels are represented by the white region of the table below:

We begin by simulating data for a typical MaxDiff study that involves 20 items shown in sets of 4. Three-hundred computer-generated respondents each received a 20-set questionnaire and "answered" randomly ("bests" only). Each respondent received a unique version of the questionnaire,

544

SSI Web v7 as generated by the MaxDiff designer. To compute design efficiency, we estimate an aggregate logit model and observe the precision of the estimates (the standard errors). We also compute D-efficiency. The efficiency of one design relative to another is found by taking the ratios of the D-efficiencies for the two designs. The base case example (with no prohibitions) is our "gold standard" for comparison to designs with prohibitions.

MaxDiff Help

545

Modest Prohibitions Case

First we implement a modest number of prohibitions. Let's consider that the last five items in the study are mutually exclusive, leading to the following prohibitions (prohibited levels marked with an "x"):

These ten prohibitions are grouped, with many occurring within the same row or column. This is potentially more damaging than if ten prohibitions were scattered randomly across the table. But, even these prohibitions have very little effect on design efficiency for MaxDiff. We generated a new series of questionnaires involving these prohibitions and again had 300 respondents answer randomly. The standard errors of the estimates were only slightly larger than in the non-prohibited case. This design is 99.99% as efficient as the "gold standard" design with no prohibitions. For all practical purposes, this design is as efficient as the "gold standard."

546

SSI Web v7

Extreme Case
Let's consider a much more extreme case. We'll retain the previous prohibitions, but add other prohibitions which at face value might seem to be quite damaging:

With these challenging (insane?) prohibitions, it becomes impossible for the MaxDiff designer to find a very balanced design. It returns an imbalanced solution wherein the most represented item (item #2) occurs over three times as often as the least represented (and most prohibited) item (item #18). The largest standard error occurs for item #18 with a standard error almost double that of the most precise estimates for items in the design with no prohibitions. While these results would be concerning, these prohibitions wouldn't ruin the MaxDiff project (especially if we had reasonable sample sizes). The overall relative design efficiency (considering all items) for this highly prohibited case is 92% as efficient as "gold standard" design.

MaxDiff Help

547

"Conjoint-Style" Prohibitions
As a final example, MaxDiff may be used for "conjoint-style" brand x price experiments, such as was presented by Keith Chrzan in the 2003 Sawtooth Software Proceedings. Keith demonstrated the use of MaxDiff for pricing options on automobiles. Each option was tested at multiple prices (i.e. each item text included a description of the option plus its price). Thus, his project was a discrete choice study implemented as a MaxDiff exercise. An exercise like this is specified by developing a list of items (such as options for automobiles) where each option is displayed at different prices. Then, prohibitions are specified such that an item isn't shown twice (at different prices) within the same set. Let's consider a study in which we have four options each at five prices. The prohibitions table looks like this:

Repeating the simulation process with synthetic data, we find that this design is 99.5% as efficient as the "gold standard" plan with no prohibitions. In this section, we've only considered statistical efficiency to assess the issue of prohibitions. We find that MaxDiff designs are remarkably robust to prohibitions. However, there may be additional considerations, such as context bias, that researchers may wish to consider when using prohibitions in MaxDiff questionnaires.

548

SSI Web v7

ACA Help

549

5
5.1
5.1.1

ACA Help
Getting Started with ACA
Getting Started with ACA
ACA is software for conducting Adaptive Conjoint Analysis studies over the Internet, over an intranet, or using PC (CAPI-based) data collection. ACA is part of the SSI Web software suite. For more general information about ACA, we suggest that you read the "ACA Technical Paper" available for downloading from our home page at www.sawtoothsoftware.com. Your license entitles you to free technical support via phone or email. We encourage you to contact us at support@sawtoothsoftware.com or 360/681-2300 if you cannot find the answer to your problem or question in this documentation. We are available Mon-Fri, 7:00 am to 5:00 pm Pacific time. (Note: we provide full technical support for composing and testing your Internet survey on your local PC, and provide written instructions on how to install your questionnaire on your server. We cannot take responsibility for or be expected to provide technical support to debug any customized HTML or JavaScript you may add to your survey. You are also responsible to make sure that Perl is running and that you can run Perl (CGI) scripts on your web server. We can provide some assistance related to SSI Web's use with your server once the fundamental Perl functionality is in place.) A sample ACA questionnaire (tut2.ssi) and supporting files are available in the \Samples folder within c:\Documents and Settings\YourUsername\My Documents\Sawtooth Software\SSI Web\Version 7.X.X. You can easily access this example study by clicking Help | Sample Studies. This questionnaire lets you see an example of an ACA survey and investigate how it is set up. If you know a little HTML (which is not necessary to use ACA), you can employ some "tricks" that will help you customize the look of your survey beyond the standard options provided by the software. By opening the sample study and looking at the settings and text we've used, you can become familiar with some of the possibilities. Using ACA requires the following skills and knowledge base: The analyst managing the survey should know the basics of conjoint analysis, including defining attributes and levels properly, correctly phrasing the questionnaire text and response scales, interpreting the conjoint part-worth utilities, and running simulations. It is not necessary to be a statistician to use our software effectively, but knowledge of basic statistics for market research is highly recommended. The questionnaire is first composed locally on the PC under Windows XP, Vista, or Windows 7. The individual inputting/formatting the survey should have working knowledge of standard Windows applications such as word processing, Web browsers and/or spreadsheets. Though it is not necessary, some knowledge of HTML is valuable. To upload the survey to the Web, one must have access to an Internet/intranet server running Perl 5.003 or later. This can be your company's own Web server, or your ISP's (Internet Service Provider's) Web server. The user must know how to use FTP software to create directories, upload files to those directories, and set permissions on the files and directories. Click here to receive detailed instructions on the directories, files and permissions. To get started, we suggest you view other areas within this online help library, especially:
What is Adaptive Conjoint Analysis? Basic Overview of ACA Sections and Flow of an ACA Questionnaire

550

SSI Web v7

5.1.2

What is ACA?
Adaptive Conjoint Analysis (ACA) was first introduced by Sawtooth Software in 1985 and went on to become the most widely used conjoint software and conjoint analysis technique in the world during the 1990s (Wittink, Huber, and Vriens 1997), (Green 2000). Most recently, Choice-Based Conjoint techniques have become more popular than ACA, but ACA still finds use and has unique benefits for certain situations. Conjoint (tradeoff) analysis is a technique frequently used for market research applications. Such applications are usually concerned with consumer preference and they attempt to assess the impact of specific product features on overall preference. Products or services are thought of as possessing specific levels of defined attributes, and a respondent's "liking" for a product is modeled as the sum of the respondent's "part-worths" (sometimes called utilities) for each of its attribute levels. For example, attributes for a car could include: Price, Type, Gas Mileage, and Color. Levels of the Gas Mileage attribute might include 15 MPG, 20 MPG, and 25 MPG; and levels for the Type attribute might include: Convertible, Minivan, Compact, and Full Size. The strength of conjoint analysis is its ability to ask realistic questions that mimic the tradeoffs that respondents make in the real world. Respondents evaluate product alternatives (concepts) described by various attributes and indicate which products they prefer. By analyzing the answers, conjoint analysis can estimate the weights and preferences respondents may have placed on the various features in order to result in the observed product preferences. In contrast to direct questioning methods that simply ask how import each attribute is or the desirability of each level, conjoint analysis forces respondents to make difficult tradeoffs like the ones they encounter in the real world. As in real purchase decisions, buyers cannot get all the best features at the lowest prices. It would be time consuming and difficult for respondents to evaluate all possible product combinations in order to provide information on their values for the various product features. Conjoint analysis offers the researcher a more efficient way to obtain such information: only a carefully chosen set of hypothetical product concepts is presented to respondents for evaluation. For example, a respondent might be asked to choose between the following two concepts:

The answers to this and successive questions are used to determine the respondent's part-worths for each of the attribute levels. Once part-worths have been determined, the respondent's overall utility (preference) for a given product can be estimated by summing the part-worths for each attribute level that describes that product. Perhaps the most serious practical problem for users of conjoint analysis is that of dealing with large numbers of attributes. Clients often have a detailed understanding of their product category, and previous research has usually identified many issues of interest. The respondent, on the other hand, usually has less interest and enthusiasm, and is rarely willing to submit to a long interview. The researcher is often in conflict. Should there be a long interview, risking unhappy respondents and data of questionable quality? Or should the researcher insist on a narrower focus in the interview, providing the client with less breadth than desired? ACA resolves this conflict by eliminating the tradeoff altogether.

ACA Help

551

For example, a traditional (non-ACA) full-profile cardsort conjoint study with 12 attributes and four levels per attribute would require estimation of 37 part-worth utilities (the number of attributes x (the number of levels per attribute -1) +1). If the questions presented to the respondent have been well chosen, they will be difficult, perhaps requiring much thought. In addition, under traditional full-profile conjoint analysis, each product concept is described using all 12 attributes, requiring much reading on the part of the respondent. At the very minimum, the respondent would have to provide 37 answers; if there is any random component to the responses we would need more observations. A rule of thumb is to require three times as many observations as parameters being estimated. This would require the respondent to answer 111 questions, each presenting a concept described on 12 attributes. Adaptive Conjoint Analysis solves this problem by customizing the interview for each respondent: the respondent is asked in detail only about those attributes and levels of greatest relevance, and the task is made more manageable due to partial rather than full profiles (only showing a subset (usually 2 to 5) of the attributes in any one question). The term "adaptive" refers to the fact that the computeradministered interview is customized for each respondent; at each step, previous answers are used to decide which question to ask next, to obtain the most information about the respondent's preferences. Once the data are collected, ACA lets the researcher simulate respondent preferences for new or modified products. The Market Simulator can be used to explore "what if" scenarios, such as changes in product formulation. The researcher describes a group of hypothetical products by specifying each product's level on each attribute. Respondent part-worth utilities are used to estimate strengths of preference or buying likelihoods for each product, and results are accumulated over respondents to provide shares of preference among competing products or average estimated buying likelihoods for each product.

552

SSI Web v7

5.1.3

Basic Overview of ACA

ACA for the Web lets the researcher design a computer-interactive conjoint interview for data collection over the Web, an intranet, or using CAPI-based (stand-alone PC) interviewing. The interview can consider many attributes and levels, paying special attention to those the respondent considers most important. Questioning is done in an "intelligent" way; the respondent's part-worths are continually reestimated as the interview progresses, and each question is chosen to provide the most additional information, given what is already known about the respondent's values. In addition to the conjointrelated ACA questions, ACA comes with a CiW/50 system that can include up to 50 "standard" survey questions for capturing additional information for use in market segmentation. These 50 questions can include randomizations and skip patterns. (More than 50 questions can be fielded in conjunction with ACA through larger CiW licenses. Both CiW and ACA are products within the SSI Web suite of Web interviewing products.) Respondent data are saved to a database (text-only file) on the Web server (or on separate PCs, if using CAPI-based data collection), and are available in real-time upon completion of the interview. ACA was developed using established industry-standard technology (Perl and HTML) available on most Web servers. The interviews can be completed on nearly any Web browser. The steps for designing and conducting an ACA study are typically as follows: 1. Using a Windows-based tool running locally on a PC (without being connected to the Web), the researcher designs the ACA study: types in the attributes and levels, adds additional survey text, and chooses from a menu of options that control the overall flow of the interview. The researcher formats questions using the point-and-click interface and views the questions one page at a time using Internet Explorer. Additionally, the researcher creates a table of passwords to assign to respondents and quotas for each password combination. Once the researcher is satisfied with the look, flow and content of the interview, the interview is uploaded onto a Web (or intranet) server (or, optionally, to multiple PCs for CAPI-based interviewing). The online help includes detailed instructions regarding the directories, files and permissions that must be included in the installation. The researcher tests the ACA survey online by accessing the interview over the Web (or intranet) using a Web browser, making sure the interview is administered as expected, and checking the resulting data using the Admin Module to ensure that the data are being stored correctly. A pretest is often conducted during this stage to evaluate the suitability of the survey and to make sure the resulting utility data have face validity. Respondents are invited to participate in the survey (usually via email) and are given the URL address and password. While the interview is in progress, the researcher and/or client can view or download the data at any time without disturbing the data collection or interviews that may be in progress. When interviewing is complete, the data are downloaded using SSI Web's Admin Module, and the respondent utilities are estimated using ACA's utility calculator locally on the researcher's PC (or using ACA/HB software for hierarchical Bayes estimation). The researcher then uses the Market Simulator software included with the ACA software to analyze the results and conduct market simulations.

2.

3.

4.

5.

ACA Help

553

5.1.4

ACA Tutorial and Example

Introduction
This tutorial leads you step-by-step through a simple Adaptive Conjoint Analysis (ACA) study, from thinking about a marketing problem, planning a questionnaire and creating a new study in SSI Web, to entering a list of the attributes and levels, generating and testing the questionnaire, and analyzing the results. SSI Web is a powerful program, and this tutorial only touches on the basics for running an ACA survey. You'll become aware of different or more sophisticated approaches by reading other sections in the online help and by experimenting with SSI Web. We also suggest the following two articles, available within our Technical Papers Library at www.sawtoothsoftware.com/techpap.shtml: "ACA Technical Paper" and "History of ACA." (Note: this example and the data presented are purely fictional. The suggested steps represent one approach to this type of marketing problem, but may not necessarily reflect the single "best" way to conduct such a research study using the SSI Web system. This tutorial only uses 5 attributes, to make sure that it can run within student lab systems that are limited to 5 attributes. Most ACA studies have between 8 to 15 attributes.)

The Marketing Problem

An old college pal of yours named Gary calls you one day and invites you to dinner. You don't have other plans, so you readily agree. After all, it's been years since you've seen him, you'll be eating at the finest steak house in the city, and--most importantly--he's buying. Gary was a very bright double major in chemistry and business. You once talked him into taking a graduate-level marketing research course with you, which he reluctantly did, but his heart was clearly in chemistry. After graduation, you both went your separate ways--you to a successful consulting career, and Gary to a manufacturing company specializing in rubber and plastics. At dinner, it doesn't take long to learn why Gary was so willing to pay. Gary mentions that his company, Electra, Inc., has been developing next-generation foam rubber compounds, and has been hunting for new markets to apply the technology. Gary, an avid mountain biker, tells you that the new materials might be used for puncture-proof bicycle tires. As you dig into your medium-rare porterhouse, Gary explains that mountain bikers often ride into remote areas. If they puncture a tire, it could mean a long walk home. Consequently, many carry extra inner tubes and tools to repair a tire if needed. "The main problem," Gary admits," is weight. Solid core tires are heavier than traditional inflatable inner tube tires, which might turn off many riders. And the more dense and durable the foam compound, the heavier the tire." Gary rattles off facts related to typical tire weights compared to weights of solid foam core tire prototypes Electra had already developed. By this time, Gary's steak is only half-eaten and getting cold, but he doesn't seem to care. He further explains why the firmness or softness of a tire matters. When biking on paved roads, a very firm tire with a smooth tread is most efficient. "Bikers who do a lot of aggressive off-road dirt riding will often take some air out of the tires to increase the amount of tread area that grips the trail, and to absorb some of the bumps," he explains. He points out that with regular inner tube tires, a person can adjust the air pressure of the tire to fit the conditions. "With a solid core tire," he notes, "you'd be stuck with just one firmness." "The tread pattern is also important," he continues. "Off-road riders need an aggressive tread, but

554

SSI Web v7 people who ride mainly on paved roads need a relatively smooth tread." By now his steak is chilled and nearly inedible, so the waiter takes it away (but not before asking Gary if the steak hadn't met his expectations). You both order up desserts. Gary rambles another ten minutes about marketing strategy--specifically whether his company can market the product on its own or if it might need to pursue an alliance with an established mountain bike tire name. "Can you help us do a research study to figure out what kind of interest bikers might have in a solid core tire, and the proper combinations of weight, tread pattern, durability, and firmness?" he suddenly asks. The abruptness of his question catches you by surprise. You confidently smile, however, as the thought occurs to you that Gary should have paid more attention in that graduate marketing research course. Only five minutes into Gary's presentation you had already been thinking about how a trade-off analysis technique like ACA might help answer these issues. You remind him about the one-week unit on conjoint analysis that you both completed in that graduate-level marketing research class. After some refresher, his eyes light up at the idea. Together, you develop a list of key questions on a napkin: 1) Would mountain bikers accept a solid core tire? What types of bikers would it most appeal to? 2) How might bikers trade off tire weight and tire firmness for the solid core tires? At what point would the heavier weight of solid core tires significantly limit demand? 3) What tire tread patterns might be preferred for a solid core tire? 4) Does it make sense to align with an established player in tire manufacturing, or can Gary's company market the tire under the Electra name effectively? As you part ways for the evening, you invite Gary to your office to further discuss the problem and possible solutions.

Study Scope and Budget

The next day, you receive an E-mail from Gary. He reports that there is good support at his company to conduct some market research. However, the budget he has been given to work with is limited. He suggests a small investigational study with about thirty respondents. (Most ACA studies in practice include much larger sample sizes.) He floats the idea that mountain bike enthusiasts might be brought to a central facility, shown the prototype tires, and take an ACA survey. In a return E-mail, you agree that Gary's approach might work well, but caution him that using just 30 people to draw conclusions regarding a market of hundreds of thousands would be quite a stretch. "The data must only be used to form initial hypotheses, " you warn. "Before drawing firm conclusions, those hypotheses should be tested by another study with a larger sample size."

Defining a List of Attributes and Levels

The following week at your office, you and Gary review the objectives written during your dinner conversation on the napkin. You propose that you take the hour together to start thinking about some conjoint attributes and levels. Gary joins you in the small conference room. It has a large white board and markers, which would prove useful, and a dish full of peanut M&Ms, which you have a particular weakness for. Before diving in, you review some ground-rules regarding conjoint attributes and levels. You summarize the following rules at the top of the white board:

ACA Help

555

1. Attributes should be as close to independent of one another as reasonably possible. ACA (like most conjoint techniques) assumes a simple additive model, and violations of simple additivity within ACA are nearly impossible to detect. 2. Because attributes are independent of one another, the levels across all attributes should be able to combine freely. 3. Levels within each attribute must be mutually exclusive. Tires are described using a single level from each attribute. 4. Because just the level text appears on ACA's screens when defining different tire choices, the text for each level must stand on its own. Because of the small sample size in the study, you recommend that each attribute include few levels-perhaps three or four at most. With larger sample sizes, you might have decided to measure four or even five levels of "quantitative" (ordered) attributes like weight and tread durability. There is rarely a need to measure more than five levels of an ordered attribute, you explain, as SMRT's market simulator lets you interpolate between measured levels during analysis. (SMRT is a companion software system to SSI Web that can analyze conjoint data.) Including too many levels for an attribute significantly lowers the precision of the utility estimates for each level. It also can lead to a problem in conjoint analysis called the "Number of Levels" effect. Attributes defined on relatively more levels tend to get an artificial boost upward in importance. ACA is less prone to this bias than other full-profile conjoint methods, you assure him, but it is still a concern. By this time, you notice Gary's eyes beginning to glass over, so you cut to the chase. "OK then," you announce, "the primary objective seems to be to compare the standard inner tube tire to the new foam core tire. Let's begin by writing down those two levels." On the white board, you write: Tire Type Standard inner tube tire Puncture-proof solid foam core tire "And we need to ask respondents to consider the different degrees of firmness of the solid core tires," Gary interjects. "Some people might want a firm tire, and others less firm." You agree, and write another candidate attribute with its three levels: Firmness of Solid Foam Core Tire Very firm (best for pavement) Somewhat firm (for pavement or dirt) Less firm (for best traction in dirt) After specifying these first two attributes, you immediately notice a problem. "It seems that these attributes already violate ground rules 1 and 2," you declare. Gary furrows his brow in thought. You continue: "It wouldn't make sense to combine any level of the second attribute, Firmness of Solid Core Tire, with the first level of the first attribute, Standard inner tube tire." Gary nods in understanding, and suggests a possible solution: "Don't conjoint analysis methods let you prohibit certain levels from combining with others? Let's just prohibit the firmness levels for solid core tires from ever appearing with the standard inner tube tire." By this time you are beginning to remember why Gary hadn't enjoyed the market research course as much as you had. "Prohibitions are usually detrimental to conjoint studies," you begin. "And if it seems that you absolutely need to prohibit certain level combinations, it might just be pointing to poor attribute design." You pause a moment for effect. "I think this is one of those cases," you say. "In ACA, each product concept is defined by exactly one level from each attribute--one cannot just skip an attribute for a given product alternative. When you have attributes that only apply to some alternatives or brands, those

556

SSI Web v7 relationships require an 'alternative-specific' design. A different kind of trade-off technique called Choice-Based Conjoint can deal nicely with alternative-specific designs." (Gary again gets a glassyeyed look.) You explain to Gary that the objectives of his study (small sample size, product design research, possibility of computerized interviewing) seem to favor ACA, so you will need to think of a way around the problem. After a few moments, the solution occurs to you. Rather than treat those two attributes ("Tire Type" and "Firmness of Solid Core Tire") separately, you recognize that there are really only four unique ways the levels of those attributes can logically combine. You decide to collapse the two separate attributes into a single one with the four possible conditions: Tire Type/Firmness Puncture-proof solid foam core. Very firm (best for pavement) Puncture-proof solid foam core. Somewhat firm (for pavement or dirt) Puncture-proof solid foam core. Less firm (for best traction in dirt) Standard inner tube tire. Adjustable firmness (for pavement or dirt) With that hurdle behind you, the next series of attributes flow more easily. As for tire weight, the folks at Electra determined that a solid core tire might weigh twice as much as a standard tire. A more expensive process and compound could reduce the weight to about 50% heavier than a standard tire. You write: Weight Tire weight: 2 lbs. (about same as regular tire) Tire weight: 3 lbs. (50% more than regular tire) Tire weight: 4 lbs. (double the weight of regular tire) Gary takes the marker from you and quickly lists three common tread types on the board: Tread Type Off-road tread On-road tread On-road / Off-road tread He then lists his company along with two key tire makers that his firm might either have to compete against or form an alliance with: Brand Name Michelin Tioga Electra Finally comes the issue of tread wear durability. He writes these levels on the board: Tread Wear Durability Tread lasts for 1000 miles Tread lasts for 2000 miles Tread lasts for 3000 miles You nod approvingly, while grabbing another handful of M&Ms. Seems that he is catching on--and quite enjoying the process. The meeting has been a success. You shake hands, and walk Gary to the door.

ACA Help

557

A Sample Scripted Questionnaire

A few days later, you receive another E-mail from Gary. He has scripted some expanded attribute definitions and a few additional questions to ask study participants. You put that together with some text you usually use for ACA studies, creating a unified document: Start: Thank you for agreeing to participate in this research study! We are asking mountain biking enthusiasts like yourself to help us plan and develop new high performance mountain bike tires. This interactive survey should take about 10 to 15 minutes of your time. Click the Next button below to continue... Miles: How many miles per week on average would you say you ride your mountain bike? m Fewer than 10 miles m About 10 to 25 miles m About 25 to 75 miles m More than 75 miles

Triptype: Which of the following best describes you? m I always ride in areas where I could call a friend or "hitch a ride" if I had mechanical difficulties. m I sometimes ride to remote areas where I would have to either repair my bike or walk my bike out if I had mechanical difficulties. m I frequently ride to remote areas where I would have to either repair my bike or walk my bike out if I had mechanical difficulties.

Terrain: Which of the following best describes your mountain bike riding habits...? m 100% pavement m 75% pavement / 25% dirt m 50% pavement / 50% dirt m 25% pavement / 75% dirt m 100% dirt

Notes to Gary: At this point in the questionnaire, we need to ask respondents to rate levels within the brands, tire types, and the tread patterns. (We can't know ahead of time which levels they prefer.) This is the first section of what is called the "Priors," also known as the "self-explicated" section.

558

SSI Web v7 IntroACA: During the rest of this survey, we're going to ask you to consider features of mountain bike tires that you might purchase. The tires might differ in terms of: Brand Name: well known versus less well known brands. Tire Type: standard versus a foam core tire. The foam core tires are not inflated with air and are therefore puncture-proof. Foam core tires come in different degrees of firmness. More firm is better for riding on pavement. Less firm is best for shock absorption and traction on off-road trails. Tire Tread Pattern: On-road, Off-road tread, or On-road/Off-road tread. On-road tread is relatively smooth on the entire surface of the tire. Off-road tread has an aggressive pattern on the entire surface. On-road/Off-road tread has a flat narrow channel on the very bottom of the tire for a smoother ride on pavement, but aggressive tread elsewhere. Tire Weight: from a standard weight (about 2 lbs. per tire) to double that weight (about 4 lbs. per tire). Tread Durability: tread lasting from 1000 to 3000 miles on average.

Introrate: In this section, we'll ask you to rate different features based on your preference. You'll be shown a list of features on the left side of the screen. Use your mouse to indicate how desirable each feature is. When making your decisions, please assume that the mountain bike tire is exactly the same in every way except for the features shown.

ACA Rating Task for Preference within Attributes:

<Repeat the same question for Tire Type/Firmness, and Tread Type>

ACA Help Introimp: In this next section, we want to find out how important different features are to you. We'll ask you to indicate how important it would be for you to get the features you'd most want to have in a mountain bike tire. We'll ask you how important the following features are: Brand Name, Tire Type, Tire Tread Pattern, Tire Weight, and Tread Durability. We'll be using a 7-point importance scale, where the top category is "Extremely Important". Before you start rating the importance of each attribute, you should decide ahead of time which attribute you'll give the highest score to.

559

Notes to Gary: The second stage of the "Priors" section asks respondents to state how important each attribute is. We display the best and worst levels of each attribute, and ask the respondent to tell us how important it is to get the best level of that attribute instead of the worst one. We use the answers to these questions to decide which attributes to focus on in the conjoint pairs section that follows. We should put these Importance questions on the same page, so that respondents can adjust their answers to consistently use the scale. If we placed each importance question on a separate page, then respondents might record ratings early on that they'd like to modify, given the features that were asked later. ACA Importance Task:

< Repeat for all attributes. Show all importances on same page.>

560

SSI Web v7 Intropair: By now, we have a rough idea of your opinions about the features being studied today. We still need a little more information to learn how you would decide which mountain bike tire to buy. Based on your previous responses, we'll make up some different alternatives for you to consider. In each question we present two tires, each described by combinations of features. One is shown on the left side of the screen, and the other on the right. We'll ask you which tire you'd prefer, and to indicate your strength of preference. We will only be asking you to compare a few features at a time. Please assume that the alternatives are identical in every other way except for the differences shown.

Notes to Gary: The "Pairs" section is the conjoint (trade-off) part of the ACA survey. ACA first focuses on the attributes that the respondent already stated were most important. It uses the information from the two earlier stages to construct two product alternatives (described by various combinations of attribute levels) that are nearly balanced in preference. Respondents are asked to choose between them. By varying the product profiles and observing the respondent's preferences, we are able to further refine the rough set of utilities already established in the initial Priors section. Respondents usually answer from a dozen to thirty of these conjoint questions. After each Pairs question, ACA updates its estimate of the respondent's utilities. It uses that information to generate a new challenging trade-off for the respondent to consider. ACA Conjoint Pairs Task:

<Use 9-point scale, where 1 = "Strongly Prefer Left," 3 = "Somewhat Prefer Left," 5 = "Indifferent," 7 = "Somewhat Prefer Right," and 9 = "Strongly Prefer Right". Repeat multiple times, rotating different attribute levels to describe the tires.>

ACA Help Introcalib: This is the last section. Based on all your responses, we're making up some mountain bike tires for you to consider. You should like the first tire least, the second one best, and any others should fall in between. We'll ask you how likely you would be to buy each tire if it were available right now. Type a number from 0 to 100 where a "0" means that you "Definitely would not buy" and "100" means that you "Definitely would buy."

561

Notes to Gary: The last section of the ACA survey is the "Calibration Concepts." Respondents are presented usually five or six product concepts (one at a time). Respondents rate each product concept using a likelihood to buy scale. This is an optional section, and is only needed if we want to run "purchase likelihood" simulations during analysis. People often exaggerate stated purchase likelihoods, and we may conclude that competitive market simulations versus the standard inner tube tire is a better way to judge relative interest in the solid foam tires. The part-worth utilities have already been estimated before this stage. The Calibration Concepts section scales the utilities to best fit the purchase likelihood ratings. ACA Calibration Concepts Task:

Finish: That completes our survey. Thank you very much for your input.

Using SSI Web to Write the Questionnaire

Start by opening the Sawtooth Software SSI Web program. If you have a standard installation, you start it by clicking Start | Programs | Sawtooth Software | Sawtooth Software SSI Web. "SSI" stands for "Sawtooth Software Inc." and "Web" conveys the idea that this software is for developing Web-based surveys (although SSI Web can also be used for standalone computer interviewing with its CAPI interviewing module.) ACA is one component within that software program. The next step is to open a new study, and define a study name.

562

SSI Web v7 After starting up SSI Web, choose File | New Study. The New SSI Web Study dialog appears, with your cursor active in the Name field. The Location field lists the folder (directory) that SSI Web currently is using to store studies. You can use any folder you like for this tutorial project. You can browse to and create new folders by clicking the Browse... button to the right of the Location field, and then (after browsing to the folder in which you want to create a new sub-folder to contain this study) by clicking the "new folder" icon. Each studyname in SSI Web has a maximum number of twenty-four characters (either letters or numbers), and SSI Web requires that the study be located within its own folder. The folder that SSI Web automatically creates for your study also carries the same twenty-four character name as the study. For this tutorial, you might create a name such as academo1. From the New SSI Web Study dialog, specify academo1 as the studyname. Click the OK button. You are returned to the main menu, and a new "Study Navigator" window is displayed along the left-hand side of the screen. As you work with this study, items appear in the Study Navigator window, listing the functional areas you have been using or that are now available to you. This list of items provides a Shortcut link to access those parts of SSI Web. Alternatively, you can access those same areas by clicking icon buttons on the toolbar or by using the pull-down menus. Click Compose | Write Questionnaire... to add an ACA exercise to your study. Each SSI Web study can contain one or multiple ACA exercises. When you insert a question or a conjoint exercise within your list of questions, it is inserted directly below the question or page break you initially highlight. Highlight the page break following the Start question, and click Add.... Specify a name for the ACA exercise, such as tires.

Entering the List of Attributes and Levels

When composing conjoint analysis studies, one typically begins by specifying the list of attributes and

ACA Help levels in the software.

563

You use the Attributes tab of the ACA Exercise Settings dialog to enter the list of attributes and levels:

To add the first attribute, click the Add button (at the bottom left of the Attributes panel).

Type the first attribute name: Brand in the Attribute Display Text field. The attribute name is a label

564

SSI Web v7 that the respondent will see in the interview. If you want to use a shorter label to display to you as the questionnaire author for program settings and data export, specify that label in the Internal Label field. If you do not specify an internal label, the label in the Attribute Display Text is used. Click OK to accept this information and close the dialog. Now that at least one attribute name is in the list, the Add button under the Levels panel becomes active. Also note that the Brand attribute is highlighted in the Attributes panel. With the Brand attribute highlighted in the Attributes panel, click the Add button under the Levels panel to add levels within the Brand attribute. The Level Text dialog is displayed. Type Michelin in the Level Display Text field. To add the next level of Brand, press the ENTER key twice. Type in the next level: Tioga. Repeat the process for the remaining level, Electra. Back on the Specify Attributes and Levels dialog, you will notice that there is a drop-down box labeled Level Ranking. This lets you inform SSI Web about any a priori ordered attributes. Some attributes have a rational preference order (e.g. lower prices always preferred to higher prices, faster speeds preferred to slower speeds, etc.). You can avoid asking respondents to rate the levels for attributes that carry an obvious preference order by choosing Worst to Best or Best to Worst level order. This is a benefit, but be forewarned that if you misinform SSI Web about the correct order of preference, the resulting part-worth utilities will be terribly wrong. The current attribute, Brand, does not have an obvious preference order. You don't know ahead of time which level(s) the respondent prefers. Therefore, you should leave this attribute as No a priori ranking. When you are ready to add the next attribute (Tire Type/Firmness), click the Add button under the left Attributes panel, type the attribute label, and click OK to place that new attribute on the attribute list. With that new attribute highlighted on the attributes list, click the Add button under the Levels panel to add the four levels of that attribute. Follow the same pattern for the remaining attributes. For your convenience, we repeat the full list of attributes below. Note that you can copy-and-paste attribute level text from this document (and other text documents) into the text fields within SSI Web. After highlighting the words to copy with your mouse, use the shortcuts Ctrl-C to copy, and Ctrl-V to paste into the desired field. (Hint: you can select a list of attributes or levels from Word and paste into SSI Web using the or spreadsheet" icon. This can save a great deal of time.) "paste from Word

Please remember to specify the Level Ranking for each attribute by highlighting each attribute in the Attributes panel and using the drop down control for each to set the Level Ranking. You should prespecify the ranking for any attributes with rational order of preference. Make sure to specify Best to Worst ranking for Weight, and Worst to Best ranking for Tread Wear Durability. Leave the Level Ranking as No a priori ranking for the other attributes. Brand Name Michelin Tioga Electra Tire Type/Firmness Puncture-proof solid foam core. Very firm (best for pavement) Puncture-proof solid foam core. Somewhat firm (for pavement or dirt) Puncture-proof solid foam core. Less firm (for best traction in dirt) Standard inner tube tire. Adjustable firmness (for pavement or dirt) Tread Type Off-road tread On-road tread On-road / Off-road tread Weight

ACA Help Tire weight: 2 lbs. (about same as regular tire) Tire weight: 3 lbs. (50% more than regular tire) Tire weight: 4 lbs. (double the weight of regular tire) Tread Wear Durability Tread lasts for 1000 miles Tread lasts for 2000 miles Tread lasts for 3000 miles

565

After you have specified the list of attributes and levels, you are ready to set up your ACA questions. To do so, select the Design tab.

Four main areas appear on this dialog: Priors Randomization, Importance Settings, Pairs Settings, and Calibration Settings. It may help to review some terms that describe the different sections of the ACA survey: Priors (the self-explicated level ratings and attribute importances.) Pairs (the conjoint section of the interview. Pairs of products defined on multiple levels are displayed, and respondents indicate the one they prefer, using a rating scale running from left to right.) Calibration (an optional section called Calibration Concepts used when you need to run purchase likelihood market simulations. Respondents rate a few product concepts individually on a likelihood of buying scale.)

566

SSI Web v7 Let's begin thinking about the Priors. Before showing any conjoint questions, we need to ask respondents to rate the levels within any attributes that we don't know ahead of time the rational preference order. For example, brand name, tire type, and tread type. We also specify control settings regarding the attribute importance question that follows the level ratings. The Priors Randomization area (with the suggested settings for this questionnaire) is shown below.

We suggest you randomize the order of the levels displayed on the screen and the order of the attribute presentation for the importance questions. Check the Randomize Level Order in Rating Questions and Randomize Attribute Order in Rating and Importance Questions boxes. These settings will help control for any psychological order bias. Next, let's move to the Pairs Settings area of this dialog. The Pairs represent the conjoint section of the interview. Pairs of products defined on multiple levels are displayed, and respondents indicate the one they prefer, using a rating scale running from left to right. Rather than take the space within this tutorial to explain the settings, we refer you to the context-sensitive help provided by typing F1. For this Mountain Bike Tires tutorial, we suggest the following settings: Maximum attributes for Pairs Section Number of Pairs questions Number of Pairs questions at each stage Number of attributes in each pairs question in the first stage Number of attributes in each pairs question in the last stage 5 20 10 2 3

We suggest including all 5 attributes in the Pairs section (Maximum attributes for Pairs Section = 5). Even though the number of pairs suggested by ACA's formula applied to this particular list of attributes and levels (explained when you press F1 for help) is just 14, you may decide that your respondents can reliably answer a few more. More Pairs information is good to have (particularly with studies that have small sample sizes), as long as respondents don't become so fatigued that they give poor quality data or drop out of the survey. You decide for the mountain bike tire study that respondents can reasonably complete 20 pairs. Larger studies with more attributes may lead you to require respondents to answer more questions, though most researchers don't ask respondent many more than about 30 Pairs questions. (See an article entitled "Accuracy of Utility Estimation in ACA" in the Technical Papers Library at www.sawtoothsoftware.com/techpap.shtml for more information about how the number of pairs questions affects the precision of part-worth utility estimates.) Let's turn to the Calibration Settings area of this dialog. This is an optional section that you might decide to drop. With the Calibration Concepts section gone, this is another justification for increasing the number of Pairs questions from the "suggested" 14 for this particular set of attributes and levels to 20. If you need to use Purchase Likelihood simulations during analysis, then you must include a

ACA Help

567

Calibration Concepts section. We'll include it in this tutorial so that you can have practice setting up a Calibration Concept section. Set these two fields to: Number of Calibration Concepts = 5 Number of attributes in each concept = 5. Close the ACA Exercise Settings by clicking OK. SSI Web automatically places the ACA questions into your survey:

SSI Web has added a number of ACA related questions to the survey. These questions are based on your settings and the default template within SSI Web.

568

SSI Web v7 Before we examine or fine tune those ACA questions, let's turn our attention to the issue of assigning passwords for respondents to enter at the opening screen of an SSI Web survey.

Passwords and the Start Screen

Placing a survey on the web makes it convenient for a geographically dispersed population to take surveys. However, the danger is that the survey may become available to people that have not been invited to take the survey. Also, some respondents might try to take the survey multiple times. Assigning respondent passwords is a way to deal with both of these issues. Password assignment is beyond the scope of this tutorial, so to make things simple we'll assume that no passwords are to be used. The Start screen is placed as the first page in your questionnaire, whether you are using passwords or not. If you are using passwords, it is the page in which respondents type passwords to access the survey. If not using passwords, you'll use this screen to specify any introductory/welcome text. Let's enter the introductory text into the Start screen using the Write Questionnaire dialog. You can access the Write Questionnaire dialog by selecting Compose | Write Questionnaire or by clicking the "pencil" icon on the toolbar.

The introductory text and other survey questions would most likely be initially developed within a word processing document. Assuming you really had such a document, you might use the Ctrl-C to copy, and the Ctrl-V shortcuts to paste the information into SSI Web. We suggest you simply copy-andpaste the text within this document into your SSI Web questions rather than re-type the text (if viewing this document with Acrobat Reader, you can use the "text select" icon from the Acrobat Reader toolbar.) The introductory screen is as follows: Start: Thank you for agreeing to participate in this research study! We are asking mountain biking enthusiasts like yourself to help us plan and develop new high performance mountain bike tires. This interactive survey should take about 10 to 15 minutes of your time. Click the Next button below to continue... We'll insert this introductory screen in the Start question that is automatically the first question of any SSI Web survey. From the Write Questionnaire dialog open the Start question by highlighting it in the list of questions and clicking Edit.... Alternatively, you can double-click the Start question, and it automatically opens. The Questionnaire Access and Passwords dialog is displayed::

ACA Help

569

Most question types in SSI Web have "Header 1", "Header 2," "Body" and "Footer" sections. (The Start question has all but the "Body" section.) These are "text" areas in which you can insert any text (including HTML). When the question is viewed with a web browser, the sections are organized roughly as follows:

For this introductory screen (that includes three paragraphs of information), it seems to make sense to place the first paragraph in the "Header 1" area, the second paragraph in the "Header 2" area, and the

570

SSI Web v7 third paragraph in the "Footer" area. Type (or cut and paste) the following text for the Start question into the text areas in the Start question. Put the first paragraph in Header 1, the second paragraph in Header 2, and the third paragraph in Footer. Thank you for agreeing to participate in this research study! We are asking mountain biking enthusiasts like yourself to help us plan and develop new high performance mountain bike tires. This interactive survey should take about 10 to 15 minutes of your time. Click the Next button below to continue... The main dialog only shows a portion of the text, but you can click the "pencil" icon each text area to expand the text editing window. adjacent to

To preview how the question will look when it runs on the respondent's computer using Windows Explorer, click the Preview button. The question is displayed in "Preview" mode (no data are saved, and no special "error checking" or "validation" is performed for the question).

SSI Web automatically places paragraph breaks (extra blank lines) between the Header 1, Header 2, and Footer sections. If you put all the text in a single section, you may see that when the web browser interprets the text, it all runs together without any blank lines between the paragraphs (unless you insert some HTML instructions to force blank lines between paragraphs). We'll talk about using HTML within your text to take greater control over the layout, font, and style later in this unit. Click the "X" icon in the upper-right hand portion of the Preview window to close that window and return to the previous SSI Web dialog. After viewing the text in preview mode, you might decide that it is really too small, or that you want to make the text bold. By default, the font appearance and color schemes are based on the Style selected for your project (Compose | Survey Settings, Style tab). You can try different styles to see how it affects the appearance of the questionnaire. You can also override the text and color settings in the Style by clicking the Paintbrush icon of this question and specifying new settings to override the appearance

After you are happy with the layout of the introductory screen, click OK to return to the Write Questionnaire dialog. Because we aren't using respondent passwords, we'll ignore the additional tabs on the Questionnaire Access and Passwords dialog.

ACA Help

571

Specifying Select Questions

(If you are already familiar with adding Select questions within SSI Web surveys, you may choose to skip forward to the section entitled "Fine-Tuning the ACA Questions"). When you add a new question to the List of Questions, it is inserted directly below the question that was highlighted when you clicked the Add... button. If you need to move a question once it has been added to the List of Questions, simply highlight the question to be moved and click the or buttons to move the questions within the list (you can also highlight a question and click Ctrl-X or Ctrl-C to cut or copy and Ctrl-V to paste questions within the current questionnaire, or even to another SSI Web study.) Now you are ready to specify the first Select-Type question: Miles: How many miles per week on average would you say you ride your mountain bike? m Fewer than 10 miles m About 10 to 25 miles m About 25 to 75 miles m More than 75 miles

First, make sure that you highlight the page break directly beneath the Start question on the List of Questions (when you add a new question, it is placed directly below the highlighted question/page break on the list). At the Write Questionnaire dialog, click Add..., choose Select as the question type, and type Miles for the question name. Click OK and the Select Question dialog is shown. Place the heading text for the question in the Header 1 field. To specify the response options, click the Response Options tab. From the Response Options tab, use the Add... button to add the four response options for this question. When you are finished, the question should look something like:

Now that you have seen how to specify Single Select question types, you have the tools you need to specify the remaining two select-type questions for the mountain bike tires questionnaire (Triptype and Terrainplease refer to the questionnaire text for these questions as presented near the front of this unit).

572

SSI Web v7

Formatting Text in SSI Web Questions

In addition to the Select questions we just added to the questionnaire, there are some text-only, instructional screens to add. Referring to the questionnaire we outlined earlier, these "questions" are named IntroACA, Introrate, Introimp, Intropair, Introcalib, and Finish. Even though the respondents aren't asked to provide specific inputs to these, we refer to these as "questions" in the sense that we add them to the questionnaire as if they were standard questions, and they are listed separately by their "question" name in the List of Questions. Such questions are called Text/HTML Filler. After the Terrain select-type question, follows the text-only instructions: IntroACA: During the rest of this survey, we're going to ask you to consider features of mountain bike tires that you might purchase. The tires might differ in terms of: Brand Name: well known versus less well known brands. Tire Type: standard versus a foam core tire. The foam core tires are not inflated with air and are therefore puncture-proof. Foam core tires come in different degrees of firmness. More firm is better for riding on pavement. Less firm is best for shock absorption and traction on off-road trails. Tire Tread Pattern: On-road, Off-road tread, or On-road/Off-road tread. On-road tread is relatively smooth on the entire surface of the tire. Off-road tread has an aggressive pattern on the entire surface. On-road/Off-road tread has a flat narrow channel on the very bottom of the tire for a smoother ride on pavement, but aggressive tread elsewhere. Tire Weight: from a standard weight (about 2 lbs. per tire) to double that weight (about 4 lbs. per tire). Tread Durability: tread lasting from 1000 to 3000 miles on average.

This text layout is a bit more challenging than we dealt with before (when we put three paragraphs of introductory text into the Start question). There are more than three separate paragraphs here, and we'll need to deal with the additional element of bolding selected text. Add a Text/HTML Filler question directly following the Terrain question in the questionnaire. To do so, highlight the Terrain question, click Add, and specify the Question Name as IntroACA and the question type as Text/HTML Filler. The Text/HTML Filler dialog is displayed. Earlier in this tutorial, when we created the Start, we placed each paragraph of text in a separate Header 1, Header 2, or Footer section. We saw that SSI Web automatically places blank lines between text in these sections. However, with the text in the IntroACA question, there are many more paragraph breaks. We'll take the opportunity here to introduce the concept of using a few simple HTML instructions within our survey text. HTML stands for "HyperText Markup Language" and provides simple ways for you to enhance the look of your surveys, such as by bolding or underlining words, or making paragraph breaks within text. Browsers know how to interpret HTML instructions when displaying the results to the screen. If you know HTML, you can use it whenever you wish within SSI Web surveys to accomplish your aims. If you don't know HTML, it isn't very difficult to learn a few HTML tricks, or you can use the toolbar available within the editor that appears when you click "Pencil" icons in SSI Web:

ACA Help

573

Either type or cut-and-paste the text for the IntroACA question into the Header 1 text field (Any of the other three fields could be used also, though the Body field is indented). Click the "Pencil" icon to display the larger text editing window. Initially, the text appears something like this:

Even though it appears that there are extra blank lines between the paragraphs, web browsers ignore these hard returns (as well as more than one consecutive space characters), so you'll need to provide HTML instructions to insert these paragraph breaks (hard return plus blank line). HTML instructions are placed within <> brackets, called "tags." For example, the HTML instruction to create a paragraph break begins with an "open" paragraph tag written as <p> and optionally ends with a "close" paragraph tag written as </p>. The text to be formatted as a separate paragraph is enclosed within these tags. You can either directly type HTML within your document, or you can highlight text to be modified and click the icons on the HTML toolbar in the editor. If you highlight the first paragraph with your mouse: During the rest of this survey, we're going to ask you to consider features of mountain bike tires that you might purchase. The tires might differ in terms of: And then (with the text in the first paragraph highlighted), click the "Paragraph" icon This inserts a <p> prior to the text and a </p> after the text: on the toolbar.

574

SSI Web v7 <p>During the rest of this survey, we're going to ask you to consider features of mountain bike tires that you might purchase. The tires might differ in terms of:</p> When the browser interprets this text, it doesn't display the "tags" but instead separates the text enclosed within the tags as a separate paragraph. Repeat the same for each of the paragraphs in the IntroACA question. Next, we need to bold certain words in the text. The HTML tags for bolding text are <b></b> (with the text to be bolded placed between the open and close bold tags). You can either directly type these tags within the document, or highlight the text to be bolded with the mouse and click the "Bold" icon on the toolbar. After you finish separating the paragraphs with <p></p> tags and bolding the appropriate text with <b></b> tags, it should look something like:

Click OK to return to the Text/HTML Filler dialog and then Preview to see how the web browser displays this question. It should look like:

ACA Help

575

You should now have the tools necessary to add the other Text/Only HTML Filler questions in their appropriate positions in the survey: Introrate, Introimp, Intropair, and Introcalib (Finish is a Terminate/Link question, explained in the next paragraph). Remember to place Introrate directly prior to the tires_Rating1 question, Introimp directly prior to the tires_Importance1 question, Intropair directly prior to the tires_Pair1 question, and Introcalib directly prior to the tires_Calibration1 question. Make sure when adding the Finish question at the end of the survey to add it as a Terminate/Link question type. On the Settings tab of the Terminate/Link question, click Terminate Respondent to indicate that respondents that reach this question are finished. SSI Web may warn you that a terminating question must stand alone on its own "page" in the survey. This leads us into our next discussion on page layout.

Page Layout and Study Settings

It may be helpful at this point to review how SSI Web breaks the survey into separate pages, and some basic global settings that affect the look and functionality of your SSI Web questionnaire. The Write Questionnaire dialog also displays the page breaks that may be currently set for your questionnaire.

576

SSI Web v7

This dialog shows how the various questions we've specified (or that SSI Web has automatically added to the list) are arranged across different pages. Page breaks are indicated by "---------------<Page>". Notice that the Select questions we've added to the questionnaire are all currently arranged on the same page. Let's assume we wanted to break these up, one question per page. You set a new page break after the Miles question by highlighting Miles on the list and clicking the Add Page Break button. Miles is now placed on its own page. Repeat this operation to place Terrain, IntroACA, and Introrate on separate pages. Delete the page break between tires_Importance3 and tires_Importance4 to place all the Importance questions on the same page. Make sure to place the final text screen Finish on its own (last) page.

From the Write Questionnaire dialog, click the Survey Settings icon

(alternatively, click

ACA Help

577

Compose | Survey Settings...). This dialog controls global aspects of the questionnaire (meaning that the settings here control the general behavior of the questionnaire rather than page- or questionspecific aspects.) Examples include the fonts and colors used in the questionnaire, the wording for the "Next" button, and the global header and footer that can appear at the top or bottom of each page of the survey. The details of this dialog are beyond the scope of this unit, but you can learn more about the items on the dialog by pressing the F1 key.

Fine-Tuning the ACA Questions

After you have finished these steps, your questionnaire should be complete (though it may lack polish). You will note that SSI Web used a set of default names to represent the many ACA questions in the questions list. SSI Web built those questions drawing from a default template. Some of the text for the questions you will certainly want to edit to fit the appropriate wording for your study. Depending on how picky you are about the look of your survey, you could spend a good deal of time adjusting each question and adding HTML elements to achieve just the right look. Let's view one question from each ACA section, as an introduction to fine-tuning ACA questionnaires. The first section in ACA is the "Ratings" section, where respondents are asked to rate any levels within attributes for which we don't know ahead of time the order of preference (levels for attributes such as brand or color). From the List of Questions in the Write Questionnaire dialog, highlight and edit the tires_Rating1 question (note that there are only three Ratings questions: one for each attribute for which we haven't assumed a Level Ranking (a priori) order. Click Preview and one of the level rating questions is displayed (due to randomization, the attribute shown and also the order of the levels may be different):

Note that SSI Web automatically "fills in" the attribute name ("Weight", in this example) in the text, and places the levels at the left-side of the grid. If you close this window and view the text in the Header 2 text field (using the "Pencil" icon is easier), you'll see that the source text for the question header is: Please rate the following [%ACAAttribute()%]s <br> in terms of how desirable they are. Notice that SSI Web uses a "system function" called ACAAttribute( ) (delimited by [%...%] ) to fill in the text you supplied for the attribute name for attribute 1. This makes it easier to compose ACA surveys,

578

SSI Web v7 as the text you supply for the attribute names is automatically carried forward in this section of the questionnaire. Of course, you may find that the text you used doesn't lead to very proper grammar when inserted this way within the template sentence. In that case, you can modify the wording as you like. However, recognize that you only supply the text once for each major section in ACA, and this same text is used across all questions in that section. So, the text must be generic enough to be meaningful across all questions in the section. Also note the use of the <br> tag, which instructs the browser to insert a line break at that point. Hint: If you want to customize the text directly above or below any ACA question, you can insert a new Text/HTML Filler question intermingled within the questions in the ACA section (you can also insert other question types mid-stream within an ACA segment should you be inclined). Make sure to format the page breaks from the Write Questionnaire dialog so that the new customized text you've inserted appears properly within the separate pages of the ACA questionnaire. From the Write Questionnaire dialog, highlight the tires_Importance1 question and click Edit. This is the Importance question in ACA, where respondents are shown the best and worst levels from each attribute and are asked to rate the contrast in those two levels in terms of importance. Click Preview, and the following is displayed (depending on your attribute randomization, you may see a different attribute):

Note that you'll need to edit the "XXXXXXXX" characters to be appropriate for the subject of your survey ("mountain bike tires" would be appropriate here). To do this, return to the ACA Importance Questions dialog and edit the text directly within the Header 2 text field. Also note the use of two System Functions (%ACABest( ) and %ACAWorst( )) within this question that dynamically return each respondent's most preferred and least preferred levels (based on the answers provided in the earlier ACARAT questions or based on your previous Level Ranking setting for attributes. When you edit the tires_Pair1 question and preview it, the following is displayed:

ACA Help

579

Again, you need to edit the "XXXXXXXX" characters to be appropriate for your survey. Finally, when you edit and display the tires_Calibration1 question, its default presentation is:

Again, you'll need to edit the "XXXXXXXX" to be appropriate.

Adding Polish and Style

You have probably noted that the survey we've created is pretty bland looking. We can add some polish and style by clicking the Survey Settings icon from the Write Questionnaire dialog (or by clicking Compose | Survey Settings from the main menu). First, let's select a style from the Styles tab. Select a style you wish to use, and click Use Selected at the bottom of the dialog to implement the style. We'll select Panama as the style, but you might want to experiment with other styles. On the General Settings tab, you can select to use graphical Next, Previous, and Select/Checkbox

580

SSI Web v7 buttons. A library of buttons is provided, available in the C:\Program Files\Sawtooth Software\SSI Web\graphics folder. On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Mountain Bike Tire Questionnaire as the header text. The Progress Bar tab lets you add a progress bar to your survey. We'll click the check box to add a progress bar to the page footer. When you go back and test run the survey, your survey should have a bit more polish. Under Panama style (and by selecting graphical Previous and Next buttons), our survey now looks like:

Test Running Your Survey Using "Local Test Server"

Although the Preview Survey function is nice, you generally will want to test your survey in runtime mode, with any skip patterns, response verification, and randomizations in place. You could upload your files and the Perl scripts to the Web Server, but SSI Web provides a more convenient way to test run your survey locally on your PC or laptop. When SSI Web was installed on your computer, web server software called "Apache" was also installed. Apache lets your computer run the questionnaire just like a remote Web Server would.

Local Test Server

From the Write Questionnaire dialog, Click the Test Survey button Locally.

and select to Test Survey

ACA Help

581

Your browser opens up the first survey page. This is your survey, as it would appear to a respondent over the Internet (there are slight visual differences from browser to browser). To close the survey, simply close the browser window (using the X in the upper-right hand corner). To run the survey again, click the Test Survey button .

After you have edited the survey to the point that you are pleased with the look, content, and functionality of your survey, you should do some careful testing to ensure you haven't made a fundamental error in your ACA setup. We suggest you take the survey yourself under Local Test Server. You should answer each question legitimately and carefully. Pay special attention to the ACA questions to make sure that the attribute levels appear correctly: the most preferred levels should always appear above the least preferred levels in the Importance questions. The Pairs questions should generally not pose "obvious" trade-offs (all the levels on the left better than the ones on the right, or vice-versa). All the worst levels should appear together within the first Calibration Concept, and the best levels in the second Calibration Concept. If the importance questions, Pairs, or Calibration Concepts are not appearing correctly, this suggests that the Level Ranking (a priori) order settings are incorrect for some attributes. You must be vigilant to avoid this error. Even experienced ACA researchers have been known to set a Level Ranking order incorrectly when rushed. After you have taken the questionnaire carefully as suggested above, you should download the data (downloading data is described in a separate tutorial called " Getting Started with SSI Web: A 45minute Hands-On Tour"), then compute and examine your part-worth utilities using the market simulator (described later). The part-worths should very closely reflect your preferences. If they don't, this suggests something may be wrong with your study setup. Generally the most common error is that you have not correctly set an ordered attribute as Best to Worst or Worst to Best. Such errors are disastrous, so double or even triple-check your attribute settings. After you are convinced that the survey is functioning properly, you should also pretest your survey among your colleagues. They can give you feedback regarding the usability of the survey, and you can examine the resulting part-worths to make sure the data at least have face validity.

Pretesting and Fielding

Let's now return to the marketing problem and story we began at the beginning of this document. After you have tested the survey using the Local Test Server, you post the survey to your web server, using Sawtooth Software's hosting service, or using an web server set up for you by an ISP (Internet Service Provider). (Setting up your survey on the web is beyond the scope of this unit, but is described in detail elsewhere. Fielding options also include hosting on your own company's server, or using Sawtooth Software's hosting services). You send an email to Gary, with a link to your survey, and he browses to and takes the survey. He suggests some minor wording changes. After you make those changes, Gary invites six other individuals at Electra take the survey. You download the data and analyze. Due to the small sample size, the results are a bit noisy (and there is an obvious bias toward the Electra brand name), but the data seem to feel right. Next, you recruit six mountain biking enthusiasts within your city to come to a central site to see a demonstration of the new foam tire technology and take the survey. After a short hands-on demonstration of the new foam core tires, they sit down to computers connected to the Web and take the survey. You and Gary watch silently as they take the survey. At the end of each session, you ask each respondent follow-up questions to ensure that there weren't any parts that were difficult to understand or just didn't make sense. You ask them specifically about the ACA sections, making sure that they understood what to do at each stage of the survey, and that they didn't feel overwhelmed by the task. After debriefing the test respondents and analyzing their data to ensure that the results look reasonable, you make a few small adjustments and proceed to field.

582

SSI Web v7 Thirty respondents are brought to the field facility, shown the solid foam tire prototype, and complete the survey. The completed data reside on the ISP's server where you uploaded the SSI Web survey. The details for viewing and downloading the data from the server are provided in other areas of the SSI Web documentation and are beyond the scope of this ACA unit. Even so, it is useful to at least describe the steps you would perform to access and download the data from the Web, estimate the part-worth utilities, and begin analysis using the market simulator.

Data Management, Utility Estimation, and Moving the Data into SMRT
SSI Web includes an Admin Module so that you can monitor or access your project from any computer connected to the Web. The Admin Module is password protected, and your passwords for access are specified on the Server Settings tab of the Survey Settings dialog. SSI Web generates random administrative access passwords whenever you create a new study, but you can modify them to suit your needs. To download the data from the 30 respondents to the mountain bike tires study, you can browse to the administrative module for your study on the web site (again, we are speaking hypothetically, as for this tutorial study there is no such site set up). Once at the administrative module, you would download your data (by clicking Download Data from the main menu), making sure to save your data (named STUDYNAMEdat.dat, where STUDYNAME is your study name) to the same folder on your hard drive in which you developed the SSI Web project. Alternatively, from the SSI Web interface, you can simply click Field | Download Data... to download your data without logging into the Admin Module. Once you have downloaded the data, you are ready to estimate part-worths. Assuming you had downloaded the data as described above, you would click Analysis | Calculate ACA Utilities Using Ordinary Least Squares. Computation takes just a few seconds. If you try this with this tutorial study, you will receive an error stating that you have no data within your project folder to analyze. However, hypothetical data for this project are stored in a tutorial folder within the accompanying SMRT software that you received together with your SSI Web system. SMRT stands for "Sawtooth Software Market Research Tools" and contains the market simulation tool used to analyze data resulting from any of Sawtooth Software's conjoint analysis systems. SMRT is the platform that (given the proper license with its user identification codes) runs ACA for Windows studies and includes the tutorial data for this Mountain Bike study. We'll discuss how you would move data from your ACA project into SMRT (hypothetically, since the data aren't actually in your ACA project) but then open the SMRT software system and continue using the Mountain Bike tutorial data that were installed there. When you calculate part-worth utilities using Analysis | Calculate ACA Utilities Using Ordinary Least Squares this process creates a text-only format file containing case IDs and part-worth estimates called STUDYNAME.utl. To move the ACA part-worth utility data into SMRT for analysis, you would (again, this is for discussion purposes onlyyou should not actually perform these steps for this Mountain Bike tutorial): 1. Start the SMRT software by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 2. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 3. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing ACA utilities (selecting ACA v4.x Utilities *.utl as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market

ACA Help Simulator and begin simulations.

583

Analyzing the ACA Data Using SMRT's Market Simulator

SMRT is a companion software system to SSI Web that can be used for analyzing the results of ACA and CBC studies. To start SMRT, click Start | Programs | Sawtooth Software | Sawtooth Software SMRT. Open the Mountain Bike Tutorial study by clicking File | Open and then browsing to find the Tires1.smt file located in the ...Program Files\Sawtooth Software\SMRT\Tutorial folder. Once the study is open, from the main menu, click Analysis | Market Simulator. You will note that there is a list of available Utility Runs (your recently computed run is available), and a list of Simulation Scenarios (currently empty). Click Compute!, and a report of average utilities and importances for your data set is displayed in the report window:
Scenario: Utility Run: ACA Utility Run Average Utility Values Rescaling Method: Total Respondents Michelin Tioga Electra Puncture-proof... Very firm... Puncture-proof... Somewhat firm... Puncture-proof... Less firm... Standard inner tube tire... Off-road tread On-road tread On-road / Off-road tread Tire weight: 2 lbs... Tire weight: 3 lbs... Tire weight: 4 lbs... Tread lasts for 1000 miles Tread lasts for 2000 miles Tread lasts for 3000 miles Average Importances Brand Name Tire Type/Firmness Tread Type Weight Tread Wear Durability Total Respondents 13.99 19.98 22.44 27.22 16.37 30 Zero-Centered Diffs 30 4.73 18.14 -22.88 -6.62 4.60 -21.91 23.94 -20.16 1.14 19.02 62.51 8.94 -71.45 -37.17 -2.14 39.31

<<The above average utilities are rescaled utilities using the zero-centered "diffs" method. The diffs method rescales utilities so that for each individual the total sum of the utility differences between the worst and best levels of each attribute across attributes is equal to the number of attributes times 100. Note: the attribute utilities are influenced by the number of respondents in the simulation and respondent weighting, but are not affected by the product specifications you enter. After you have seen these once for a particular group of respondents, you may choose to omit them in subsequent simulations by un-checking the Display Utilities box in the Scenario Specification dialog.>>

584

SSI Web v7 It is very important that you learn how to interpret part-worth utilities properly. Please see a document entitled "Interpreting Conjoint Analysis Data" available in our Technical Papers library at www.sawtoothsoftware.com/techpap.shtml. It is difficult to make firm conclusions from just these thirty respondents, but we might try to form some tentative conclusions: First, looking just at the report of average importances (the second section above), it appears that the most important thing to respondents is the weight of the tire. This confirms some of Gary's concerns and implies that Electra should pay attention to controlling the weight of the solid core tires. The least important attribute is brand, which is good news if Electra wants to break into this market without an established name. Paying attention to the Average Utility Values report, Electra seems less preferred than the other two brands (no surprise there). On average, most respondents seem to prefer standard tires to the new technology. It looks like there might be a slight inclination toward a hybrid (On-road/Off-road) tread. There appears to be significant resistance in increasing tire weight, particularly from 3 to 4 lbs. Although you don't have much data to work with for segmentation purposes in the Tires1 study (just 30 respondents), you should note that you can run market simulations by "banner" point. Banner points are the segmentation variables that run across the columns of a cross-tabulation report. If you select a Banner variable using the drop-down menu at the upper-left of the Market Simulator dialog, and then click Compute!, you will note that the utility/simulation report is broken out by different groups of respondents, depending on how they answered to the Banner variable. A lot of important effects due to the competitive context, respondent heterogeneity and correlations of preferences among attribute levels can occur within a data set. These may be completely obscured by viewing just average utilities and importances. Also, the concept of utility scores seems difficult to grasp for many managers. For these reasons, running market simulations offers a more powerful and revealing way to look at conjoint data.

Market Simulations
The first time you open the Market Simulator for a study, the list of Simulation Scenarios is empty. A Simulation Scenario is a particular "what-if" scenario, wherein certain products are placed in competition with one another in a simulated marketplace. Prior to fielding your study, you had confirmed with Gary which specific competitive scenarios he wanted to test. You checked those against the list of attributes you and he had developed to ensure that the attributes covered the proper ranges and were specified adequately. Gary was most interested in two specific scenarios. The first one assumed that Electra was offering the only solid core tire in the market. Each brand was assumed to offer tires to appeal to different riding styles/terrain. Simulation Scenario 1:

ACA Help Product Electra1 Electra2 Electra3 Michelin1 Michelin2 Michelin3 Tioga1 Tioga2 Tioga3 Brand Electra Electra Electra Michelin Michelin Michelin Tioga Tioga Tioga Tire Type/Firmness Tread Solid/Very firm On-road Weight 3.75 lbs. Durability 2500 miles 2000 miles 1000 miles 3000 miles 2500 miles 1500 miles 3000 miles 2500 miles 1500 miles

585

Solid/Somewhat first On/Off-road 3.5 lbs. Solid/Less firm Standard Standard Standard Standard Standard Standard Off-road On-road 3 lbs. 2 lbs.

On/Off-road 2 lbs. Off-road On-road 2 lbs. 2 lbs.

On/Off-road 2 lbs. Off-road 2 lbs.

Notice that some of the levels of Weight and Durability are in between the levels we measured in the study. The market simulator lets us interpolate between levels, so this is not a problem. However, it will be easier to specify these levels if we have assigned "Level Values" to each level. To assign Level Values, from the main menu click Analysis | Market Simulator and then click the Assign Level Values button at the very lower left-side of the dialog. The Assign Level Values dialog is displayed. Using the drop-down box, select Weight. Click the first row in the Value column (corresponding to 2 lbs.). Type "2." Click the next two cells and enter "3" and "4." Next, select Tread Wear Durability using the drop-down box, and type "1000," "2000," and "3000" for the values. Click OK to return to the Market Simulation menu. Create a new simulation scenario by clicking Add.... When you click the Add... button, the Scenario Specification dialog is displayed. The first step is to type a scenario name into the Name field, in the upper left-hand corner of the dialog. Type Base Case. Next, you'll specify the nine products for the first scenario. The area you type the products into looks like the grid of a spreadsheet. The first column is where you type the product label. As you click the various columns associated with the attribute levels, the level codes will appear in the window below to remind you of the codes associated with each attribute's levels. The attribute level codes for the nine products for the first scenario were:

586

SSI Web v7 Product Electra1 Electra2 Electra3 Michelin1 Michelin2 Michelin3 Tioga1 Tioga2 Tioga3 Brand 3 3 3 1 1 1 2 2 2 Tire Type/Firmness Tread 1 2 3 4 4 4 4 4 4 2 3 1 2 3 1 2 3 1 Weight 3.75 3.5 3 2 2 2 2 2 2 Durability 2500 2000 1000 3000 2500 1500 3000 2500 1500

Specify these nine products in the product entry grid. To add new rows to the grid (for additional products) click the Insert Product button. Hint: While working within the product specification grid, you can press the Enter key to advance to the next cell. You can use the arrow keys to move the cursor within the grid. You can also cut-andpaste information within the grid, or from a spreadsheet. After you have specified these nine products, next choose a simulation method. The default method is Randomized First Choice. We generally recommend this method for competitive market scenarios like this one, so we'll use the default. Because we have already examined the utilities and importances, un-check the Display Utilities box. Click OK again to close the Scenario Specification dialog and return to the main Market Simulator dialog. Choose a utility run to use for your simulation. At this point, you should only have one run in the list, so this is your only choice. If there were multiple utility runs (for example, another run from the ACA/HB system), you could select that run by highlighting it in the list of Utility Runs. To simulate shares for a scenario you have specified, place a check mark in the box next to the scenario name Base Case in the Simulation Scenarios list, and then click Compute!. SMRT examines each respondent's record and predicts which product(s) each respondent would likely choose based on the product specifications and each respondent's utilities. The following report is displayed (we've inserted commentary within brackets):
Scenario: Base Case Utility Run: ACA Utility Run Product Simulation Settings Simulation Mode: Model: Total Sampling Iterations: Attribute Variability Multiplier: Product Variability Multiplier: Exponent: Simulation Randomized First Choice 100000 Autocalibrated Value (0.509807) 0 1

<<Above are the settings that govern this run. It is not necessary at this point for you to know what

ACA Help

587

these settings mean (#iterations, Multipliers, Exponent). These are explained in other areas of this online help and also in the manual.>>
Product Specifications Electra1 Electra2 Electra3 Michelin1 Michelin2 Michelin3 Tioga1 Tioga2 Tioga3 Brand 3 3 3 1 1 1 2 2 2 Tire Type 1 2 3 4 4 4 4 4 4 Tread Type 2 3 1 2 3 1 2 3 1 Weight 3.75 3.5 3 2 2 2 2 2 2 Durability 2500 2000 1000 3000 2500 1500 3000 2500 1500

<<Above are the product level codes you specified for the nine products in this simulation. Below are the simulated shares of preference (choice) for these products, along with their standard errors.>>
Product Shares of Preference Electra1 Electra2 Electra3 Michelin1 Michelin2 Michelin3 Tioga1 Tioga2 Tioga3 Shares 7.55 9.23 8.89 14.51 11.66 7.43 19.07 13.68 7.99 30 Std Err 1.22 1.41 1.48 1.49 1.36 1.12 2.37 1.48 1.04

Total Respondents

Though a total is not shown, the shares of preference sum to 100%. Standard errors are presented next to each product's shares, which are useful for expressing confidence intervals around the share estimates. Again, it is difficult to form anything but tentative conclusions from just 30 respondents. Even so, we might conclude from this simulation run that if Electra were to market a solid core tire it would generally achieve a lower share of the market than established brands with standard tires. These simulated shares assume that Electra could achieve equal distribution and brand awareness relative to Michelin and Tioga. The base case simulation suggests that there is relatively balanced demand for the three types of solid core tires. The Market Simulator is a powerful tool for testing nearly an unlimited number of possible market scenarios. It is important to remember that the shares of preference resulting from conjoint predictions are not equivalent to actual market shares, and often look quite different. Many other factors in the real world influence market shares and cannot be measured and reflected solely by conjoint data. Conjoint assumes perfect information, equal distribution and availability, and that each respondent is in the market and able to purchase. Conjoint results reflect the potential market acceptance, given proper promotion, distribution and time. The second scenario Gary was interested in is nearly the same as the first, but assumes that Michelin also comes out with solid core tires (products 10-12). It also assumes that the weight and durability of the new Michelin tires are slightly inferior to Electra's tires.

588

SSI Web v7 Simulation Scenario 2: Product Electra1 Electra2 Electra3 Michelin1 Michelin2 Michelin3 Tioga1 Tioga2 Tioga3 MIchelin4 Michelin5 MIchelin6 Brand Electra Electra Electra Michelin Michelin Michelin Tioga Tioga Tioga Michelin Michelin Michelin Tire Type/Firmness Solid/Very firm Solid/Somewhat firm Solid/Less firm Standard Standard Standard Standard Standard Standard Solid/Very firm Solid/Somewhat firm Solid/Less firm Tread On-road On/Off-road Off-road On-road On/Off-road Off-road On-road On/Off-road Off-road On-road On/Off-road Off-road Weight 3.75 lbs. 3.5 lbs. 3 lbs. 2 lbs. 2 lbs. 2 lbs. 2 lbs. 2 lbs. 2 lbs. 4 lbs. 3.75 lbs. 3.5 lbs. Durability 2500 miles 2000 miles 1000 miles 3000 miles 2500 miles 1500 miles 3000 miles 2500 miles 1500 miles 2000 miles 1500 miles 1000 miles

You can save a lot of time in entering this scenario if you retain the settings for the first nine products when you create the new scenario. SMRT lets you do this. Click Add..., and from the Add Scenario dialog use the drop-down control to choose Base Case as the starting template for the new scenario. You can name this second simulation scenario Competitive Reaction. After placing your cursor in the final row of the product specification grid, click Insert Product three times to open three additional rows at the bottom to enter Michelin's three new tire products. The level codes/values for the additional three products are: Product MIchelin4 Michelin5 MIchelin6 Brand
1

Tire Type/Firmness Tread 1 2 3 2 3 1

Weight 4 3.75 3.5

Durability 2000 1500 1000

1 1

After entering the new product specifications, click OK to return to the Market Simulator dialog. Before running the new Competitive Reaction scenario, you may wish to clear the previous results from the report window by clicking Clear and un-checking the box for the Base Case scenario. Then, click Compute!, and the following is produced as part of the market simulation output:
Product Shares of Preference Electra1 Electra2 Electra3 Michelin1 Michelin2 Shares 5.44 7.02 7.18 12.98 9.88 Std Err 0.88 1.21 1.31 1.41 1.23

ACA Help
Michelin3 Tioga1 Tioga2 Tioga3 Michelin4 Michelin5 Michelin6 5.76 18.39 13.13 7.58 4.66 4.34 3.63 30 0.84 2.26 1.46 0.95 0.81 0.63 0.66

589

Total Respondents

It is important to note that the simulation model we are using (Randomized First Choice) does a good job of accounting for differential substitution for similar (near substitute) products. This Competitive Reaction reflects a line extension for Michelin, and new products that should compete very closely with Electra. For the purposes of illustration, let's place the results from the two simulation runs within the same data table:
Product Shares of Preference Base Case Shares 7.55 9.23 8.89 14.51 11.66 7.43 19.07 13.68 7.99 N/A N/A N/A ----100.00 Competitive Reaction Shares 5.44 7.02 7.18 12.98 9.88 5.76 18.39 13.13 7.58 4.66 4.34 3.63 ----100.00

Electra1 Electra2 Electra3 Michelin1 Michelin2 Michelin3 Tioga1 Tioga2 Tioga3 Michelin4 Michelin5 Michelin6 Totals

(Note that you can copy-and-paste results directly from the Market Simulator report window directly into a spreadsheet. A few simple copy-and-paste steps can build the above table. You can even copyand-paste results from within the Market Simulator report window to other places within the same window.) Examining the above table, note that the introduction of the solid core Michelin tires took significant share from both Electra's solid core and Michelin's standard tires. The new entries compete strongly with existing Michelin tires due to the same brand name and with Electra's tires due to an imitation of the new technology. In contrast, Tioga's shares are not much changed from the Base Case to the Competitive Reaction scenario. Though, with the additional competition offered by Michelin's line extension, the shares had to decrease at least some. Note that Michelin's solid core tires achieve a lower share than Electra's solid core tires. One tentative conclusion from this is that the edge in performance specifications for Electra's solid core tires can overcome the negative of having a less preferred brand name. How might the market change if Michelin and/or Tioga can produce solid core tires that match Electra's specifications? You may wish to run additional scenarios to test those possibilities.

590

SSI Web v7

5.1.5

Sections and Flow of ACA Questionnaires

Suppose the attributes being studied describe automobiles, with these levels: BODY TYPE Two-Door Sedan Four-Door Sedan Hatchback Minivan Convertible DRIVE CONFIGURATION Rear-Wheel Drive Front-Wheel Drive All-Wheel Drive ORIGIN Made in USA Made in Japan Made in Europe COLOR Red Blue Green Yellow White Black PRICE $18,000 $20,000 $22,000 $24,000 $26,000 The ACA interview has several sections, each with a specific purpose. We've provided examples of the questions in each section, together with brief explanations of their purposes.

1) Preference for Levels (Required Section)

(The "ACA Rating" question type.) First, the respondent rates the levels for preference. This question is usually omitted for attributes (such as price or quality) for which the respondent's preferences should be obvious. (When you input the attributes and levels, you can specify that the order of preference for levels is "best to worst" or "worst to best" and the ratings question is skipped for such attributes.) The screen may look like the following:

ACA Help

591

(The rating scale can be defined from 2 to 9 points. We suggest using at least 5 scale points. In any case, it is probably not wise to use fewer scale points than the number of levels in any one attribute for which the Rating question is asked.) The respondent is required to check one radio button per attribute level. If a level is skipped, ACA will prompt the respondent to complete the question prior to moving to the next page in the interview.

2) Attribute Importance (Optional Section)

(The "ACA Importance" question type.) Having learned preferences for the levels within each attribute, we next determine the relative importance of each attribute to this respondent. This information is useful in two ways. First, it may allow us to eliminate some attributes from further evaluation if the interview would otherwise be too long. Second, it provides information upon which to base initial estimates of this respondent's utilities. (Notes: if using ACA/HB for part-worth utility estimation, you may elect to omit the Importance section. If the standard format of the Importance question as described below doesn't fit your needs, you can also customize your own importance questions. Getting the Importance section right is critical. For example, we strongly recommend educating respondents regarding the full array of attribute options prior to asking them importance questions, so they use the scale consistently and know ahead of time which attribute(s) should be reserved to receive the highest scores.) As a matter of fundamental philosophy, we do not ask about attribute importance with questions such as "How important is price?" The importance of an attribute is clearly dependent on magnitudes of differences among the levels being considered. For example, if all airline tickets from City A to City B were to cost between $100 and $101, then price couldn't be important in deciding which airline to select. However, if cost varied from $10 to $1000, then price would probably be seen as very important. Our questioning is based on differences between those levels the respondent would like best and least, as illustrated:

592

SSI Web v7

(The rating scale can be defined from 2 to 9 points. We suggest using at least 4 scale points.) It is worth mentioning that there are different ways to phrase the "importance" question, and indeed referring to attribute "importance" is sometimes difficult for some respondents to comprehend. How the wording of this question should be phrased depends on the audience. Other possibilities with the ACA software include: The "Regrets" Format:

The "Unexpected Windfall" Format:

At this point we have learned which attributes are most important for this respondent and which levels are preferred. From now on, the interview is focused on those most important attributes and combinations of the levels that imply the most difficult trade-offs.

ACA Help

593

3) Paired-Comparison Trade-Off Questions (The "ACA Pairs" question.) Next, a series of customized paired-comparison trade-off questions is presented. Up to this point in the interview, we have collected "prior" information; no conjoint analysis has been involved. The Pairs section elicits the conjoint tradeoffs. In each case the respondent is shown two product concepts. The respondent is asked which is preferred, and also to indicate strength of preference. The example below presents concepts differing on only two attributes. Although concepts may be specified on up to five attributes, simple concepts like these present an easy task for the respondent, and are a useful way to begin this section of the interview.

(The rating scale can be defined from 2 to 9 points. We suggest using at least 7 scale points.) The number of attributes appearing in each concept is specified by the author, and can be varied during the interview. Concepts described on more attributes have the advantage of seeming more realistic. It is also true that statistical estimation is somewhat more efficient with more attributes. However, with more attributes, the respondent must process more information and the task is more difficult. Experimental evidence indicates that a tradeoff occurs: as the number of attributes in the concepts is increased, respondents are more likely to become confused. It appears best to start with only two attributes. Most respondents can handle three attributes after they've become familiar with the task. Preliminary evidence suggests that beyond three attributes, gains in efficiency are usually offset by respondent confusion due to task difficulty. The computer starts with a crude set of estimates for the respondent's part-worths, and updates them following each submitted page (you can specify how many pairs questions will be completed per page). The crude estimates are constructed from the respondent's preference ranking or rating for levels, and ratings of importance of attributes. Each pairs question is chosen by the computer to provide the most incremental information, taking into account what is already known about this respondent's part-worths. The interview continues in this manner until a termination criterion (specified by the author) is satisfied. Every time the respondent completes a page of pairs question(s), the estimate of the respondent's part-worths is updated. Updating the part-worths improves the quality of subsequent pairs questions. We strongly encourage you to place page breaks often throughout the pairs questions. If you can count on fast load times (respondents will be using fast web connections), we suggest page breaks after each pairs question.

594

SSI Web v7

4) Calibration Concepts (Optional Section)

(The "ACA Calibration" question type). Finally, the computer composes a series of "calibration concepts" using those attributes determined to be most important. These concepts are chosen to occupy the entire range from very unattractive to very attractive for the respondent. The respondent is asked a "likelihood of buying" question about each. We first present the concept we expect the respondent to like least among all possible concepts, and the second is the one we expect to be liked best. Those two concepts establish a frame of reference. The remaining concepts are selected to have intermediate levels of attractiveness. This information can be used to calibrate the part-worth utilities obtained in the earlier part of the interview for use in Purchase Likelihood simulations during analysis. Conjoint part-worths are normally determined only to within an arbitrary linear transformation; one can add any constant to all the values for any attribute and multiply all part-worths by any positive constant. The purpose of this section is to scale the part-worths non-arbitrarily, so that sums of part-worths for these concepts are approximately equal to logit transforms of the respondent's likelihood percentages. The screen format is:

If you plan only to conduct share of preference simulations, and particularly if you are using the ACA/HB system for hierarchical Bayes estimation, there is no need to include calibration concepts in your ACA survey. See Estimating ACA Utilities for more information on the use of calibration concepts.

Setting Page Breaks in ACA Surveys

ACA lets you include multiple questions per page. If the page cannot fit on the respondent's screen, a scroll bar is provided for respondents to scroll down the page. At the bottom of each page is a "Next" button that respondents click when they have completed the current page. Page breaks must occur between each of the major ACA sections: Rating Importance

ACA Help Pairs Calibration Concepts

595

You may include additional page breaks within each section if you think it is more convenient for your respondents and provides a more desirable overall layout. We suggest you include page breaks within the Pairs section so that the utilities can be updated and the subsequent pairs have an opportunity to become more customized and better balanced. The number of pairs questions to include per page represents a tradeoff between the benefits of updating versus the time required to process and receive a new page of pairs questions over the respondent's Web connection.

596

SSI Web v7

5.1.6

Defining Attributes and Levels

You are limited to 30 attributes (depending on your license) with at most 15 levels of an attribute, though we encourage you to develop attributes that have about 5 or fewer levels whenever possible. (Hint: you can select a list of attributes or levels from Word or Excel and paste into SSI Web using the "paste from word processor or spreadsheet" icon. This can save a great deal of time.) An attribute is a characteristic of a product (e.g. color) which can take on various levels (e.g., red, yellow, blue). Every attribute must have at least two levels. The underlying theory of conjoint analysis holds that a buyer places a certain part-worth (or utility value) on each attribute level, and that the overall utility of any product is obtained by summing up the part-worth values of its specific attribute levels. In conjoint experiments, we show respondents product concepts described using different combinations of attribute levels and ask them to somehow express their preferences for those concepts. One way of thinking about conjoint analysis is that we are conducting a designed experiment for each respondent to see how preferences vary as we change attribute levels. After observing how respondents evaluate products in response to changes in the underlying attribute levels, we can estimate the impact (part-worth) each attribute level has upon overall product preference. Once we learn respondents' preferences for the various attribute levels, we can predict how buyers might respond to any potential combination of levels in our study, whether or not that actual product was ever displayed during the interview.

Conditional Relationships
(Not available within ACA studies.) The conditional relationships button lets you associate conditional prices or graphics with one or more attributes in your study. Conditional Graphics: Sometimes, you may want to associate a graphic with an attribute or multiple attributes. For example, if a combination of brand and form factor make up a graphic, one can specify the two attributes separately and associating a conditional graphic with their combined levels. This permits estimation of separate effects for brand and form factor.

Guidelines
In generating attributes and levels, consider the following guidelines: 1. Attributes should be independent. It is therefore important to economize; including attributes with overlapping meanings is wasteful and can lead to biased part-worth utilities. Furthermore, levels for related attributes often cannot combine naturally and freely with one another. Though it can lead to more realistic interviews, it is often detrimental (and sometimes fatal) to prohibit levels from occurring with others. 2. Levels within each attribute should be mutually exclusive. This point becomes clear when you specify products using the market simulator (during the analysis phase) and are forced to choose only a single level from each attribute.

ACA Help Consider the following attributes for a study on optional automobile features: Optional Features: Sunroof GPS (Global Positioning System) Extended warranty

597

This formulation doesn't permit simulating preference for a car that has both a Sunroof and a GPS. Similarly, we could not simulate preference for an automobile that had none of these features. There are two ways to resolve this quandary: a) Create an attribute with all potential combinations of these features. This results in an eight-level attribute, if you include the option that none of these features is available. b) Formulate three separate attributes each with two levels: (No Sunroof, Sunroof), (No GPS, GPS), (No Warranty, Warranty). Option (a) adds seven parameters to the model and forces the measurement of an explicit three-way interaction. With the more complex model definition, we can investigate whether there are diminishing returns by bundling the features. By splitting the options into three distinct binary attributes (Option b), only three parameters are added to the model; however, interactions are not measured. 3. Attribute levels should cover the full range of possibilities for existing products as well as products that may not yet exist, but that you want to investigate. Although the market simulator allows you to extrapolate and interpolate, only linear interpolation and extrapolation are possible. Although interpolation is likely to produce acceptable results, extrapolation is prone to error and should be avoided. One way to ensure that you are including the appropriate levels and ranges is to ask your client to specify ahead of time the market simulations to be run during the analysis phase of your study. That exercise can often reveal weaknesses in your attribute specifications. 4. Prohibitions, if at all possible, should be avoided. Specifying unnecessary or excessive prohibitions is one of the most common mistakes. The problem usually begins when either the analyst (or the analyst's client) notices that some product combinations displayed during the interview are not realistic, given what currently exists in the market. Sometimes a product is shown with all the best features at the lowest price; or two attribute levels that would not naturally occur in the real world are paired together. The inclination is simply to prohibit such combinations. We urge you to exercise restraint when considering prohibiting levels from occurring with one another. Too many prohibitions can lead to confounded effects and the complete inability to calculate stable utilities. It is better to prompt respondents that they will see combinations during the interview that are not yet available in the market or that seem unlikely. You can urge respondents to answer as if these products were actually available today. There are other strategies for dealing with prohibitions. Consider the example below with brands of soda and package types: Brand: Sawtooth Spritz Kong Kola Splut Package Type: 2-liter bottle 6-pack of 12-oz cans Suppose that Splut was only available in 6-packs of cans. Furthermore, you are displaying actual pictures of the products, and thus can only display actual (not potential) products. Rather than define a

598

SSI Web v7 prohibition between Splut and the 2-liter bottle, it would make more sense to combine these two attributes as a single attribute with five levels: Sawtooth Spritz in a 2-liter bottle Sawtooth Spritz in a 6-pack of 12-oz cans Kong Kola in a 2-liter bottle Kong Kola in a 6-pack of 12-oz cans Splut in a 6-pack of 12-oz cans Under this strategy, no prohibitions are required. 5. The number of levels you choose to define an attribute can have a significant bearing on the results. The first concern has been called the "Number-of-Levels Effect." All else equal, attributes defined on more levels tend to get more importance. The second concern is that you limit the number of levels on which quantitative attributes are described. We suggest not including more than about five levels to describe attributes such as price or speed. It's usually better to have more data at each price point than to have thinner measurements at more price points. Measuring too many points along a quantitative function can result in troublesome reversals. If you cover the entire range of interest with fewer levels, you can interpolate between levels within the market simulator to get finer granularity if needed. 6. Attributes that cannot be adequately described in words should be represented in multimedia. But if attributes do not require multimedia to adequately communicate their properties, it would probably be a mistake to make them multimedia. Though the interview might appear more attractive, it might bias the results in favor of multimedia attributes.

ACA Help

599

5.1.7

Specifying Prohibitions in ACA

This dialog lets you specify which combinations of attribute levels (or attributes altogether) may not be shown together in pairs (and for ACA questionnaires in calibration concept questions).

A Warning
Prohibitions, if at all possible, should be avoided. Specifying unnecessary or excessive prohibitions is one of the most common mistakes. The problem usually begins when either the analyst (or the analyst's client) notices that some product combinations displayed during the interview are not realistic, given what currently exists in the market. Sometimes a product is shown with all the best features at the lowest price; or two attribute levels that would not naturally occur in the real world are paired together. The inclination is simply to prohibit such combinations. We urge you to exercise restraint when considering prohibiting pairs. Too many prohibitions can lead to imprecise part-worth estimation. It is better to prompt respondents that they will see combinations during the interview that are not yet available in the market or that seem unlikely. You can urge respondents to answer as if these products were actually available today.

How Many Prohibitions Can I Safely Add?

A common question that users ask is "how many prohibitions can I specify without seriously damaging the results of my study?" This cannot be answered without more information. For it is not the sheer number of prohibitions but the specific pattern of prohibitions that more directly affects the degree of correlation among the attribute levels in the design matrix, and thus the design efficiency. Assume that the researcher wants to specify 3 prohibitions between a 2-level attribute and a 3-level attribute. There are just 6 possible combinations that can occur when combining those two attributes. If the researcher prohibits 3 of those combinations from occurring, this eliminates half of the possible combinations between those attributes. There would probably be less damage to the efficiency of the design if three total prohibitions were allocated across three separate attribute combinations (one per pair of attributes). As a side note, the researcher in this instance may decide to specify the three non-prohibited combinations of those two attributes as a single attribute, rather than as two separate (and not independent) attributes. The main drawback of this approach is that after combining these two attributes, the researcher will not be able to compute the attribute importance or the relative partworths of each attribute independently.

600

SSI Web v7

5.1.8

Setting ACA Control Parameters

You control the way the ACA questionnaire is implemented using the Design tab on the ACA Exercise Settings dialog. This dialog provides the following areas: Priors Randomization Pairs Settings Calibration Settings Importance Settings The controls on each dialog are described below.

Priors Randomization Randomize Level Order in Rating Questions You can randomize the order of attribute levels (such as levels of brand, color, style, etc.) that are displayed in ACARAT (level ratings) questions. This can control for order bias. Randomize Attribute Order in Rating and Importance Questions If you select this option, the order that attributes are presented in the ACARating (level ratings) and ACAImportance (importance) questions is randomized. (The same randomized order is used for both sections, for a given respondent.)

Pairs Settings Maximum Attributes for Pairs Section This controls how many attributes will be carried forward to the pairs questions. If you specify as many attributes as are in your study, all attributes will be used within the pairs questions. If you specify a number n which is less than your total number of attributes, only the n most important attributes (as rated by the respondent in the Importance question) will be used in the Pairs section. You can take as many as 20 attributes forward to the Pairs section. We generally recommend taking all attributes (up to 20) forward to pairs questions if you are using ACA/HB estimation. If using the standard OLS estimation, we suggest you bring around 8 to 12 attributes into Pairs. However, we have not conducted any research to confirm that about 8 to 12 is optimal, and it depends on the number of Pairs questions you plan to ask. The more Pairs questions asked, the more attributes you might consider carrying into the Pairs section. Number of Pairs Questions (0-50) This number controls how many total conjoint pairs are to be asked. We suggest that you ask a number of pairs equal to 3 (N - n - 1) - N, where: N = total number of levels taken into the Pairs section n = total number of attributes taken into the Pairs section There are many times when the number of pairs suggested by the above formula are more than respondents can manage. (Most ACA surveys will have between 15 to 30 pairs questions.) We suggest you determine how many pairs questions respondents can reliably answer, and not try to ask more pairs than is prudent. There are also instances with very small ACA designs in which the above formula leads to very few suggested pairs (i.e. sometimes six or fewer). In that case, we recommend that you increase

ACA Help

601

the number of pairs questions to a number that your respondents can reasonably answer. Number of Pairs Questions at Each Stage This value controls how many pairs questions are asked before the next degree of complexity is implemented. For example, if you want the first 10 pairs questions to show two attributes at a time, and the next 10 pairs to show three at a time, you should specify "10." Number of Attributes in Each Pairs Question in the First Stage (2-5) This value specifies how many attributes will be presented at a time throughout the first stage. We recommend showing two attributes at a time in the first few pairs. Number of Attributes in Each Pairs Question in the Last Stage (2-5) Controls the maximum complexity (number of attributes shown) in conjoint pairs questions. We generally recommend that three be the maximum number of attributes in the last pairs. Although, if attribute text is particularly short and you feel that respondents would not become overly confused or fatigued, you may wish to display four or five attributes in the last pairs.

Calibration Settings Number of Calibration Concepts (0,3-8) Controls how many calibration concept questions will be shown. If you want to use the Purchase Likelihood model during analysis, you should include at least three Calibration Concepts. We recommend using at least five if you plan to include this section. If you plan to use ACA/HB to compute part-worths (hierarchical Bayes) and if you do not need to run the purchase likelihood model, you should probably not include a Calibration Concepts section. The total number of calibration concepts cannot exceed the number of attributes (or the number of attributes in each concept--see below) plus one. Number of Attributes in Each Concept (2-8) Specifies how many attributes are to appear within each Calibration Concept question. Only the most important attributes are included (as rated in the Importances section). We recommend including no more than about six attributes. If you have specified prohibited pairs, there are situations in which fewer attributes than the number you specify are actually displayed within the Calibration Concept section.

Importance Settings Use Default ACA Importance Questions By default, an importance question is asked for each attribute. Skip Importance Questions (Advanced) Advanced users may wish to skip the importance questions (ACA/HB estimation required). We strongly suggest you read the "Omitting the 'Importance' Question" documentation within this online manual and also the article in the Technical Papers section of our website entitled "The 'Importance' Question in ACA: Can It Be Omitted?" If you select this option, you then choose among the following advanced options: Use Group Means for Prior Importances You can use importances based on previous respondents (collected on the same server or CAPI installation). These importances are used for selecting the attribute level combinations in the conjoint pairs. ACA/HB estimation is required.

602

SSI Web v7 Number of Preliminary Respondents before Using Group Means This defines how many respondents must complete the survey prior to using group means. Before group means are used, we assign prior equal importances. Set Prior Importances Equal for All Attributes If you select this option, all attributes will be assumed to have equal prior importances. The order of attributes used in the pairs section will be randomized. Utility balance for conjoint pairs will be based on the level ratings/rankings already established in the questionnaire or by the author when setting a priori attribute level order, and then updated by the respondent's subsequent pairs questions. ACA/HB estimation is required. Set Prior Importances Based on other Questions Some users may wish to customize the importance questions. You can use any series of questions or variables that reflects positive values (decimal values allowed). Click the Specify Questions... button to associate a question/variable with each attribute importance.

ACA Help

603

5.1.9

How ACA Data Are Stored

ACA stores information in its data file detailing the attribute level combinations shown at each stage of the interview along with the respondents' answers. You can download the data file from the Web server using ACA's Admin Module, or accumulate the files from different PCs if using CAPI-based data collection. Then, using SSI Web you can export the responses to the "standard" (non-conjoint related) questions to a text-only fixed/delimited format using File | Export Data | All Data . You can also export the ACA data to an "audit trail" .ACD file by clicking File | Export Data | Prepare ACA Data Files (*.acd and *.aca).... The .ACD file records information about the combinations of levels displayed at each stage of the interview along with respondent answers. Our ACA/HB software reads the .ACD file when estimating part-worth utilities under HB. To obtain OLS estimates for the attribute levels, you should run ACA's utility calculator. The utility calculator reads the information from the respondent data file (STUDYNAMEdat.dat), computes the part-worths, and saves the results to a STUDYNAME.utl file. This format is identical to the .UTL files used in ACA version 4 for the PC. Click HERE to view the formats for the .UTL and .ACD files.

604

SSI Web v7

5.1.10

Computing Utilities and Analyzing ACA Data

Before computing part-worth utilities, you must first have downloaded a data file (STUDYNAMEdat.dat) from your Web server (or accumulated the data from PCs), and placed it within your study directory, where all other study-related files are resident. We use Ordinary Least Squares to compute part-worth utilities. The resulting part-worths are saved to a STUDYNAME.utl file, which is a text-only file with a specific format. (Please see Appendix A for more information on interpreting conjoint analysis results.) Select Analysis | Calculate ACA Utilities using Ordinary Least Squares.... Choose Settings.... You are able to specify a number of options on this dialog: Respondent Numbering Method You can choose from five options for assigning respondent numbering. This lets you specify how the respondent numbers (case IDs) are to be generated. User Name is one of the passwords from the passwords module. You should only choose this if these values were purely numeric (no letters). Password is also one of the passwords from the passwords module. You should also only use this if the passwords were purely numeric (no letters). Internal Interview Numbers are those that the system assigns when respondents complete the survey over the Web. There can be gaps in the sequence if some respondents started a survey but didn't finish. Sequentially Assigned Numbers (default) are integers 1 through n. Other lets you choose another numeric variable to use for respondent numbering. This last option is useful if you are linking surveys from another source and need to merge the data sets by a "passed-in" variable. Respondent Filter All Respondents: Choose this option to export all respondents: both complete and incomplete. Data for questions that were not completed are left blank. Qualified/Completes Only: Choose this option to export only qualified complete data records. A survey is considered qualified complete when respondents click the Submit (Next) button on a terminating question that is marked with a qualified status. Disqualified and Incompletes Only: Choose this option to export only disqualified and incomplete data records. A survey is considered disqualified when respondents click the Submit (Next) button on a terminating question that is marked with a disqualified status. A survey is considered incomplete if respondents have not reached a terminating question. Prepare for Utility Computation... Click Prepare for Utility Computation... to compute utilities and save the STUDYNAME.utl file to the current folder. A report appears summarizing the results of the computation, and (on a separate tab) listing the respondent utilities one respondent record per row. You can save the results to an Excel file by clicking Save As....

Output When OLS estimation is completed, the output is displayed in a report window. The first (Summary) tab displays the average part-worth utilities across respondents (normalized by the zero-centered diffs method), and the average attribute importances. On the second (Individual Utilities) tab are the raw utility data for each respondent, organized as one row per respondent. Click the Save As... button to save the data as an Excel (.XLS) file. Zero-Centered Diffs Method: The Summary tab displays the average rescaled utilities (as well as the standard deviations) under the zero-centered "diffs" method. The diffs method rescales utilities so that for each individual the total sum of the utility differences between the worst and best levels of each attribute across attributes is equal to the number of attributes times 100. This normalizes the data so that each respondent has equal impact when computing the population average.

ACA Help

605

A studyname ACA.utl file is also automatically generated, which is a text-only file with a specific format. This file may be directly imported into the SMRT program for market simulations.

606

SSI Web v7

5.1.11

Is ACA the Appropriate Technique?

Many methods are available for collecting and analyzing conjoint data, and the researcher contemplating a conjoint study must choose among them. We at Sawtooth Software have had many years of direct experience with these methods, as well as the benefit of many conversations with users of our own and other software. Based on that experience, we offer the following suggestions: The Full Profile Method (such as used in CVA) was the first conjoint method introduced to the marketing research community, and it is considered the original standard. Green and Srinivasan (1990) recommended use of the full profile method when the number of attributes was six or fewer. Our opinion is that this is a useful guideline, but there isn't a hard limit, and the number of attributes should depend on the complexity of the attribute descriptions and respondent interest/knowledge of the product category. During the 1980s and 1990s, we tended to favor ACA for projects involving about six or more attributes. As choice-based approaches with HB estimation have captivated the industry and shown their worth, we have shifted our attention toward CBC and Adaptive CBC. Adaptive CBC combines some of the best aspects of ACA of CBC, and the early evidence suggests that it is a valuable new technique. We have not conducted nor seen research that directly compares ACA to Adaptive CBC, but we expect that ACBC should have important advantages when the number of attributes is between 5 and 12. When the number of attributes exceeds about 15, and pricing research isn't a goal, we don't know which method would prevail. Over the last 10 years, it has become clear to us that the self-explicated importance question is the aspect of ACA that either "makes or breaks" it. If the importance questions are asked poorly, they can misinform utility estimation and be harmful. An example of poor execution is to not educate respondents ahead of time regarding the array of attributes, and to ask the importance questions oneby-one on separate screens. Some ideas for improving the quality of the importance questions are given in the section of this documentation entitled Customizing the Importance Question. ACA is a "main effects only" model, and assumes there are no interactions among attributes. Many conjoint practitioners agree that one must remain alert for the possibility of interactions, but that it is usually possible to choose attributes so that interactions will not present severe problems. Like other conjoint methods, ACA can deal with interactions in a limited way by collapsing two attributes into one. For example, we could deal with an interaction between car color and body style by cross-classifying the levels: Red Convertible Black Convertible Red Sedan Black Sedan However, if the attributes in question have many levels, or if an attribute (such as price, for example) is suspected of having interactions with many others, then collapsing attributes will not be enough. In that case too many parameters must be estimated to permit analysis at the individual level, and the most common solution is to evaluate interactions by pooling data from many respondents. ACA has been shown to have weaknesses in pricing research, where it often underestimates the importance of price. Since about 1995, some ACA researchers have included price in their ACA studies, but have adjusted its importance based on other information included in the survey. This was called the "dual conjoint" approach, and there are articles on this in our Technical Papers Library on our website. When ACA is executed well, with a good importance section implementation, it provides solid results for applications such as product design and segmentation work. It can work with the smallest of sample sizes, even in the case of revealing the preferences for a few or even one individual. Even though the bulk of interest lately is in favor of CBC and ACBC, those who have become expert in ACA

ACA Help

607

methodology and ACA study execution will continue to use the method confidently in appropriate contexts for many years to come. For more information about selecting the appropriate conjoint method, see the article in the technical papers library on our website entitled: "Which Conjoint Method Should I Use?" or use the Interactive Advisor at http://www.sawtoothsoftware.com/products/advisor/.

608

SSI Web v7

5.1.12

Staying out of Trouble with ACA

Here are some hints for staying out of trouble with ACA: Proper execution of the Importance questions: Over the last 10 years, it has become clear to us that the self-explicated importance question is the aspect of ACA that either "makes or breaks" it. If the importance questions are asked poorly, they can misinform utility estimation and be harmful. An example of poor execution is to not educate respondents ahead of time regarding the array of attributes, and to ask the importance questions one-by-one on separate screens. Some ideas for improving the quality of the importance questions are given in the section of this documentation entitled Customizing the Importance Question. Using too many prohibitions: ACA lets you specify that certain combinations of attribute levels shouldn't occur together in the questionnaire. But if you prohibit too many combinations, ACA won't be able to produce a good design, and may fail altogether. You can present combinations of levels that do not exist in the market today, and including unusual combinations can often improve estimation of utilities. During market simulations, you can avoid those combinations that seem unusual. In short, prohibitions should be used sparingly. Reversing signs of ordered attribute levels: If you already know the order of preference of attribute levels, such as for quality or price, you can inform ACA about which direction is preferred and avoid asking respondents those questions. You inform ACA by means of the a priori settings: Worst to Best, or Best to Worst. A common mistake is to accidentally specify the wrong order, which can lead to nonsensical data that can be difficult to salvage. To avoid this situation, take the interview yourself, making sure that the questions are reasonable (neither member of a pair dominates the other on all included attributes). Also, answer the pairs section with mid-scale values and then check to make sure the utilities are as you expect them. If you do happen to misspecify the a priori order of levels, the ACA/HB module can be used quite effectively to recompute the utilities and help salvage the situation. Using ACA for pricing research when not appropriate: There are three aspects to this point. All "main effects" conjoint methods, including ACA, assume that every product has the same sensitivity to price. This is a bad assumption for many product categories, and CBC or ACBC may be a better choice for pricing research, since they can measure unique price sensitivity for each brand.

When price is just one of many attributes, ACA may assign too little importance to it. In a few previously published articles, researchers have reported that it may sometimes be appropriate to increase the weight that ACA attaches to price. This is particularly likely if the author includes several attributes that are similar in the minds of respondents, such as Quality, Durability, and Longevity. If redundant attributes like these are included, they may appear more important in total than they should be, and other attributes, such as price, may appear less important than they really are. This problem is exacerbated if a wide range for price is specified.

It is not a good idea to use the "share of preference with correction for product similarity" with quantitative variables such as price. Suppose there are five price levels, and all products are initially at the middle level. As one product's price is raised, it can receive a "bonus" for being less like other products which more than compensates for its declining utility due to its higher price. The result is that the correction for product similarity can lead to nonsensical price sensitivity curves. This problem also can occur (but typically to a lesser degree) when using the improved method for dealing with corrections for product similarity: Randomized First Choice (RFC). We suggest conducting sensitivity analysis with the Share of Preference method when modeling demand curves for quantitative attributes like price. Using unequal intervals for continuous variables: If you use the ranking rather than the rating

ACA Help

609

option, ACA's prior estimates of utility for the levels of each attribute have equal increments. That works well if you have chosen your attribute levels to be spaced regularly, for example with constant increments such as prices of .10, .20, .30, or proportional increments such as 1 meg, 4 megs, or 16 megs. But if you use oddly structured intervals, such as prices of $1.00, $1.90, and $2.00, ACA's utilities are likely to be biased in the direction of equal utility intervals. This problem can be avoided if you use the ACA/HB module to compute ACA utilities. Including too many attributes: ACA lets you study as many as 30 attributes, each with up to 15 levels. But that doesn't mean anyone should ever have a questionnaire that long! Many of the problems with conjoint analysis occur because we ask too much of respondents. Don't include n attributes when n-1 would do! Including too many levels for an attribute: Some researchers mistakenly use many levels in the hope of achieving more precision. With quantitative variables such as price or speed, you will have more precision if you measure only 5 levels and use interpolation for intermediate values. If you must measure more than 5 levels, we strongly encourage you to use the ACA/HB module for estimating utility values. It can do a much better job at measuring those levels that are not studied in detail by any one respondent. Interpreting simulation results as "market share": Conjoint simulation results often look so much like market shares that people sometimes forget they are not. Conjoint simulation results seldom include the effects of distribution, out-of-stock, or point-of-sale marketing activities. Also, they presume every buyer has complete information about every product. Researchers who represent conjoint results as forecasts of market shares are asking for trouble. Not including adequate attribute ranges: It's usually all right to interpolate, but usually risky to extrapolate. With quantitative attributes, include enough range to describe all the products you will want to simulate. It is a good idea prior to data collection to ask the client to list the product scenarios that should be investigated in market simulations. This exercise can often reveal limitations or oversights in your attribute level definitions. Imprecise attribute levels: We assume that attribute levels are interpreted similarly by all respondents. That's not possible with "loose" descriptions like "10 to 14 pounds," or "good looking." Attribute levels not mutually exclusive: Every product must have exactly one level of each attribute. Researchers new to conjoint analysis sometimes fail to realize this, and use attributes for which many levels could describe each product. For example, with magazine subscription services, one might imagine an attribute listing magazines respondents could read, in which a respondent might want to read more than one. An attribute like that should be divided into several, each with levels of "yes" and "no."

610

SSI Web v7

5.2
5.2.1

Advanced ACA Questionnaires

Omitting the "Importance" Question
Background Although ACA has proven to be a useful and popular technique over two decades, many researchers have argued that the self-explicated importance section in the ACA "Priors" may be a weak link. The self-explicated importances can be confusing to some respondents, and the tendency for respondents to state that most attributes are important may flatten the final derived importances. Flattened importances mean that the effect of attributes in market simulations is biased toward greater similarity across attributes. As a result, ACA users sometimes report that critically important attributes tend to carry too little weight, and attributes of very little consequence tend to carry too much weight. This problem is exacerbated when there are very many attributes in the study. Field Experiments At the 2004 Sawtooth Software conference, we reported on an experiment in which we dropped the importance questions altogether. The experiment was conducted within an actual commercial study and included 20 total attributes and around 1500 respondents. (See "The 'Importance' Question in ACA: Can It Be Omitted?" available within our technical papers library at www.sawtoothsoftware.com/techpap.shtml). We tested two methods of obtaining prior attribute importance information for the purposes of selecting attribute combinations to display in the conjoint pairs: Assume equal prior attribute importances Assign prior importances based on the average derived importances from previous respondents We should note that in either case updating of part-worth utilities occurred per the standard practice for ACA: once conjoint pairs had been answered by the respondent, this respondent's answers were added to his/her prior information, and the updated part-worth utility estimates were used for selecting the next conjoint pair(s). We used ACA/HB to estimate final part-worth utilities, where the utilities were constrained by the within-attribute level ranking/ratings but the importances were not constrained. Therefore, the only information used for deriving the relative strength of attributes came from the conjoint pairs section. We found that dropping the importance questions... improved the prediction of shares for holdout choice sets. resulted in degraded predictions of individual choices (hit rates), resulted in more differentiated and different derived importances, reduced the interview time. And, we found that both methods of dropping the importances worked about equally well.

The margin of victory in predicting holdout shares would have been even greater had we used the time savings from dropping the self-explicated importance section to ask even more pairs. Most researchers and managers are more interested in the accuracy of aggregate share predictions rather than individual-level classification rates. And, if a key deliverable is a summary of respondents' preferences using part-worth utility or importance charts, dropping the importance questions should result in more valid displays. Therefore, there are advantages to dropping importances. The advantages come, however, at the expense of a bit larger sample size to stabilize the results compared to standard ACA. Also, we recommend using the saved time to ask additional pairs questions. We repeated the study in a different context (international study, with 10 attributes) and confirmed the

ACA Help

611

findings of the first study. Based on these two favorable outcomes, we are offering the ability to drop the importance question in the commercial version of ACA. We hope that providing this added functionality will encourage further research in this area. Using Importances from Prior Respondents We should note that in the second experiment referenced above we improved the method for using prior group importances for selecting the design in the conjoint pairs (relative to that used in the first experiment). Rather than use the previous respondents' group importances as a definitive rank order for arranging the attributes in the cyclical plan used for the first-stage 2x2 pairs, we assigned the rank order according to random draws from a rectangular distribution, such that the probability of an item achieving a high rank-order position was proportional to its prior importance score. This leads to greater variety in the conjoint pairs design (especially in the first pairs showing two attributes at a time, where the attributes are arranged in a circle, from best to worst), while still using information regarding the relative strength of the attributes for selecting paired product concepts with a reasonable degree of utility balance. Especially in cases where the researcher is taking a subset of the attributes into the pairs section, selecting the subset of most important attributes with likelihood proportional to prior importances ensures that even least important attributes (based on the prior group average) will still have a chance to be evaluated in the pairs section. Since part-worth estimation is performed using HB, the respondents that evaluate least important attributes in the pairs will contribute useful information for refining the estimates of these part-worths across the population. Once respondents have completed the interview, their final part-worth utilities (OLS estimation) are used to derive importance scores, and those importance scores are used to update the previous population importances (each respondent equally weighted). SSI Web writes a file containing updated group importances named STUDYNAMEavgimp.cgi on the server during data collection. This file is reset (to equal average importances) if the Reset Web Survey is selected in the admin module. Recommendations Generally, when you can share information across respondents on the same server (or CAPI installation) we'd recommend using the method of assuming prior importances based on previous respondents. We think that based on the improvement to the algorithm, this method will prove slightly better than assuming equal importances for priors. We again stress that if dropping the importance section, much of the time savings should be devoted to asking additional pairs questions. When you omit the importance questions and use ACA/HB for estimation, the appropriate setting is to fit Pairs only (fitting Priors is inappropriate), and to not use prior importances as constraints. However, even if you did try to use the importances as constraints, we write out importances that are equal for all respondents such that no between-attribute importance constraints would apply.

612

SSI Web v7

5.2.2

Customizing the "Importance" Question

The stated (self-explicated) importance questions in ACA have been the target of some criticism over the years (see "The 'Importance' Question in ACA: Can It Be Omitted?" available within our technical papers library at www.sawtoothsoftware.com/techpap.shtml). Some of the main arguments against the current implementation of an importance rating scale in ACA are: 1. Respondents tend to say that many attributes are very important, which can result in "flatter" final importances (lower discrimination) than reality. 2. Respondents may have a hard time understanding the framing of the question (assigning a weight to the difference between extreme levels). 3. There is evidence that respondents may not answer the questions as reliably as the withinattribute ratings (ACARAT questions). Starting with version 6 of SSI Web, the Importance questions can be customized or omitted altogether, drawing upon aggregate importances from previous respondents as "priors" to inform the pair generation algorithm and subsequent calibration concepts. With this approach, ACA/HB is required for proper part-worth utility estimation (since after dropping importances there is not enough information available at the individual level to estimate part-worths under standard OLS estimation). Some users, however, may not have the luxury of relatively large sample sizes needed to stabilize partworth estimates after dropping the Importance questions and furthermore may not be interviewing from a common server (or SSI CAPI installation) such that prior importance scores could be shared and updated across respondents. To help overcome some of the issues noted above, some researchers may wish to substitute customized self-explicated importance questions and have ACA use that information for informing the subsequent pair generation algorithm, calibration concepts, and final partworth utility estimation under OLS. Custom Importance Question Ideas One example that may work well is to first educate respondents about the full list of attributes (including the range of levels for each attribute), and then ask the respondent to identify the one most important attribute. Then, the researcher asks the respondent to assign a "10" to that most important attribute, and to rate the other attributes with respect to that most important attribute. Respondents could be instructed that if an attribute is half as important as the most important one, they should give it a "5." As another example, assume that ACA is being used to interview just a few key business managers within an organization regarding the desirability of different projects or capital expenditures. Further suppose that these managers are analytically sophisticated and are quite comfortable with the notion of assigning weights to attributes using a constant-sum (chip-allocation) question. Under these conditions, the researcher might believe that these particular respondents will provide better information if using a constant-sum allocation of importance across attributes rather than the standard ACA approach for asking about attribute importance. Implementing Custom Importance Questions To implement a customized importance questions, click the Set Prior Importances Based on other Questions/Variables option, and then click the Specify Questions... button. For each attribute in the table, specify a question name (or variable value). For example, CS_1 returns the value contained in variable CS_1. This might refer to a constant sum question named CS where the first response option (_1) refers to the first attribute. The customized importance scores must contain only positive numeric values. ACA assumes these

ACA Help

613

positive values have ratio scaling (i.e. that a score of 10 is twice as important as a score of 5). For internal utility calculation purposes, the importance data are rescaled to the range 0 and 7. The rescaled importance values (range of 0 to 7) are stored in the dat.dat file in place of the "skipped" variables Exercise_Importance1 through Exercise_Importancen. Variables containing customized importance information can come from most any question type within SSI Web, asked at any point in the survey, including Free Format and hidden variables. They may even be passed into the questionnaire through the "one-click access to survey."

614

SSI Web v7

5.3
5.3.1

Algorithms and Estimation Techniques

Details of ACA/Web Utility Estimation
The following describes ACA's method of part-worth utility computation using OLS (Ordinary Least Squares regression). ACA includes two major sections: self-explicated priors and conjoint pairs. Part-worth utility estimates for these two sections are determined as follows:

Prior Utilities:
If rank orders of preference are asked (not currently offered in ACA) we convert them to relative desirabilities by reversing them. For example, ranks of 1, 2, and 3 would be converted to values 3, 2, and 1, respectively. If desirability ratings are asked (only method offered in ACA), those are retained "as is." The average for each attribute is subtracted to center its values at zero. For example, desirability values 3, 2, and 1 would be converted to 1, 0, and -1, respectively. The values for each attribute are scaled to have a range of unity. For example, desirability values of 1, 0, and -1 would be converted to .5, 0, and -.5. The importance ratings for each attribute are scaled to range from 1 to 4, and then used as multipliers for the unit-range desirability values. Thus, if an attribute has desirabilities of .5, 0, and -.5 and an importance of 3, we get -1.5, 0, and 1.5. The resulting values are initial estimates of part-worths, with these characteristics: For each attribute the range of utility values is proportional to stated importance, and attribute importances differ by at most a factor of 9 (the maximum number of scale points that can be specified for importance questions). Within each attribute the values have a mean of zero, and differences between values are proportional to differences in desirability ratings or rank orders of preference.

Pairs Utilities:
An independent variable matrix is constructed with as many columns as levels taken forward to the pairs questions. If a level is displayed within the left concept, it is coded as -1; levels displayed within the right-hand concept are coded as +1. All other values in the independent variable matrix are set to 0. A column vector is created for the dependent variable as follows: the respondents' answers are zerocentered, where the most extreme value for the left concept is given a -4, and the most extreme value on the right +4. Interior ratings are fit proportionally within that range. Each pairs question contributes a row to both the independent variable matrix and dependent variable column vector. Additionally an n x n identity matrix is appended to the independent variable matrix, where n is the total number of levels taken forward to the pairs questions. An additional n values of 0 are also appended to the dependent variable matrix. The resulting independent variable matrix and dependent variable column vector each have t + n rows, where t is the number of pairs questions and n is the total number of levels taken forward to the pairs questions. OLS estimates of the n attribute

ACA Help

615

levels are computed by regressing the dependent variable column vector on the matrix of independent variables.

Combining the Priors and Pairs Utilities:

The priors and pairs part-worths are normalized to have equal sums of differences between the best and worst levels of each attribute across all attributes. (Note that the procedures described above automatically result in zero-centered part-worths within attribute for both sets of part-worths.) The prior part-worth utilities for levels also included in the pairs questions are multiplied by n/(n+t), where n is the total number of levels used in the Pairs section, and t is the number of pairs questions answered by the respondent. Any element in the priors that was not included in the Pairs section is not modified. The pairs utilities are multiplied by t/(n+t). The two vectors of part-worths (after multiplication by the weights specified above) are added together. These are the final part-worths, prior to calibration. As a final step, the part-worth utilities are calibrated. It is widely recognized that the part-worths arising from most conjoint methods are scaled arbitrarily, and that the only real information is contained in the relative magnitudes of differences among them. So far, that is true of ACA as well. However, the calibration concepts permit scaling of part-worths in a non-arbitrary way. In any product category, some respondents will be more interested and involved than others. We attempt to measure each respondent's degree of involvement by asking "likelihood of buying" questions for several concepts that differ widely in attractiveness. The data obtained from those concepts is useful in three ways: Correlations between part-worths and likelihood responses may be used to identify unmotivated or confused respondents. Respondents whose likelihood responses are not related to their part-worths should probably not be included in subsequent preference simulations. The level of likelihood responses may identify respondents who are more or less involved in the product category. Respondents who give low likelihood responses even to concepts custom-designed for them should probably be treated as poor prospects in simulations of purchase behavior. Variation in likelihood responses may also identify respondents who are "tuned in" to the product category. A respondent who gives a low likelihood rating to the least attractive concept and a high rating to the most attractive should be made to respond sensitively in preference simulations, whereas someone who gives every concept similar likelihood values should be made insensitive in simulations. Each respondent is first shown what should be the least attractive possible concept, followed by the most attractive possible concept, as determined from his or her own answers. Those two concepts establish a frame of reference. The remaining concepts are of middling attractiveness. We determine an intercept and one regression coefficient to apply to utilities to best predict logits of likelihood responses. Those parameters are then used in a final scaling of utilities, which are therefore no longer arbitrarily scaled. The procedure is as follows: Let: p = x1 = b1 = a = the predicted likelihood of buying a concept the concept's utility based on the final "uncalibrated" utilities the coefficient used to weight the utilities an intercept parameter

The actual likelihood response is a single digit on a scale with n points. Responses are trimmed to the range of 5 to 95. Then, using the logit transformation, we model buying likelihood as a function of the respondent's utilities as:

616

SSI Web v7 ln [ p / (100 - p) ] ~ a + b1x1 If the regression coefficient is less than 0.00001, we assume the estimation is faulty and use a conservative positive value (0.00001). The r-squared (measure of fit) reported in the .UTL file is set to 0 in such cases. If the calibration concepts section is not included in the interview, the respondent is assumed to have answered 0 and 100 to the worst and best concepts, respectively, and 50 to the other concepts. To calibrate the part-worths, each is multiplied by b1. The intercept a is divided by the number of attributes, and the quotient added to the part-worth for every attribute level. The part-worths can be added up and antilogs of their sums are predictions of odds ratios for claimed likelihood of purchase of any concept, just as though that concept had been included in the calibration section of the questionnaire.

A Note about Hierarchical Bayes Estimation

OLS has been successfully used in ACA calculations for well over a decade. However, a new technique called hierarchical Bayes estimation provides a more theoretically satisfying way of combining information from priors and pairs. The results are also usually better from the practical standpoint of improved predictions of holdout questions. We recommend that the interested reader investigate ACA/HB by downloading the technical paper from our Web site ( http://www.sawtoothsoftware.com/download/techpap/acatech.pdf).

ACA Help

617

5.3.2

Choosing the Next Paired-Comparison Question

The first part of an ACA interview is concerned with screening the attribute levels and learning enough about the respondent's preferences to construct initial part-worth estimates. After that is done we begin the paired-comparison section, in which pairs of concepts are shown and preference questions are asked. Following each response (submitted page of pairs questions) we update our estimates of part-worths and then decide what pair of concepts to present next. The number of possible concepts is very large, and we need some reasonably efficient procedure to choose a pair of them at each stage that will be most beneficial in some way. There are several principles to keep in mind when thinking about how to choose concepts. Concepts should be chosen by a method that gives the author as much control as possible over the interview, in terms of the complexity of the concepts and the number of questions asked. The design should be as "balanced" as possible. Observations should be spread as evenly as possible over all attribute levels, and the columns of the design matrix should be as nearly orthogonal as possible. We should ask the respondent questions that require careful consideration. There is no point in asking questions for which we already know the answer, such as "High quality at a low price" versus "low quality at a high price." We learn more if we choose concepts nearly equal in attractiveness. Our procedure addresses these points. The author may specify the number of attributes to appear in each concept. The range is from two to five. It is possible to start with only two attributes per concept and, after the respondent has gained experience, to increase their complexity. The concepts in a pair always have different levels of the same attributes. Our procedure for choosing those concepts is: Count the number of times each pair of attributes has appeared together in any concept. Pick a set of attributes at random from among those whose members have previously appeared together the fewest times. For each of the chosen attributes, repeat similar logic to find levels that have been paired least frequently. Examine all possible ways of combining these levels into concepts (with just two attributes there are only two possible ways; with 5 attributes there are 16 ways. Find the pair of concepts most nearly equal in attractiveness, using the current estimates of the respondent's part-worth utilities. Randomly determine which concept will appear on each side of the screen. Accordingly, ACA presents pairs of concepts that are as nearly equal as possible in estimated utility. At the same time, constraints are imposed to ensure that the overall design is nearly orthogonal. Within concepts, each pair of attributes is presented with equal frequency, and within each attribute, each pair of levels is presented with equal frequency. In addition, if the paired-comparison questions show only two attributes at a time, further steps are taken to insure that the overall design is "connected." Such an approach has these benefits: 1) It gives the respondent the impression that the system is paying attention to his or her answers, and it seems to be asking increasingly insightful questions.

618

SSI Web v7 2) It helps keep the respondent operating within the defined range of a response scale rather than at its ends. 3) It provides data on "tight" inequalities if estimation is later to be done by nonmetric methods. ACA lets the author specify certain pairs of attribute levels that must not appear together in the same concept. The procedure described above is modified slightly to take account of such prohibitions. When concepts are described on only two attributes, ACA chooses the first few questions in a slightly different way. (When the concepts are described on only two attributes, it would be possible to blunder into a design in which the attributes would be divided into subsets in such a way that those in one subset would never be paired with those in another subset. Such designs would provide no information about the relative importance of attributes in different subsets, and ACA automatically corrects the design in such a situation.) ACA's designs usually have good statistical efficiency, although they are not strictly orthogonal. Statistical efficiency is increased as more attributes are used in each concept, and it is also possible to produce concepts more nearly equal in attractiveness when there are more attributes with which to work. However, using larger numbers of attributes has the unfortunate consequence of making the questions more complicated, and respondents are more easily confused. Both anecdotal and experimental evidence has shown that it is usually best to start with only two attributes per concept and, after a few pairs, to increase the number of attributes to three. Beyond three attributes, gains in efficiency are usually offset by respondent confusion due to task difficulty.

ACA Help

619

5.3.3

Updating Utilities During the Interview

Estimates of the respondent's part-worth utilities are updated after each page of paired-comparison responses is submitted. First consider the general case of how least squares regression coefficients can be updated to include the effect of an additional observation. Let X be a matrix of predictor variables with a row for each of n observations and a column for each variable. Let y be a vector of responses for the first n observations. Let z' be a row vector of predictor values for a new observation, appended as a row to X. Let r be a response for the new observation. Considering only the first n observations, we have the regression equation: Xb ~ y n

where -1 b = (X'X) n (X'y) (1)

is the vector of coefficients that would be obtained by least squares estimation based on the first n observations. Now consider adding one observation. The expanded layout is: | X | | | b ~ | z'| n+1 where -1 b n+1 ~ = (X'X + z'z) (X'y + zr) | y | | |' | r |

(2)

is the least squares estimate based on n+l observations. Suppose we already have bn , X, y, z, and r, and we want to obtain bn+1. First consider an identity. Let -1 v = (X'X) Z (3)

Then it can be shown that

620

SSI Web v7 -1 (X'X + z'z) = -1 (X'X) v'v ----1 + v'x

(4)

Substituting into equation (2), we get r - z'b n b = b + v n+1 n -------1 + v'z (5)

Equation (5) gives a formula for updating the estimate of part-worths following each response, a relatively easy computation since the numerator and denominator on the right are both scalars. We must also update the inverse as in equation (4). That is also fairly easy since the vector v is already available. If we are dealing with k attribute levels, then an updating cycle requires about 2k (k + 1) multiply and add operations. This is a significant savings when compared to the cost of re-estimating "from scratch" after each response, and the final results are identical. Now consider how this scheme is applied to the specific situation in ACA: Before the first updating we set X equal to the identity matrix and both bn and y equal to the initial utility estimates. The vector z consists of plus and minus 1's and 0's. An element equals 1 if the corresponding attribute level appeared in the concept on the right of the screen, -1 if in the concept on the left of the screen, and 0 if that level did not appear in either concept. The response r is coded so that +4 means "strongly prefer right," -4 means "strongly prefer left," and 0 means indifference.

ACA Help

621

5.3.4

ACA/HB Estimation
HB estimation is considered the "gold standard" for ACA part-worth utility estimation, exceeding the quality of the default OLS estimation. The benefits of HB estimation for ACA include: 1. 2. 3. 4. Greater precision of estimates for each individual, It relaxes the assumption of equidistant part-worths in the Priors for a priori ordered attributes, Improved accuracy of part-worths for predicting holdout concepts, It provides a theoretically more defensible approach for combining self-explicated and conjoint data.

We recommend that the interested user refer to the ACA/HB documentation (or the ACA/HB Technical Paper, available in our Technical Papers library at www.sawtoothsoftware.com) for more details about ACA/HB and its settings dialog. Almost all of the capabilities of the standalone ACA/HB product are available within the SSI Web interface, except: 1. 2. 3. 4. Ability to save draws Unacceptable extrapolation Ability to restart interrupted HB runs Advanced output files are not written (.dra, .cov, .hbv, .log)

Output Files When HB estimation is completed, the output is also displayed in a report window. The first (Summary) tab displays the average part-worth utilities across respondents (normalized by the zerocentered diffs method), and the average attribute importances. On the second (Individual Utilities) tab are the raw utility data for each respondent, organized as one row per respondent. Click the Save As... button to save the data as an Excel (.XLS) file. Zero-Centered Diffs Method: The Summary tab displays the average rescaled utilities (as well as the standard deviations) under the zero-centered "diffs" method. The diffs method rescales utilities so that for each individual the total sum of the utility differences between the worst and best levels of each attribute across attributes is equal to the number of attributes times 100. This normalizes the data so that each respondent has equal impact when computing the population average. When you run ACA/HB through the SSI Web interface, a studyname ACA.hbu file is also automatically generated. The layout of this file is supplied in the ACA/HB manual. This file may be directly imported into the SMRT program for market simulations.

622

SSI Web v7

5.4
5.4.1

ACA File Formats

Format of .UTL and .ACD Files
The .UTL File
The .UTL file contains the estimated part-worths for each respondent. Each respondent's data are formatted across multiple lines. An example is as follows:
104 9.465 -9.999 -2.473 935 0 23 7 9.746 -9.999 -9.999 -9.999 -9.999 -9.999 -6.029 -4.550 -9.999 -1.665 -2.108 -1.764 -1.305 -4.172 -1.275 -2.473 -0.547 -4.131 -0.635 -3.254

Following is a description of each line: Line Line 1 Field 1 2 Description Respondent Number Correlation Notes Number identifies record A "-1" indicates Calibration Concepts were not shown and as a result a correlation was not calculated; consequently the Purchase Likelihood model cannot be used for simulations Always zero when generated by ACA Number of levels in record Always a zero in ACA

3 4 5 Line 2+ 1 2 ...

Merged Variables Number of Utility Levels Interview Duration

First Utility Level Second Utility Level Remainder of Utility Levels

Value of level Value of level And so on, for all existing utility levels.

There can be up to 18 lines of part-worth utility levels; each line has 8 columns per level, up to 10 levels per line; up to a maximum of 180 levels. Each value has 3 places after the decimal point.

The STUDYNAME.ACD File (Audit Trail)

The STUDYNAME.ACD file contains an "audit trail" of the questionnaire for each respondent. The file contains information about which questions were presented and about the responses. In earlier versions of ACA, such as ACA v4 for DOS, this file also contained three sets of part-worths, computed at various stages of the interview. We expect that most ACA users will use the .ACD file generated by ACA with ACA/HB (Hierarchical Bayes estimation). ACA/HB does not use the three sets of part-worths available in previous versions of the .ACD file, so we have inserted "0.000" values to maintain compatibility with the prior .ACD file format. There is one record for each respondent, in the order respondents appear in the STUDYNAME.UTL file. Following is an example of one respondent's record in the audit trail file:

ACA Help
104 12 5 5 1 4 1 4 7 6 4 1 3 14 9 2 1 2 16 17 3 2 3 19 23 1 6 5 -1 -7 2 8 0 -6 -10 7 9 0 -9 -16 10 17 0 -17 -20 15 18 0 -1 -20 2 21 0 -6 -10 -19 8 9 -1 -16 -19 2 15 -1 -10 -16 2 9 -6 -10 -20 7 9 -8 -17 -21 6 16 -1 -7 -10 -15 2 -1 -9 -16 -19 2 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 4 5 1 6 9 15 21 2 7 10 16 17 1 7 10 16 17 2 6 9 14 20 0.000 0.000 0.000 21 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 7 0 0 0 0 0 6 0 0 0 0 0 3 0 0 0 0 0 4 0 0 0 0 0 8 18 0 0 0 0 4 17 0 0 0 0 7 15 0 0 0 0 7 21 0 0 0 0 7 17 0 0 0 0 7 8 9 17 0 0 7 10 15 21 0 0 7 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 12 99 24 55 3 2 5 1 4 2 0 4 0 3 5 0 3 0 2

623

First line: The first line contains the respondent number (104); the number of pairs that were shown in the Pairs section of the interview (12), the number of attributes in the study (5); and the number of attribute levels in the study (21). Attribute levels are assigned consecutive numbers. Numbers are assigned beginning with the levels of the first attribute, then the levels of the second attribute, and so on. For example, the chart below shows how numbers are assigned to the levels of the attributes in the questionnaire that generated this data file:
Body Type Two-Door Sedan (1) Four-Door Sedan (2) Hatchback (3) Minivan (4) Convertible (5) Configuration Rear-Wheel Drive (6) Front-Wheel Drive (7) All-Wheel Drive (8) Color Manufacturer Price Red (9) US Manufacturer (14) $16,000(17) Blue (10) Japanese Manufacturer (15) $18,000(18) Green (11) European Manufacturer (16) $20,000(19) Yellow (12) $22,000(20) White (13) $24,000(21)

Second and next lines, the Importance Ratings: Next follows one line for each attribute. Each line has 12 columns (unless there are more than 9 levels for any attribute, in which case the layout is expanded to 18 columns). These 12 columns allow for the maximum number of levels to be displayed (9), the two attribute levels that were shown, and the respondent's answer to the Importance question. The numbers in the first and second columns in each line represent the attribute levels that were shown in the Importance Rating section. The number in the third column is the respondent's answer to the Importance question. Then, the remaining 9 columns have numbers for each level that show the preference rating for each level, in order, for as many levels as exist for the attribute. These numbers

624

SSI Web v7 are ratings, where the "1" indicated the least-preferred level. Any remaining columns contain zeros. Using our example record that has 5 attributes, the lines just described are lines 2-6:
5 7 14 16 19 1 6 9 17 23 4 4 2 3 1 1 1 1 2 6 4 3 2 3 5 3 2 5 1 4 2 0 4 0 3 5 0 3 0 2 0 0 6 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

The second line represents the second attribute (Drive Configuration); the first number, 7 (Front-Wheel Drive), and the next number, 6 (Rear-Wheel Drive) represent the attribute levels that were shown in the Importance Rating Section. The third number, 4, is the respondent's answer to the Importance question. The next numbers, 1, 3, and 2, indicate that the first level (Rear-Wheel Drive) was least preferred (1), the second level (Front-Wheel Drive) was most preferred (3), and the preference for the third level (All-Wheel Drive) was in between (2). The second attribute has only three levels, so the remaining columns on this line contain zeros. Next lines, the Paired-Comparisons: Next follows one line for each of the pairs that were shown in the Pairs section of the interview. Each line has 11 columns. These 11 columns allow for the maximum number of levels to be displayed in the Pairs (5 in each of 2 concepts) and one column for the respondent's answer. The numbers represent the levels that were shown. Numbers with negative signs indicate that the level appeared on the left side of the screen, and those without negative signs indicate that the level appeared on the right side of the screen. The final number on the line is the respondent's answer. Using our example record that has 12 pairs, the lines just described are 7-18:
-1 -6 -9 -17 -1 -6 -1 -1 -6 -8 -1 -1 -7 -10 -16 -20 -20 -10 -16 -10 -10 -17 -7 -9 2 8 7 9 10 17 15 18 2 21 -19 8 -19 2 -16 2 -20 7 -21 6 -10 -15 -16 -19 0 0 0 0 0 9 15 9 9 16 2 2 0 0 0 0 0 18 17 15 21 17 8 10 0 0 0 0 0 0 0 0 0 0 9 15 0 0 0 0 0 0 0 0 0 0 17 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 6 3 4 8 4 7 7 7 7 7 7

The first of those lines represents the first pair; the first number, -1, indicates that level number 1, TwoDoor Sedan, appeared on the left of the screen. The second number, -7, indicates that level number 7, Front-Wheel Drive, appeared on the left of the screen. Those levels were paired with the next two numbers, which appeared on the right of the screen: 2 (Four-Door Sedan) and 8 (All-Wheel Drive). The final number in the line, 7, indicates the response to this Pairs question. Next lines, the Utilities Before Calibrations: In ACA v4, the next lines represent the part-worths for each level of each attribute, in order, for this respondent, prior to calibration. In ACA, we insert "0" values as placeholders. Each line has up to 10 part-worths. Using our example record that has 23 attribute levels, the lines just described are 19-21:
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

ACA Help The values are displayed for each level. Next lines, the Prior Utilities:

625

In ACA v4 the next lines represent approximate part-worths for each level of each attribute, in order, for this respondent. These part-worths are based only on the information from the interview prior to the Pairs section. In ACA, we insert 0 values as placeholders. Each line has up to 10 part-worths. Using our example record, the lines just described are 22-24:
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

The values are displayed for each level. Next lines, the Contribution to the Utilities from the Pairs Section: In ACA v4 the next lines represent part-worths for each level of each attribute, in order, for this respondent. These are the components of the part-worths from the Pairs section. In ACA, we insert 0 values as placeholders. Each line has up to 10 utilities. Using our example record, the lines just described are 25-27:
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

The values are displayed for each level. Next line, Number of Calibration Concepts and Number of Attributes: The next line contains two numbers that report the number of Calibration Concepts shown to the respondent and the number of attributes in each concept. In our example record, this is line 28:
4 5

The first number, 4, indicates that 4 concepts were shown. The second number, 5, indicates that there were 5 attributes in each concept. Next lines, Calibration Concepts: Next, there is one line for each calibration concept. For as many columns as there were attributes in each concept, numbers indicate the attribute levels included in the concept. Then, the next column shows the calculated utilities for the concept (in ACA we set those to 0.000). The last column contains the respondent's purchase likelihood value for the concept. In our example record, the lines just described are lines 29-32:
1 2 1 2 6 7 7 6 9 10 10 9 15 16 16 14 21 17 17 20 0.000 0.000 0.000 0.000 12 99 24 55

The previous line, 28, indicated there were 4 concepts, so there are 4 lines (29-32 ), one for each concept. Line 28 also indicated there were 5 attributes in each concept, so each of the 4 lines uses the first 5 columns to show which attribute levels were included in that concept. For example, the first five

626

SSI Web v7 numbers in line 29 indicate these levels were shown: 1 (Two-Door Sedan), 6 (Rear-Wheel Drive), 9 (Red), 15 (Japanese Manufacturer), and 21 ($24,000). In ACA v4, the next number indicates the total utility for the concept. In ACA we put a value of 0.000 there as a placeholder. The last number, 12, represents the respondent's purchase likelihood value for the concept. Last line, Calibrating Coefficients: In ACA v4 the last line contains the calibrating coefficients and measure of fit from the logit regression, which is used to predict likelihoods of purchase and to weight the priors and pairs components when estimating "optimal" utilities. In ACA we insert "0.000" as placeholders for those values. In our example record, the line just described is line 33:
0.000 0.000 0.000 0.000

ACA Help

627

5.5

Real-Time Feedback to Respondents

There are some situations in which you might want to show respondents their attribute importances and/or part-worth utilities at the completion of an ACA survey. ACA enables this by making the partworths and attribute importances available as system functions that you can display on the screen. We suggest the following steps: 1. Generate a Text question after the ACA questions. This question can be in any position following the last ACA Pairs question. This question will be used to display the ACA utilities report. 2. There is a template (HTML) text file installed with your system called acarpt.htm. Make a copy of that file and modify it with a text editor (such as Wordpad or Notepad) to suit your list of attributes and levels. Follow the instructions and patterns in that template file (it is helpful to know something about HTML coding when working with this file.) Copy the coding patterns as many times as needed to match your attribute list. You will note that the template refers to system functions that return part-worth utility and importance values. The attribute importances are returned with the function ACAImportance(Attribute#) (or you may use ACAStrictImportance(Attribute#) which controls for reversals). The level part-worths are returned using the function ACAUtility(Attribute#, Level#). In both cases, you will probably want to round them to the nearest integer for display purposes (using the Round function). You can use these functions at any time during an ACA interview (following the priors questions). If you are using the function within a text field (such as Header1, Header2, etc.) that uses HTML instructions, you should enclose the function within [% ... %]. [% ACAImportance(ExerciseName, 4) %] (Returns the value for the importance of attribute 4) [% ACAUtility(ExerciseName, 6, 3) %] (Returns the value for the utility of the third level of attribute 6) [% Round (ACAImportance(ExerciseName, 4)) %] (Returns the value for the importance of attribute 4, rounded) [% Round (ACAUtility(ExerciseName, 6, 3)) %] (Returns the value for the utility of the third level of attribute 6, rounded) We suggest that you open (view) the template file with your browser to check your work as you modify that file. Once you are happy with the look of your report, you are ready to paste the HTML text into the Text question you have defined in the ACA questionnaire. 3. Using a Windows editor such as Word, Wordpad or Notepad, highlight the appropriate text of the renamed template HTML file you've been editing and cut-and-paste that text into either the Header, Footer or Body section of the Text question you generated in Step 1 above. When the survey runs on the Web, values will be inserted in place of the functions, and the bars in the bar chart for importances will extend the appropriate lengths to reflect the relative importance of each attribute. Notes: Utilities are reported as "zero-centered diffs." Importances are reported as values that sum to 100. The utilities are generated by the efficient updating routine used when the ACA Survey runs on the Web. These utilities are very similar to, but not exactly the same, as the final utilities generated by the ACA OLS Utility Calculator. For each page that includes a report of ACA utilities or importances, the utilities are re-computed "from scratch". This can be computationally intensive for the server. If you show these values over many pages, the computations must be repeated for each page.

628

SSI Web v7 If you change the number of attributes, levels in your study, or the text for attributes/levels, make sure to manually change the HTML report to reflect those changes. There are so many aspects about ACA that are automatic that it is easy to forget to do this.

ACA Help

629

5.6

Moving Data from SSI Web to SMRT for Analysis

If conducting conjoint analysis studies (CBC, ACBC, ACA and CVA) with SSI Web, you'll often want to move the data from SSI Web into the SMRT software package for analysis (market simulations, counting analysis, logit, etc.). Sometimes data are moved directly into SMRT, and other times data are moved from SSI Web into a preliminary analysis program such as hierarchical Bayes, and then afterward to SMRT.

CBC (Choice-Based Conjoint) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. Export the CBC data (just the data directly involving the choice tasks) to a studyname.CHO (text-only format) file. From the SSI Web menu, click File | Export Data | Prepare CBC Data files (*.cho/*.chs and *.att). 2. Start the SMRT software (a different program from SSI Web, also downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the .CHO file into SMRT, by clicking (from the SMRT software menu) File | Import, browsing to your .CHO file, and clicking Import. The raw choice data (respondent answers and experimental design) are now within the SMRT study. You can proceed with counting analysis or logit within SMRT. Note: If you use CBC/HB or Latent Class software to estimate part-worth utilities, use the .CHO (or .CHS, in the case of constant sum responses to be analyzed with CBC/HB) and .ATT files with those systems (see their respective manuals for details) to produce a data file containing respondent partworths (a .HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .HBU or Latent Class probability file (.Pxx). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

ACBC (Adaptive Choice-Based Conjoint) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. Estimate part-worth utilities within SSI Web by clicking Analysis | Calculate ACBC Utilities Using HB.... This process creates a text-only format file called STUDYNAME.hbu. 2. Start the SMRT software (a different program from SSI Web, also downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new

630

SSI Web v7 study functions independently of your original SSI Web study. 4. Import the STUDYNAME.hbu file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CBC utilities (selecting .HBU file type). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use CBC/HB or Latent Class software to estimate part-worth utilities, use the .CHO (or .CHS, in the case of constant sum responses to be analyzed with CBC/HB) and .ATT files with those systems (see their respective manuals for details) to produce a data file containing respondent partworths (a .HBU file if HB analysis, or .LCU and .Pxx files if Latent Class analysis). Then, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .HBU or Latent Class probability file (.Pxx). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

ACA (Adaptive Conjoint Analysis) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. If using the default OLS (Ordinary Least Squares) part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate ACA Utilities using Ordinary Least Squares. This process creates a text-only format file called STUDYNAME.utl. 2. Start the SMRT software (a different program from SSI Web, downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing ACA utilities (selecting ACA v4.x Utilities *.utl as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use ACA/HB software to estimate part-worth utilities, you can either run ACA/HB as a separate program, or if you own ACA/HB v3 or later, you can run the program as an integrated component from within the SSI Web interface. If running the estimation outside of SSI Web, from the SSI Web menu, select File | Export Data | Prepare ACA Data Files (*.acd and *.aca)... to export the ACA information to a STUDYNAME.acd format into the same folder containing your ACA/HB system. See the ACA/HB manual for details of producing a data file containing respondent part-worths (a .HBU file). After estimating part-worth utilities using either ACA/HB approach (within SSI Web menu or outside program), import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .hbu file. Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

ACA Help

631

CVA (Conjoint Value Analysis) Data

When you download SSI Web data from the web server, your data reside in your SSI Web project folder in a text-only format STUDYNAMEdat.dat file. The steps for moving data into SMRT are: 1. If using the OLS (Ordinary Least Squares) part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate CVA Utilities using Ordinary Least Squares. If using the Monotone Regression part-worth utility estimation method, from the SSI Web menu, click Analysis | Calculate CVA Utilities using Monotone Regression. Either process creates a text-only format file called STUDYNAME.utl. 2. Start the SMRT software (a different program from SSI Web, downloadable from our website), by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 3. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 4. Import the STUDYNAME.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CVA utilities (selecting CVA Traditional Conjoint Utilities *.utl as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Note: If you use CVA/HB software to estimate part-worth utilities, run the program as an integrated component from within the SSI Web interface. After estimating part-worth utilities using the CVA/HB approach, import the resulting utility file into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts to browse to and select your .hbu file. Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

Moving Segmentation (Generic Survey) Data from SSI Web to SMRT

We assume you have first followed the procedures above for estimating part-worth utilities and establishing those as a utility run in a new project within SMRT. 1. From the SSI Web system, export the data you wish to merge into SMRT. From the SSI Web menu, select File | Export Data | All Data, select the respondent numbering method (the numbering method must match the method you chose when exporting the conjoint data or estimating utilities), and export the data you wish to merge into SMRT (only numeric data may be merged). Select either a .CSV file (comma-delimited text-only file) or a fixed text-only file (.TAB). 2. Within the SMRT software, open the project previously established that contains the conjoint data/utility run. Use File | Merge Variables | Add to merge the data into your SMRT project (see the SMRT manual or online help within SMRT for details). After merging the data, if using SMRT v4.11 or earlier, you'll additionally need to specify Custom Segmentation Variables (Under Analysis | Custom Segments) if you wish to use the new variables as banner points or respondent filters during simulations.

632

SSI Web v7

5.7

Analyzing ACA Data with the Market Simulator

You were given a companion piece of software called the Sawtooth Software Market Simulator, which is also referred to as SMRT. You use this system to analyze your ACA part-worth utilities and to run market simulations (what-if scenarios). Follow the instructions in the Sawtooth Software Market Simulator manual to import and analyze your ACA data. To get started: 1. Open the SMRT application. If you have a default installation, you start the application by choosing Start | Programs | Sawtooth Software | Sawtooth Software SMRT. 2. Choose File | New to define a new study. If asked, specify ACA (Adaptive Conjoint Analysis) as the type of study. 3. Select Analysis | Run Manager. Click the Import button. Under Files of Type, specify ACA Utilities (.UTL). Browse for the .UTL file that the utility calculator generated, highlight it, and click the Open button to import the data. 4. Once the utility data have been imported, choose Analysis | Market Simulator to start the Market Simulator.

CVA Help

633

6
6.1
6.1.1

CVA Help
Getting Started with CVA
Introduction to CVA
Overview
The CVA System is Sawtooth Software's package for traditional full-profile conjoint analysis. The traditional full-profile conjoint method was the first "flavor" of conjoint used by marketing academics in the early 1970s (Paul Green and colleagues). It continues to be useful today, nearly 40 years after its first introduction. CVA (also known as CVA) is a component within Sawtooth Software's SSI Web platform. CVA ("Conjoint Value Analysis") includes: A designer for creating the stimuli (product profiles) displayed in conjoint interviews A questionnaire writer for formatting the conjoint questions for computer-based interviews or paper-and-pencil surveys Two utility calculators for estimating each respondent's part-worth utilities (strengths of preference) for product features Access to the standard SMRT market simulator that is also used in ACA (Adaptive Conjoint Analysis), CBC (Choice-Based Conjoint), or ACBC (Adaptive CBC). Optionally, users may purchase the CVA/HB add-on module for hierarchical Bayesian estimation of part-worth utilities. This add-on module functions seamlessly within the SSI Web interface. CVA is one of four conjoint software packages from Sawtooth Software. The other three are CBC, Adaptive CBC (ACBC), and ACA. We have designed each package to bring unique advantages to different research situations.

About Conjoint Analysis

Conjoint analysis is useful for learning how potential buyers of a product or service value its various aspects or features. Conjoint analysis is equally useful for studying products and services (for convenience, we use the word "product" in both cases). Examples of questions that lead market researchers to use conjoint analysis are: Which would be more valuable to the customer: longer warranty or more product performance? What would be the effect of raising our price 10%? What would happen if we added another product to our line that had 20% less performance but a 30% lower price? When using conjoint analysis, a product is thought of as being made up of various attributes (sometimes called "factors"). For example, automobile attributes may be: manufacturer, color, type of engine, number of doors, and price. Each attribute has several possible levels. For example, the levels of "number of doors" may be: two, four, and five (to include station wagons). The levels of "engine type" may be: 4 cylinder, 6 cylinder, and 8 cylinder. (The word "feature" is often used in place of "attribute level.")

634

SSI Web v7 Conjoint data are collected from survey respondents by asking about their preferences for hypothetical product concepts, described in terms of specific attributes and levels. In analyzing the data, one infers the value to each respondent of having each possible attribute level. These estimated values are called part-worths or utilities. With CVA, we can estimate each respondent's part-worth utility values for each level of each attribute from data collected by the conjoint questionnaire. These values are then used to make predictions about how respondents would choose among new or modified products. The goal is to conclude what product changes would have the most beneficial (or least detrimental) effect on share of preference, or which would maximize the likelihood that buyers would choose specific products. In ratings-based conjoint analysis, product concepts can be shown to respondents one at a time, or they can be presented in pairs. Pairwise presentation can be harder for the respondent, because each question requires understanding two concepts rather than just one. However, the comparative nature of the pairwise task may let the respondent make finer distinctions and contribute more information than single concept (popularly called card sort) presentation. The term full profile refers to product concepts that are described with respect to all the attributes being studied. For example, if the products in a particular category had six important attributes, a full profile product concept would describe the product on all six attributes. CVA produces either single concept or pairwise full profile designs. It is questionable that respondents can consider product concepts carefully that are described using more than about six attributes in CVA. However, over the years we occasionally have heard of people wanting to use CVA's experimental design program for applications that we had not envisioned. To give users this flexibility, and because modern PCs offer processing speeds and memory not available before, over the last few years we have decided to allow CVA to support up to 30 attributes, each with up to 15 levels. Despite this capacity, we recommend that the number of attributes and levels for studies in practice be much smaller than these limits. With more than about six attributes, the questions may become too complex for most respondents to handle, and the number of questions that must be asked of each respondent may become excessive. If more attributes and levels are necessary, it can be advantageous to show respondents pairs of concepts described on subsets of the attributes (partial profile), as in our ACA System for Adaptive Conjoint Analysis.

About CVA
CVA features a variety of options and functionalities. Not all of these tools need to be used in a particular project. The authoring and design tools can be used to produce computer-administered or paper-and-pencil questionnaires. We do not recommend the questionnaire design tool be used for studies with many attributes. With too many attributes, the questions are likely to provide too much information to the respondent, who may become overloaded and confused. For studies with more than about six to eight attributes, we often recommend using the Adaptive CBC or the ACA System for Adaptive Conjoint Analysis. The calculation routines can be used to analyze questionnaire data and calculate respondent utilities. The two standard methods are "Ordinary Least Squares" (OLS) regression and "monotone" (nonmetric) regression. The OLS routine is appropriate for ratings-based data, and the monotone program for rankings-based data. An optional advanced module called CVA/HB is available for hierarchical Bayes (HB) estimation. CVA may also be used to analyze traditional full-profile conjoint data not designed or fielded using Sawtooth Software's tools. You may import designs and respondent data into the CVA system for subsequent analysis using the utility calculation and market simulation tools. Sawtooth Software's market simulator (SMRT platform) is shared across our four conjoint systems.

CVA Help

635

The market simulator lets the researcher specify a market consisting of several products defined in terms of attribute levels, and then estimate the shares of preference that each product would receive. If you have used single-concept presentation (described later) and purchase likelihood ratings, then the market simulator can be used to estimate purchase likelihood for product profiles.

Question Formats
Pairwise Presentation The default questionnaire layout in CVA is Pairwise Comparison. With pairwise presentation, respondents are asked to compare two products, as shown below:

When respondents compare two products side-by-side, it can help them draw finer distinctions between products offering different features. However, if two products are shown per question, respondents must evaluate twice as much information as single concept designs (described below). We suggest the pairwise design for most CVA conjoint projects. Single Concept Presentation Single concept presentation is commonly known as "card-sort." Respondents can be shown one product at a time and asked to rate how likely they would be to purchase each. An example is shown below:

636

SSI Web v7 This method is particularly useful if the "purchase likelihood model" is to be used in the simulator. CVA's ordinary least squares (OLS) routine is appropriate for analyzing single concept ratings. Hierarchical Bayesian (HB) estimation may also be used with single concept ratings. Alternatively, each of a set of product profiles can be printed on individual cards, and respondents can be asked to sort the cards from most to least preferred. In this case, the responses are rank orders of preference, and CVA's monotone regression routine is suggested for developing utilities. Evidence to date suggests that either questionnaire technique (pairwise or single-concept) produces quite similar relative part-worths and importances.

CVA Help

637

6.1.2

CVA Tutorial and Example

Overview
This tutorial leads you step-by-step through a simple CVA (Conjoint Value Analysis) study, from thinking about a marketing problem, planning a questionnaire and creating a new CVA study within SSI Web, to entering a list of the attributes and levels, generating the questionnaire and analyzing the results. CVA is a powerful program, and this tutorial only touches on the basics. You'll become aware of different or more sophisticated approaches by studying other sections within this on-line help and by experimenting with the CVA System. (Note: this example and the data presented are purely fictional. The suggested steps represent one approach to this type of marketing problem, but may not necessarily reflect the single "best" way to conduct such a research study using the CVA system. This tutorial only uses 3 attributes, to keep things simple and to make sure that it can run within student lab systems.)

The Marketing Problem

Over the last five years, the Wendell State University football program has risen to prominence on the national scene. As a graduate of this prestigious university and a rabid football fan, you have reveled in the success. Wendell State's football coach, Biff Brierton, has developed arguably the best defensive line in the nation, three years running. It is nearly impossible any more to buy tickets after the season starts, as the somewhat undersized stadium sells out early. While you are perusing the media guide for this year's team, the phone rings. It is your former marketing professor, Dr. Coldridge. You've kept in contact with her over the last few years as you've built your small strategic consulting practice. It seems Dr. Coldridge also has your favorite subject (Wendell State football) on her mind. The administration at Wendell State has been looking for ways to capitalize on the football team's newfound success. One plan is to add a second tier to the East side of the stadium with additional seating and between 25 to 35 luxury skyboxes. Each skybox would be an enclosed suite, seating from 12 to 24 people, depending on the size. The glass front of each climate-controlled skybox would provide an excellent view of the playing field. Dr. Coldridge explains that an initial study has already been completed that showed a high level of interest among key alumni and corporate customers for the skybox concept at Wendell State stadium. But further questions remain regarding how to design the skyboxes and the price premium to charge for boxes nearer the center of the field. Knowing how often you apply conjoint analysis within your practice, Dr. Coldridge asks if you are interested in assisting her with a follow-up study. You, of course, agree. The following week, Dr. Coldridge visits your office. Always well organized, she has brought a list of objectives for the research: 1) What is the relative demand for 12-seat, 18-seat, and 24-seat skyboxes? (The planners need to know how many to build of each size) 2) How much price premium should be charged for skyboxes near the center of the stadium (on the 40- or 50-yard lines) versus those near the sides (near the 20- to 30-yard lines)? You have a limited sample frame to work with: just 180 names of key supporters who have the economic means to buy skyboxes. Based on your past experience, you think that you may achieve a

638

SSI Web v7 response rate of around 25 to 50%. This would yield between 45 and 90 respondents. Given the relatively small sample size, the limited number of attributes to study, and the objectives of pricing research, you decide to use traditional full-profile conjoint analysis. The sponsors of the study understand that the relatively small sample size will lead to good directional information, but the findings will be accompanied by a relatively large margin of error. Because you are dealing with rating scales, you cannot know the standard errors ahead of time, because you cannot predict the standard deviation of the ratings. This is different from common opinion polls that measure proportions (i.e. the percent projected to vote for candidate A or B) where there are well-known formulas for estimating the standard errors of proportions given different sample sizes.

Defining a List of Attributes and Levels

The next step for you and Dr. Coldridge is to formulate some attributes and levels. You suggest that she ask the planners to specify some preliminary scenarios (an array of competitive skybox configurations/features) to measure. These scenarios in the language of the CVA software will form a base case, or variations from the base case, to be used in the market simulator. The next week, Dr. Coldridge returns with three principal configurations the planners are considering. She stresses that there can be some variations on the three themes (mixing and matching different costs and field locations) but that these specifications definitely cover the full range of options that the planners want to measure.

Satisfied with the progress so far, you both decide to formulate a list of attributes and levels. Based on the information in the CVA documentation, you review the following list of ground rules with Dr. Coldridge. 1. Attributes should be as close to independent of one another as reasonably possible. CVA (like most conjoint techniques) assumes a simple additive model, and violations of simple additivity within CVA are nearly impossible to detect. 2. Because attributes are independent of one another, the levels across all attributes should be able to combine freely. 3. Levels within each attribute must be mutually exclusive. Skyboxes are described using a single level from each attribute. Most conjoint analysis projects involve no more than about five levels per attribute. There is rarely a need to measure more than about five levels of an ordered attribute, you explain, as SMRT's market simulator lets you interpolate between measured levels during analysis. Including too many levels for an attribute significantly lowers the precision of the utility estimates for each level. It also can lead to a problem in conjoint analysis called the "Number of Levels" effect. Attributes defined on relatively more levels tend to get an artificial boost upward in importance.

CVA Help

639

Using the white board in your office, you begin working on a preliminary list. First you write down the three seating capacities. Because each level must communicate fully the meaning of the attribute, you add some additional wording. Seating 12-person seating capacity 18-person seating capacity 24-person seating capacity Next, you turn to the location of the skybox. You note that there are only two variations available in the base case scenario provided by the planners. Because you want to roughly balance the number of levels across attributes, you decide to treat this also as a three-level attribute. Location On about the 20-yard line On about the 30-yard line Between the 40- and 50-yard lines Finally, the issue of pricing: You don't like the fact that the prices are described in terms of ranges (e.g. $15,000 to $20,000 for the least expensive options). This leaves too much interpretation up to the respondent. Do we mean $15,000 or do we mean $20,000? You suggest that each level be defined at a specific price, but that you cover the full range of possible prices to be studied: Price $15,000 per year $22,500 per year $32,500 per year You next figure how many conjoint questions you would need to ask to achieve reasonably stable estimates for each individual. The total number of parameters to estimate is equal to: Total Number of Levels - Total Number of Attributes + 1 For this list (9 total levels and 3 attributes), the number of parameters to estimate is 9-3+1=7. CVA software suggests asking three times as many questions as parameters to estimate. This would imply 21 questions. You both decide this is quite a reasonable task. But then Dr. Coldridge gets a concerned look on her face. She suggests that some of the combinations don't seem to make much sense. It seems hard to believe that a skybox on the 50-yard line would be available for just $15,000 per year, or that skyboxes on the 20 yard line would go for $32,500. You both agree that ideally you'd like to avoid all prohibitions in this design. Given the small sample size, you need to have as efficient plan as possible for estimating stable part-worths. You decide to investigate the damage these prohibitions might do to the efficiency of the design using the CVA experimental design program. If the reduction in design efficiency is only slight, then a modest prohibition or two may be well worth achieving more realistic product profiles and some added utility balance in the design (by avoiding dominated alternatives).

A Sample Scripted Questionnaire

A few days later, you receive an E-mail from Dr. Coldridge She has scripted some expanded attribute definitions and a few additional questions to ask study participants. You put that together with some text you usually use for CVA studies, creating a unified document:

640

SSI Web v7 Start: Thank you for agreeing to participate in this study! We are asking prominent supporters of the Wendell State football program to give us their opinions regarding a new expansion project to the existing football stadium. The expansion will involve additional seating, including about 25 to 30 luxury skybox suites. These luxury suites offer premier views of the playing field, catering, and air-conditioned comfort. Skyboxes range in size from 12-seat to 24-seat suites and cost from $15,000 to $32,500 per year to lease (usually 6 or 7 home football games). You should use your mouse to answer all questions. Click the Next button below to continue...

Attendance: Thinking about last year's season, how often did you attend home football games at Wendell State stadium? m m m m Every home game Most every home game Some home games No home games

IntroCVA1: During this survey, we're going to ask you to consider features of luxury skyboxes. These luxury boxes offer plenty of space for you and your guests to relax and enjoy the game. These are fully enclosed suites, with floor-to-ceiling glass fronts for complete visibility of the playing field. Features include: Air conditioning/heating Fully carpeted, padded swivel chairs Video monitors Buffet table in the back Valet parking and catering are also available. IntroCVA2: The skyboxes might differ in terms of: Seating Capacity: 12, 18, or 24-person seating capacity Location: On about the 20-yard line, on about the 30-yard line, or between the 40- and 50-yard lines Cost per Year: between $15,000 to $32,500 per year. Minimum lease 2 years.

IntroCVA3: Based on these features, we'd like you to compare different options for skyboxes. In each comparison, one skybox will be on the left, and another on the right. Each skybox option will have slightly different features. In each case, we'd like to know which one you prefer. We'll ask you a number of these tradeoff questions so that we can learn what features of skyboxes affect your opinions and preferences.

CVA Help CVA Conjoint Pairs Task (21 of these questions) Which of these two luxury skyboxes would you prefer? <Show skybox on left> or <Show skybox on right>

641

Respondents are asked to choose between different skyboxes, using a 9-point sliding scale. By varying the skybox profiles and observing the respondent's preferences, we are able to determine the preferences respondents have for the different features of the skybox. Respondents will answer 21 of these questions. Interest: During this survey, you have had a chance to trade off different options for luxury skyboxes. We now would like you to consider just this one luxury skybox: 18-person seating capacity On about the 30-yard line $22,500 per year How likely would you actually be to purchase this luxury skybox (2-year lease) if it was available for the next season? m m m m m Holdout: If these were the only three skybox alternatives offered, which would you prefer? Option A: 12-person seating capacity, on about the 20-yard line, $17,500 per year Option B: 18-person seating capacity, on about the 30-yard line, $22,500 per year Option C: 24-person seating capacity, between the 40- and 50yard lines, $30,000 per year Extremely likely Very likely Somewhat likely Not very likely Not at all likely

For the sake of brevity, we have only included one holdout choice task. In an actual CVA study, you would probably want to include at least three or four different holdout choice tasks. Finish: That completes our survey. Thank you very much for your input.

Using SSI Web to Write the Questionnaire

Start by opening the Sawtooth Software SSI Web program. If you have a standard installation, you start it by clicking Start | Programs | Sawtooth Software | Sawtooth Software SSI Web. "SSI" stands for "Sawtooth Software Inc." and "Web" conveys the idea that this software is for developing Web-based surveys (although SSI Web can also be used for standalone computer interviewing with its CAPI interviewing module.) CVA is one component within that software program. The next step is to open a new study, and define a study name. After starting SSI Web, choose File | New Study. The New SSI Web Study dialog appears, with

642

SSI Web v7 your cursor active in the Name field.

The Location field lists the folder (directory) that SSI Web currently is using to store studies. You can use any folder you like for this tutorial project. You can browse to and create new folders by clicking the Browse... button to the right of the Location field, and then (after browsing to the folder in which you want to create a new sub-folder to contain this study) by clicking the "new folder" icon. Each studyname in SSI Web has a maximum number of four characters (either letters or numbers), and SSI Web requires that the study be located within its own folder. The folder that SSI Web automatically creates for your study also carries the same four-character name as the study. For this tutorial, you might create a name such as cvademo1. From the New SSI Web Study dialog, specify cvademo1 as the studyname. Click the OK button. You are returned to the main menu, and a new "Study Navigator" window is displayed along the left-hand side of the screen. As you work with this study, items appear in the Study Navigator window, listing the functional areas you have been using or that are now available to you. This list of items provides a Shortcut link to access those parts of SSI Web. Alternatively, you can access those same areas by clicking icon buttons on the toolbar or by using the pull-down menus. Now that we have created an SSI Web study called cvademo1, we can add our CVA exercise to that study. Click Compose | Write Questionnaire... to access the Write Questionnaire dialog. Place your cursor at the place in the questionnaire when you'd like to add the CVA exercise. Right now, there are no questions in your survey, so you can just click the last page break in the questionnaire list, and click the Add... button. Specify to add a new CVA exercise named CVAskybox to the study (you can have multiple CVA exercises within the same study).

CVA Help

643

Click OK, and the CVA Exercise Settings dialog is shown.

Entering the List of Attributes and Levels

When composing conjoint analysis studies, one typically begins by specifying the list of attributes and levels in the software. To enter the list of attributes and levels you developed, go to the Attributes tab on the CVA Exercise Settings dialog.

644

SSI Web v7

To add the first attribute, click the Add button (at the bottom left of the Attributes panel).

Type in the first attribute name: Seating. The attribute name is a label that the respondent may see in the interview (if you select the option to show the attribute labels at the left of each choice task). If you want to use a shorter label to display to you as the questionnaire author for program settings and data export, specify that label in the Internal Label field. If you do not specify an internal label, the label in the Attribute Display Text is used. Click OK to accept this information and close the dialog. Now that

CVA Help

645

at least one attribute name is in the list, the Add button under the Levels panel becomes active. Also note that the Seating attribute is highlighted in the Attributes panel. With the Seating attribute highlighted in the Attributes panel, click the Add button under the Levels panel to add levels within the Seating attribute. The Level Text dialog is displayed. Type 12-person seating capacity in the Level Display Text field. To add the next level of Seating, press the ENTER key twice. Type in the next level: 18-person seating capacity. Repeat the process for the last level of seating (24-person seating capacity). When you are ready to add the next attribute (Location), click the Add button under the left Attributes panel, type the attribute label, and click OK to place that new attribute on the attribute list. With that new attribute highlighted on the attributes list, click the Add button under the Levels panel to add the three levels of that attribute. Follow the same pattern for the last attribute, Price. For your convenience, we repeat the full list of attributes below. Note that you can copy-and-paste attribute level text from this document (and other text documents) into the text fields within SSI Web. After highlighting the words to copy with your mouse, use the shortcuts Ctrl-C to copy, and Ctrl-V to paste into the desired field. (Hint: you can select a list of attributes or levels from Word and paste into SSI Web using the processor or spreadsheet" icon. This can save a great deal of time.) Seating 12-person seating capacity 18-person seating capacity 24-person seating capacity Location On about the 20-yard line On about the 30-yard line Between the 40- and 50-yard lines Price $15,000 per year $22,500 per year $32,500 per year Make sure to specify a Worst to Best ranking for Location, and a Best to Worst ranking for Price. On the Attribute Specification dialog, there is a drop-down box at the bottom of the dialog for specifying the Level Ranking. You should pre-specify the ranking for any attributes with rational order of preference. "paste from word

Generating the CVA Questions

After you have specified the list of attributes and levels, you are ready to generate the CVA questions. You use CVA's designer to create the combinations of products to show in the questionnaire. The combination of features that make up the conjoint questions across all conjoint questions in your study is collectively called the experimental design. To generate the CVA questions, go to the Design tab of the CVA Exercise Settings dialog:

646

SSI Web v7

If you are uncertain regarding the different settings available and decisions to make about your CVA study, if in doubt, the default settings are ones that seem to work well in practice. We recommend you press F1 (help) if you have questions about any of the controls on any of these tabs. Context-sensitive help will appear when you press F1 to help you understand what each setting does. In many cases, advice regarding different settings is given. Let's first turn to the Design tab. For the CVAskybox study, we have decided to use the default number of suggested questions (21) and Pairwise Comparison Tasks. We'll use the default settings to generate an experimental design. We'll assume that we will be interviewing online so that the software can easily manage multiple versions of the questionnaire. By default, CVA will create 10 different versions of the questionnaire (if you were interviewing via paper-and-pencil, you probably wouldn't want to manage so many different versions of the questionnaire). Each questionnaire version will have strong enough characteristics to stand on its own to provide enough information to lead to precise utility estimates for each respondent. But, having more than one version of the questionnaire will additionally control psychological order and context effects. The process of creating the conjoint questions for each version of the questionnaire is as follows: 1. CVA generates a candidate pool of questions much larger than the number of questions you intend to use. 2. CVA searches within the candidate set for the most nearly optimal subset to use (according to the number of questions you've requested) to estimate part-worths with a high degree of precision. The search uses the criterion of D-optimality. 3. The search process is repeated multiple times, and the best result retained. 4. Steps 1 through 3 are repeated for each additional questionnaire version. By default, CVA uses a random seed to begin the search process. By default, we will use a "1" as the Random seed (you can review this by clicking the Show Advanced Settings button to view the advanced settings). We are now ready to generate the conjoint questions for each of 10 versions of the questionnaire (the experimental design). From the CVA Design tab, click Generate Design. After a short period of computation, a report is shown. Your cursor is automatically positioned at the top of the report, which displays the summary of the results for the different questionnaire versions: The top 10 version D-Efficiencies:

CVA Help Pass# D-Efficiency --------------------40 0.99740 98 0.99740 67 0.99740 74 0.99740 9 0.99740 29 0.99739 30 0.99707 68 0.99674 7 0.99674 88 0.99674

647

To generate ten questionnaire versions, CVA repeated the search process 100 separate times (passes). The D-Efficiencies for the best 10 passes are displayed (and these best 10 passes are used as the 10 versions of the questionnaire). The 40th, 98th, 67th, 74th and 9th passes all produced DEfficiencies of 0.99740. At first glance, it might appear to the casual observer that these five questionnaires are identical. However, CVA's design algorithm ensures that each of these questionnaires, despite identical D-Efficiencies, involves a unique series of questions. (To illustrate how this can occur, consider replicating a conjoint design to create a second version, and for that second version interchanging the labels for levels 1 and 2 of the first attribute. The design efficiency is unaffected by interchanging the labels (it doesn't change the coding of the X matrix), but the questions are now different across the two versions.) D-efficiency ranges from a low of 0 to a high of 1.0. Design efficiencies near 1.0 suggest that this design is about as good as can be achieved to estimate main effects with a high degree of precision, given our particular settings and the number of questions we requested (21). With most design specifications, it is impossible to achieve D-efficiency of exactly 100%. Should we be concerned that designs 6 through 10 have slightly lower D-Efficiency than the best 5 designs? For all practical purposes, the difference in a D-Efficiency of 0.99740 versus 0.99674 is negligible. However, you can always repeat the design procedure and ask CVA to work harder (by altering the Advanced Settings) so that the top 10 designs all have a better chance of being the optimally efficient design. Close the CVA Design Report window by clicking Close.

Investigating the Effect of Prohibitions

(Note: the example below shows how to use prohibitions to achieve more realistic pricing for different skyboxes. Please note that one could use the advanced "conditional pricing" approach as described elsewhere in this documentation. To keep things simpler, we demonstrate the "prohibitions" approach within this introductory tutorial.) You may recall that Dr. Coldridge had suggested that some combinations of Location and Price might be prohibited. A 50-yard line skybox at the lowest price may be entirely unreasonable. Prohibitions will always lower the D-efficiency of the design as reported by CVA. However, there are other considerations when designing good conjoint studies. Conjoint tradeoffs that result in an obvious answer (e.g. an alternative on the right clearly superior on all aspects over the one on the left) are less efficient than those questions implying more difficult tradeoffs. However, too much balance in terms of expected utility among competing alternatives can make for very challenging questionnaires and increased response error. The researcher should strive for a good balance between efficiency and utility balance. Sacrificing a small degree of D-efficiency for the sake of better utility balance and avoiding nonsensical combinations is probably a wise compromise.

648

SSI Web v7 The two prohibitions suggested by Dr. Coldridge might be represented using the following grid of joint occurrences:

The "Xs" represent joint occurrences of Location and Price that should be prohibited. We refer to these as extreme "corner" prohibitions, meaning that the expected utility of these cells is expected to be both highest and lowest in the table. Prohibiting these corner conditions will lead to pairwise conjoint questions that imply more difficult tradeoffs, and the unlikely combinations that were concerning Dr. Coldridge will be eliminated. Extreme corner prohibitions are less damaging to design efficiency than prohibitions on the same row or column of the joint frequency table. Let's specify the prohibitions represented in the grid above. To specify prohibitions, go to the Attributes tab of the CVA Exercise Settings dialog. Click the Prohibitions... button. The Prohibitions dialog is displayed. Click Location in the upper panel, and then click Price in the second lower panel that appears. Then, check the two cells in the grid representing the prohibitions between Attribute 2 Level 3 and Attribute 3 Level 1; and Attribute 2 Level 1 and Attribute 3 Level 3. As you hover your mouse over each cell, tool tips appear to remind you what level text is associated with each prohibition.

CVA Help

649

Click OK, and a message appears warning you that the design you previously created is now invalid. Click Yes to delete the previous design file. Return to the CVA Exercise Settings on the Design tab, and click Generate Design. When you recompute the design, you will discover that the design efficiencies for the top 10 versions of the questionnaire range from 0.95451 to 0.95630. This represents about a 4% decline in efficiency relative to the design with no prohibitions (recall that the D-efficiencies for the top 10 versions previous ranged from 0.99674 to 0.99740). Together with Dr. Coldridge, you decide that the small loss in overall D-Efficiency is worth the added realism that the prohibitions bring to the conjoint tradeoffs. To see the effect of the prohibitions on each attribute in the design, you should run a more in-depth diagnosis of the design as provided by CVA's Test Design facility. Close the CVA Design Report dialog by clicking Close. Then, click the Test Design button from the Design tab.
CVA Design Efficiency Test Copyright Sawtooth Software Tasks are 'Pairwise' using a seed of 1. Based on 10 version(s). Includes 210 total tasks (21 per version). Each task includes 3 attributes. Design Version Efficiencies ------------------------------------------------------------Version Efficiency 1 0.95630 2 0.95601 3 0.95546 4 0.95516 5 0.95516 6 0.95511 7 0.95508 8 0.95477 9 0.95477 10 0.95451 Attribute Efficiency Coefficients (including all versions) ------------------------------------------------------------Attribute Efficiency Seating 0.996 Location 0.839 Price 0.849

The D-Efficiencies for the top 10 versions are repeated. Then, efficiencies for each attribute are reported (considering all 10 versions have been appended). The prohibitions we specified were between attributes 2 and 3. We see evidence in a loss in efficiency particularly concentrated on these two attributes (relative efficiencies of 0.839 and 0.849). Note that the efficiency estimation for each attribute is based on a different method than D-Efficiency, the details of which are provided in the CVA Technical Paper. We suggest you pay most attention to D-Efficiency, as it is a stronger measure of design efficiency and is more accepted in the industry. The relative efficiency coefficients by attribute are useful for pointing out problem areas in a design. The next section of the Test Design report shows the one-way frequency of levels across all versions of the questionnaire (the Total column), and for each of the 10 questionnaire versions:
One-Way Frequencies (by version) ------------------------------------------------------------Level Total Ver 1 Ver 2 Ver 3 Ver 4 Ver 5 Ver 6 Ver 7 Ver 8 Ver 9 Ver 10 12-person seating ca 139 14 14 14 14 14 14 14 13 14 14 18-person seating ca 141 14 14 14 14 14 14 14 15 14 14 24-person seating ca 140 14 14 14 14 14 14 14 14 14 14 On about the 20-yard On about the 30-yard Between the 40- and 122 176 122 12 18 12 12 17 13 12 17 13 12 18 12 12 18 12 13 17 12 12 18 12 12 18 12 12 18 12 13 17 12

650

SSI Web v7
$15,000 per year $22,500 per year $32,500 per year 124 173 123 12 18 12 12 17 13 12 17 13 13 17 12 13 17 12 13 17 12 13 17 12 12 18 12 12 17 13 12 18 12

Note that the ten versions of the design are nearly perfectly balanced with respect to the first attribute (only version 8 shows a slight imbalance). This could be remedied by repeating the design procedure and using the Advanced Settings to ask CVA to work longer on the problem. Attributes 2 and 3 are not nearly as balanced, due to the prohibitions we imposed. Finally, the two-way frequencies for the aggregate design (across all ten versions) are displayed:
Two-Way Frequencies (all versions) ------------------------------------------------------------Att/Lev 1/1 1/2 1/3 2/1 2/2 2/3 1/1 139 0 0 42 58 39 1/2 0 141 0 40 60 41 1/3 0 0 140 40 58 42 2/1 42 40 40 122 0 0 2/2 58 60 58 0 176 0 2/3 39 41 42 0 0 122 3/1 43 41 40 61 63 0 3/2 56 60 57 61 52 60 3/3 40 40 43 0 61 62

3/1 43 41 40 61 63 0 124 0 0

3/2 56 60 57 61 52 60 0 173 0

3/3 40 40 43 0 61 62 0 0 123

Note that Attribute 2 level 1 was prohibited from ever showing with Attribute 3 Level 3; and Attribute 2 Level 3 was prohibited from being shown with Attribute 3 Level 1. We haven't taken the space to show the two-way frequency tables for each version of the questionnaire, but they are also available for your reference. Click Close to exit the CVA Test Design Report, then click OK to exit the CVA Exercise Settings dialog.

Questionnaire Layout
Let's now examine the questionnaire we've built to this point. Click Compose | Write Questionnaire or click the "pencil" icon on the toolbar, and the following dialog is displayed:

CVA Help

651

You'll note that SSI Web has added 21 conjoint tasks (CVAskybox_1 through CVAskybox_21) to the survey. These questions are based on your settings and the default template within SSI Web. We can edit any CVA task to preview how SSI Web displays the conjoint questions to respondents. Highlight the CVAskybox_1 question on the list, and click Edit... (or, you can double-click the CVAskybox_1 question to edit it).

652

SSI Web v7

The CVA question type has main four areas where you control supporting text: Header 1, Header 2, Or Text, and the Footer. To see what your question looks like (using the default layout options), click Preview.

Note that by default, we use "alternating colors" for the background color of the product concept areas. The colors alternate between left and right in subsequent questions to give respondents a visual clue that the aspects of the questions (the product characteristics) are changing. The colors that are currently being used are those supplied automatically by the Style that has been selected by the survey. Later, we'll show you how to select a different style.

CVA Help

653

The questionnaire we developed with Dr. Coldridge included the following text in the header: "Which of these two luxury skyboxes would you prefer?". We need to change the default text provided by CVA to include the text we intend to use. Go ahead and close the browser window and return to edit the CVA question. Note that the default text to change also includes some "commented-out" text intended for use with the single-concept at a time format. (Note: the <!-- --> tag in HTML tells the browser to ignore any text found within the tag.) Change the Header 1 text to say: Which of these two luxury skyboxes would you prefer? Click Preview, and the following is displayed:

You only need to edit one of the CVA questions to create the formatting as you desire. The changes you make in one CVA question apply to all CVA questions in your survey.

Passwords and the Start Screen

Placing a survey on the web makes it convenient for a geographically dispersed population to take surveys. However, the danger is that the survey may become available to people that have not been invited to take the survey. Also, some respondents might try to take the survey multiple times. Assigning respondent passwords is a way to deal with both of these issues. Password assignment is beyond the scope of this tutorial, so to make things simple we'll assume that no passwords are to be used. The Start screen is placed as the first page in your questionnaire, whether you are using passwords or not. If you are using passwords, it is the page in which respondents type passwords to access the survey. If not using passwords, you'll use this screen to specify any introductory/welcome text. Let's enter the introductory text into the Start screen using the Write Questionnaire dialog. You can access the Write Questionnaire dialog by selecting Compose | Write Questionnaire or by clicking the "pencil" icon on the toolbar.

The introductory text and other survey questions would most likely be initially developed within a word processing document. Assuming you really had such a document, you might use the Ctrl-C to copy, and the Ctrl-V shortcuts to paste the information into SSI Web. We suggest you simply copy-andpaste the text within this document into your SSI Web questions rather than re-type the text (if viewing

654

SSI Web v7 this document with Acrobat Reader, you can use the "text select" icon toolbar.) The introductory screen is as follows: Start: Thank you for agreeing to participate in this study! We are asking prominent supporters of the Wendell State football program to give us their opinions regarding a new expansion project to the existing football stadium. The expansion will involve additional seating, including about 25 to 30 luxury skybox suites. These luxury suites offer premier views of the playing field, catering, and air-conditioned comfort. Skyboxes range in size from 12-seat to 24-seat suites and cost from $15,000 to $32,500 per year to lease (usually 6 or 7 home football games). You should use your mouse to answer all questions. Click the Next button below to continue... from the Acrobat Reader

We'll insert this introductory screen in the Start question that is automatically the first question of any SSI Web survey. From the Write Questionnaire dialog open the Start question by highlighting it in the list of questions and clicking Edit.... Alternatively, you can double-click the Start question, and it automatically opens. The Questionnaire Access and Passwords dialog is displayed::

Most question types in SSI Web have "Header 1", "Header 2," "Body" and "Footer" sections. (The

CVA Help

655

Start question has all but the "Body" section.) These are "text" areas in which you can insert any text (including HTML). When the question is viewed with a web browser, the sections are organized roughly as follows:

For this introductory screen (that includes three paragraphs of information), it seems to make sense to place the first paragraph in the "Header 1" area, the second paragraph in the "Header 2" area, and the third paragraph in the "Footer" area. Type (or cut and paste) the following text for the Start question into the text areas in the Start question. Put the first paragraph in Header 1, the second paragraph in Header 2, and the third paragraph in Footer. Thank you for agreeing to participate in this study! We are asking prominent supporters of the Wendell State football program to give us their opinions regarding a new expansion project to the existing football stadium. The expansion will involve additional seating, including about 25 to 30 luxury skybox suites. These luxury suites offer premier views of the playing field, catering, and airconditioned comfort. Skyboxes range in size from 12-seat to 24-seat suites and cost from $15,000 to $32,500 per year to lease (usually 6 or 7 home football games). You should use your mouse to answer all questions. Click the Next button below to continue...

The main dialog only shows a portion of the text, but you can click the "pencil" icon each text area to expand the text editing window.

adjacent to

To preview how the question will look when it runs on the respondent's computer using Windows Explorer, click the Preview button. The question is displayed in "Preview" mode (no data are saved, and no special "error checking" or "validation" is performed for the question).

656

SSI Web v7

SSI Web automatically places paragraph breaks (extra blank lines) between the Header 1, Header 2, and Footer sections. If you put all the text in a single section, you may see that when the web browser interprets the text, it all runs together without any blank lines between the paragraphs (unless you insert some HTML instructions to force blank lines between paragraphs). We'll talk about using HTML within your text to take greater control over the layout, font, and style later in this unit. Click the "X" icon in the upper-right hand portion of the Preview window to close that window and return to the previous SSI Web dialog. After you are happy with the layout of the Start screen, click OK to return to the Write Questionnaire dialog. If you need to move a question once it has been added to the List of Questions, simply highlight the question to be moved and click the or buttons to move the questions within the list (you can also highlight a question and click Ctrl-X or Ctrl-C to cut or copy and Ctrl-V to paste questions within the current questionnaire, or even to another SSI Web study.)

Select-Type Questions
Now you are ready to specify the first Select-Type question: Attendance: Thinking about last year's season, how often did you attend home football games at Wendell State stadium? m m m m Every home game Most every home game Some home games No home games

First, make sure that you highlight the page break directly beneath the Start question on the List of Questions (when you add a new question, it is placed directly below the highlighted question/page break on the list). At the Write Questionnaire dialog, click Add..., choose Select as the question type, and type Attendance for the question name. Click OK and the Select Question dialog is shown. Place the heading text for the question in the Header 1 field. To specify the response options, click the Response Options tab. From the Response Options tab, use the Add button to add the four response options for this question. When you are finished, the question should look something like:

CVA Help

657

Now that you have seen how to specify Single Select question types in SSI Web, you have the tools you need to specify the other two select-type questions for this questionnaire (Interest and Holdout-Please refer to the questionnaire text for these questions as presented near the front of this unit). The Holdout question layout poses some interesting challenges. The simplest approach is to format it as a Select-type question with three columns (specify #columns on the Settings tab of the Select Question dialog). The Holdout question formatted as a Select-type question looks like:

To do this, we formatted each option as a separate Response Option, with additional HTML text to force hard returns and a blank line. For example, the first response option is formatted as:

658

SSI Web v7

However, you can make this holdout choice task look even nicer if you format it as a Grid-type question (one row, three columns, radio buttons). On the Rows tab, make sure to un-check the Show Row Labels box. From the Columns tab, add three Predefined List Members (List Member Text just as shown directly above) to show each of the three product concepts. When you click Preview, the grid question should look something like:

Formatting Text in SSI Web Questions

In addition to the Select questions we added to the questionnaire previously, there are four text-only instructional screens to add. Referring to the questionnaire we outlined earlier, these "questions" are named IntroCVA1, IntroCVA2, IntroCVA3, and Finish. Even though the respondents aren't asked to provide specific inputs to these, we refer to these as "questions" in the sense that we add them to the questionnaire as if they were standard questions, and they are listed separately by their "question" name in the List of Questions. After the Attendance question follows the text-only instructions:

CVA Help IntroCVA1: During this survey, we're going to ask you to consider features of luxury skyboxes.

659

These luxury boxes offer plenty of space for you and your guests to relax and enjoy the game. These are fully enclosed suites, with floor-to-ceiling glass fronts for complete visibility of the playing field. Features include: Air conditioning/heating Fully carpeted, padded swivel chairs Video monitors Buffet table in the back Valet parking and catering are also available.

This text layout is a bit more challenging than we dealt with before (when we put three paragraphs of introductory text into the Start question). There are more than three separate paragraphs here, there is a bullet point list, and we'll need to deal with the additional element of bolding selected text. Add a Text/HTML Filler question directly following the Attendance question in the questionnaire. To do so, highlight the Attendance question, click Add, and specify the Question Name as IntroCVA1 and the question type as Text/HTML Filler. The Text/HTML Filler dialog is displayed. Earlier in this tutorial, when we used the Start question to format the opening page of the survey, we placed each paragraph of text in a separate Header 1, Header 2, or Footer section. We saw that SSI Web automatically places blank lines between text in these sections. However, with the text in the IntroCVA1 question, there are more paragraph breaks to insert. We'll take the opportunity here to introduce the concept of using a few simple HTML instructions within our survey text. HTML stands for "HyperText Markup Language" and provides simple ways for you to enhance the look of your surveys, such as by bolding or underlining words, or making paragraph breaks within text. Browsers know how to interpret HTML instructions when displaying the page on the screen. If you know HTML, you can use it whenever you wish within SSI Web surveys to accomplish your aims. If you don't know HTML, it isn't very difficult to learn a few HTML tricks, or you can use the toolbar available within the editor that appears when you click "Pencil" icons in SSI Web:

Either type or cut-and-paste the text for the IntroCVA1 question into the Header 1 text field (Any of the other three fields could be used also, though the Body field is indented). Click the "Pencil" icon to display the larger text editing window. Initially, the text appears something like this:

660

SSI Web v7

Even though it appears that there are extra blank lines between the paragraphs, web browsers ignore these hard returns (as well as more than one consecutive space characters), so you'll need to provide HTML instructions to insert these paragraph breaks (hard return plus blank line). HTML instructions are placed within <> brackets, called "tags." For example, the HTML instruction to create a paragraph break begins with an "open" paragraph tag written as <p> and optionally ends with a "close" paragraph tag written as </p>. The text to be formatted as a separate paragraph is enclosed within these tags. You can either directly type HTML within your document, or you can highlight text to be modified and click the icons on the HTML toolbar in the editor. If you highlight the first paragraph with your mouse: During this survey, we're going to ask you to consider features of luxury skyboxes. and then (with the text in the first paragraph highlighted) click the "Paragraph" icon this inserts a <p> prior to the text and a </p> after the text: on the toolbar,

<p> During this survey, we're going to ask you to consider features of luxury skyboxes. </p> When the browser interprets this text, it doesn't display the "tags" but instead separates the text enclosed within the tags as a separate paragraph. Repeat the same for each of the paragraphs in the IntroCVA1 question. Next, we need to add a bullet point list. The following HTML tags are used to create bullet point lists: <ul> </ul> <li> </li> (open and close tags for an unordered bullet list) (open and close tags for list elements within an unordered bullet list)

When you apply these HTML elements within the text for this question, you'll specify: <p>Features include: <ul>

CVA Help <li>Air conditioning/heating</li> <li>Fully carpeted, padded swivel chairs</li> <li>Video monitors</li> <li>Buffet table in the back</li> </ul> </p>

661

Next, we need to bold certain words in the text. The HTML tags for bolding text are <b></b> (with the text to be bolded placed between the open and close bold tags). You can either directly type these tags within the document, or highlight the text to be bolded with the mouse and click the "Bold" icon on the toolbar. After you finish separating the paragraphs with <p></p> tags, applying the bullet list instructions, and bolding the appropriate text with <b></b> tags, it should look something like:

Click OK to return to the Text/HTML Filler dialog and then Preview to see how the web browser displays this question. It should look like:

662

SSI Web v7

When you add the Finish question at the end of the survey, make sure to add it at the end of the survey as a Terminate/Link question type. On the Settings tab of the Terminate/Link question, click Terminate Respondent to indicate that respondents that reach this question are finished. SSI Web may warn you at this point that a terminating question must stand alone on its own "page" in the survey. This leads us into our next discussion on page layout.

Page Layout and Study Settings

It may be helpful at this point to review how SSI Web breaks the survey into separate pages, and some basic global settings that affect the look and functionality of your SSI Web questionnaire. The Write Questionnaire dialog also displays the page breaks that may be currently set for your questionnaire.

CVA Help

663

This dialog shows how the various questions we've specified (or that SSI Web has automatically added to the list) are arranged across different pages. Page breaks are indicated by "---------------<Page>". Notice that the preliminary questions (Attendance, IntroCVA1, IntroCVA2, and IntroCVA3) we've added to the questionnaire are all currently arranged on the same page. Let's assume we wanted to break these up, one question per page. You add a page break after the Attendance question by highlighting Attendance on the list and clicking the Add Page Break button. Attendance is now placed on its own page. Repeat this operation to place IntroCVA1, IntroCVA2, and IntroCVA3 on separate pages. You may also choose to place Interest and Holdout questions on their own pages. Make sure to place the final Terminate/Link screen Finish on its own (last) page.

664

SSI Web v7

Adding Polish and Style

You have probably noted that the survey we've created is pretty bland looking. We can add some polish and style by clicking the Survey Settings icon from the Write Questionnaire dialog (or by clicking Compose | Survey Settings from the main menu). First, let's select a style from the Styles tab. Select a style you wish to use, and click Use Selected at the bottom of the dialog to implement the style. We'll select Panama as the style, but you might want to experiment with other styles. On the General Settings tab, you can select to use graphical Next, Previous, and Select/Checkbox buttons. A library of buttons is provided, available in the C:\Program Files\Sawtooth Software\SSI Web\graphics folder. On the Headers and Footers tab, you can specify a Header and a Footer. We'll specify Skybox Questionnaire as the header text. The Progress Bar tab lets you add a progress bar to your survey. We'll click the check box to add a progress bar to the page footer. When you go back and test run the survey, your survey should have a bit more polish. Under Panama style (and by selecting graphical Previous and Next buttons), our survey now looks like:

Test Running Your Survey Using "Local Test Server"

Although the Preview Survey function is nice, you generally will want to test your survey in runtime mode, with any skip patterns, response verification, and randomizations in place. You could upload your files and the Perl scripts to the Web Server, but SSI Web provides a more convenient way to test

CVA Help run your survey locally on your PC or laptop.

665

When SSI Web was installed on your computer, web server software called "Apache" was also installed. Apache lets your computer run the questionnaire just like a remote Web Server would.

Local Test Server

From the Write Questionnaire dialog, Click the Test Survey button Locally.

and select to Test Survey

Your browser opens up the first survey page. This is your survey, as it would appear to a respondent over the Internet (there are slight visual differences from browser to browser). To close the survey, simply close the browser window (using the X in the upper-right hand corner). To run the survey again, click the Test Survey button .

After you have edited the survey to the point that you are pleased with the look, content, and functionality of your survey, we suggest you take the survey yourself under Local Test Server. You should answer each question legitimately and carefully. After you have taken the questionnaire carefully as suggested above, you should download the data (downloading data is described in a separate tutorial called " Getting Started with SSI Web: A 45minute Hands-On Tour"), then compute and examine your part-worth utilities. The part-worths should very closely reflect your preferences. If they don't, this suggests something may be wrong with your study setup. After you are convinced that the survey is functioning properly, you should also pretest your survey among your colleagues. They can give you feedback regarding the usability of the survey, and you can examine the resulting part-worths to make sure the data at least have face validity.

Pretesting and Fielding

Let's now return to the marketing problem and story we began at the beginning of this document. After you have tested the survey using the Local Test Server, you post the survey to the Web using an account set up for you by an ISP (Internet Service Provider). (Setting up your survey on the web is beyond the scope of this unit, but is described in detail elsewhere. Fielding options also include hosting on your own company's server, or using Sawtooth Software's hosting services). You send an email to Dr. Coldridge, with a link to your survey, and she browses to and takes the survey. After she give you some feedback, you invite six other individuals in administration and planning take the survey. You download the data and compute part-worth utilities. Due to the small sample size, the results are a bit noisy, but the data seem to feel right. Next, you recruit six respondents from the actual sampling frame to come to a central site to take the survey. You watch silently as they take the survey. At the end of each session, you ask each respondent follow-up questions to ensure that there weren't any parts that were difficult to understand or just didn't make sense. You ask them specifically about the conjoint questions, making sure that they understood what to do, and that they didn't feel overwhelmed by the task. After debriefing the test respondents and analyzing their data to ensure that the results look reasonable, you make a few small adjustments and proceed to field.

666

SSI Web v7

Data Management
SSI Web includes an Admin Module so that you can monitor or access your project from any computer connected to the Web. The Admin Module is password protected, and your passwords for access are specified on the Server Settings tab of the Survey Settings dialog. SSI Web generates random administrative access passwords whenever you create a new study, but you can modify them to suit your needs. To download the data from the respondents to the Skybox study, you can browse to the administrative module for your study on the web site (again, we are speaking hypothetically, as for this tutorial study there is no such site set up). Once at the administrative module, you would download your data (by clicking Download Data from the main menu), making sure to save your data (named STUDYNAMEdat.dat, where STUDYNAME is your study name) to the same folder on your hard drive in which you developed the SSI Web project. Alternatively, from the SSI Web interface, you can simply click Field | Download Data... to download your data without logging into the Admin Module. You can also use the Admin Module to tabulate results from your Select-type questions. For example, you might be interested in the responses to the Holdout question. Once within the Admin Module, you can request a tabulation of each question in the survey (the Marginals function). The tabulation for the Holdout question for 50 hypothetical respondent records for this tutorial study is as follows: Holdout Option A: 12-person seating capacity... Option B: 18-person seating capacity... Option C: 24-person seating capacity... Total 9 18.0% 20 40.0% 21 42.0% 50 100.0%

The report shows that 18% of respondents chose Option A, 40% chose Option B, and 42% chose Option C. To download the data from the 50 respondents to the Skybox study, you would browse to the administrative module for your study on the web site (again, we are speaking hypothetically, as for this tutorial study there is no such site set up). Once at the administrative module, you would accumulate and download your data (by clicking Download Data from the main menu), making sure to save your data (named STUDYNAMEdat.dat, where STUDYNAME is your four-character study name) to the same folder on your hard drive in which you developed the SSI Web project.

Computing/Examining Utilities
Once you have downloaded the data, you are ready to estimate part-worths. If you try this with this tutorial study you created, you will receive an error stating that you have no data within your project folder to analyze. However, hypothetical data for this project are available as a sample CVA study in your SSI Web installation. If you wish to follow along with the analysis steps below, open the CVA Tutorial project we have provided by clicking Help | Sample Studies | Sample CVA. To compute utilities, click Analysis | Calculate CVA Utilities Using Ordinary Least Squares.

CVA Help

667

Select Settings and the CVA OLS Settings dialog is displayed. These controls govern the OLS estimation. The range of respondent answers is shown (as specified in your questionnaire). The recode method is the dependent variable transformation. For most applications, we suggest a logit rescaling. It is beyond the scope of this tutorial to discuss dependent variable transformations, but more information on that subject is available in a section entitled: " CVA Ordinary Least Squares Settings". The range of the respondent answers (1 to 9) and the scale direction are indicated. This pairwise questionnaire uses a sliding scale with the lowest number on the left and the highest number on the right, so the correct scale direction to choose is Highest Number on Right. While still at the CVA OLS Settings dialog, click Show Advanced Settings, and then Edit... to edit the utility constraints. Utility constraints impose any rank order of preference assumptions you have made for the levels within attributes. By default, any a priori order settings you used when specifying the attributes are also used as utility constraints. Currently, you have set the Location attribute as Worst to best ranking, meaning the first level ("On about the 20-yard line") is worst and the last level ("Between the 40- and 50-yard lines") is best. The Price attribute is set as Best to worst. These constraints will be enforced during utility estimation. If you wish to modify these selections prior to calculating utilities, you can do so from this screen. There is an additional area for specifying constraints that involves just pairs of attribute levels. If some attributes are partially ordered, you may specify those constraints using the Custom Constraints button. Click the Cancel button to return to the previous dialog, and then click the Cancel button again to return to the main Compute Utilities dialog. Once you are satisfied with your utility estimation settings, click Prepare for CVA OLS Computation. SSI Web first scans your data file and gives a summary report of the conjoint information:
** Data Summary ** Number of respondents included = 50 Total tasks included across all respondents = 1050 The data will be written to the following file: c:\temp\cva2\cva2 OLS.utl

When you click Compute Utilities, SSI Web examines each respondent's data and computes a set of utilities for each level in your study for each respondent. For more information about interpreting utility values in conjoint analysis, please see Appendix B in this documentation, entitled: " Interpreting Conjoint Analysis Data". SSI Web then reports that the utilities have been saved to a file named STUDYNAME OLS.utl, for use in market simulations using the separate SMRT software (Sawtooth Software's Market Simulator). But, the information is also made available to you within a report. A report window is displayed in a spreadsheet format with two sheets (tabs located at the bottom of the display): Summary and Individual Utilities. On the Summary sheet, the average part-worth utilities (scaled according to the normalization procedure called Zero-Centered Diffs) and attribute importances for the sample are displayed, along with standard deviations.

668

SSI Web v7

The Summary Results report also contains diagnostic information about the utility run:
Tasks Included: All Pairwise Recode Method: Logit Additional Utility Constraints: All Attribute Levels: Attribute 2, Attribute 3 Successfully computed utilities for 50 respondents. Average R-squared = 0.78849 Median R-squared = 0.81081 Number of Attribute Attribute Attribute Attribute constraint violations resolved: 2 Level 3 preferred to Level 2: 2 Level 2 preferred to Level 1: 3 Level 1 preferred to Level 2: 3 Level 2 preferred to Level 3: 1 1 1 2 Respondents Respondents Respondents Respondents

You are shown the average and median R-squared (fit) values for the data set. R-squared is bounded by 0 and 1.0, with 1.0 indicating perfect fit (note that the reported R-squared is the fit after imposing any constraints on the data). Below that, SSI Web reports the reversed utility relationships that had to be resolved. Respondents answer with some degree of error, and most data sets in practice will have many reversals at the individual level, especially for attributes with relatively low importance. This tutorial data set was artificially developed and has very few out-of-order relationships. If you discover that a large proportion of respondents in your data sets have certain out-of-order relationships, this suggests one or more of the following explanations: 1. The violated preference relationships are of low importance to respondents (minimal actual difference in "true" part-worths), 2. Your assumptions regarding the utility relationships are faulty (you may need to change your assumptions under the Constraints dialog, accessed by clicking Analysis | Calculate CVA Utilities Using Ordinary Least Squares... | Settings... | Show Advanced Settings >> |

CVA Help

669

Edit...), 3. The respondents answered with a high degree of error, 4. Your design was inefficient or did not include enough questions to stabilize the utility estimates, 5. You have made a data processing error. If more than half of the respondents violate a particular rational preference relationship, it is definite cause for concern. If you open the Individual Utilities sheet (by clicking the Individual Utilities tab near the bottom of the report), you can view the raw individual-level utilities for each respondent:

This report displays similar results as are saved to the STUDYNAME OLS.utl file (which is used for simulations in the SMRT market simulation software). The Respondent number and R-Squared (actually, R-Squared x 1000) are displayed in the first two columns. The part-worth utilities for each level within the study start in column 4. You can save results from this report to an Excel spreadsheet (containing both the Summary and individual Utilities sheets) by clicking the Save As... button.

Moving the Data into SMRT

SMRT stands for "Sawtooth Software Market Research Tools" and contains the market simulation tool used to analyze data resulting from any of Sawtooth Software's conjoint analysis systems. We'll now describe how you move data from your CVA project within the SSI Web Platform into SMRT. When you calculated part-worth utilities using Analysis | Calculate CVA Utilities Using Ordinary Least Squares... this process creates a text-only format file containing case IDs and part-worth estimates called STUDYNAME OLS.utl. To move the CVA part-worth utility data into SMRT for analysis: 1. Start the SMRT software by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 2. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname, making sure to specify that this is a CVA study type). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study.

670

SSI Web v7 3. Import the STUDYNAME OLS.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CVA utilities (selecting CVA (Traditional Conjoint) Utilities (*.utl) as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Click close to return to the main menu. Now that you have imported the utilities, the Market Simulator menu item becomes active. From the main menu, click Analysis | Market Simulator. You will note that there is a list of available Utility Runs (your recently computed run is available), and a list of Simulation Scenarios (currently empty). Click Compute!, and a report of average utilities and importances for your data set is displayed in the report window (these data should match the similar report that was earlier displayed within SSI Web, directly following utility estimation):
Scenario: Utility Run: CVA OLS Run

Average Utility Values Rescaling Method: Zero-Centered Diffs 12-person seating capacity -26.57 18-person seating capacity 4.25 24-person seating capacity 22.32 On about the 20-yard line -52.30 On about the 30-yard line -0.13 Between the 40- and 50-yard lines 52.42 $15,000 per year $22,500 per year $32,500 per year Total Respondents Average Importances Seating Location Price Total Respondents 58.32 9.57 -67.88 50

23.03 34.91 42.07 50

<<The above average utilities are rescaled utilities using the zero-centered "diffs" method. The diffs method rescales utilities so that for each individual the total sum of the utility differences between the worst and best levels of each attribute across attributes is equal to the number of attributes times 100. Note: the attribute utilities are influenced by the number of respondents in the simulation and respondent weighting, but are not affected by the product specifications you enter. After you have seen these once for a particular group of respondents, you may choose to omit them in subsequent simulations by un-checking the Display Utilities box in the Scenario Specification dialog.>> For more about interpreting conjoint utilities, please see Interpreting Conjoint Analysis Data. It is difficult to make firm conclusions from just these fifty respondents, but we might try to form some tentative conclusions: First, looking just at the report of average importances (the second section above), it appears that the least important attribute (among the three attributes and the feature ranges we measured) is whether the skybox has 12-person or 24-person seating capacity. Looking at average utility values, we see that if everything else was held constant (price and location on the field), respondents on average would prefer to have more seating capacity than less in the skybox. Although you don't have much data to work with for segmentation purposes in the Skybox1 study (just

CVA Help

671

50 respondents), you should note that you can run market simulations by "banner" point. Banner points are the segmentation variables that run across the columns of a cross-tabulation report. If you select a Banner variable using the drop-down menu at the upper-left of the Market Simulator dialog, and then click Compute!, you will note that the utility/simulation report is broken out by different groups of respondents, depending on how they answered the Banner variable. Many important effects due to the competitive context, respondent heterogeneity and correlations of preferences among attribute levels can occur within a data set. These may be completely obscured by viewing just average utilities and importances. Also, the concept of utility scores seems difficult to grasp for many managers. For these reasons, running market simulations offers a more powerful and revealing way to look at conjoint data.

Market Simulations
To open the Market Simulator, click Analysis | Market Simulator or double-click Market Simulator from the Study Navigator window. The first time you open the Market Simulator for a study, the list of Simulation Scenarios is empty. A Simulation Scenario is a particular "what-if" scenario, wherein certain products are placed in competition with one another in a simulated marketplace. Prior to fielding your study, Dr. Coldridge had conferred with the administration's planning group regarding potential configurations for skyboxes. The result was three principal alternatives, which were used as a holdout question in your CVA survey:

You may recall that the tabulation of the Holdout question showed that 18% of respondents preferred the Bronze option, 40% preferred the Silver option, and 42% preferred the Gold option. Notice that two of the levels of Price ($17,500 and $30,000) are in between the levels we measured in the study. The market simulator lets us interpolate between levels, so this is not a problem. However, it will be easier to specify these levels if we have assigned "Level Values" to each level. To assign Level Values, from the main menu click Analysis | Market Simulator and then click the Assign Level Values button at the very lower left-side of the dialog. The Assign Level Values dialog is displayed. Using the drop-down box, select Price. Click the first row in the Value column (corresponding to $15,000). Type "15000" Click the next two cells and enter "22500" and "32500." Click OK to return to the Market Simulation menu. Create a new simulation scenario by clicking Add.... When you click the Add... button, the Scenario Specification dialog is displayed. The first step is to type a scenario name into the Name field, in the upper left-hand corner of the dialog. Type Base Case. Next, you'll specify the three products for the first scenario. The area you type the products into looks like the grid within a spreadsheet. The first column is where you type the product label. As you click the various columns associated with the attribute levels, the level codes will appear in the window below to remind you of the codes associated with each attribute's levels.

672

SSI Web v7 The attribute level codes for the three products for the planned skybox configurations were: Base Case:

Specify these three products in the product entry grid. To add new rows to the grid (for additional products) click the Insert Product button. Hint: While working within the product specification grid, you can press the Enter key to advance to the next cell. You can use the arrow keys to move the cursor within the grid. You can also cut-andpaste information within the grid, or from a spreadsheet. After you have specified these three products, you should choose a simulation method. The default method is Randomized First Choice. We generally recommend this method for competitive market scenarios like this one, so we'll use the default. Because we have already examined the utilities and importances, un-check the Display Utilities box. Click OK again to close the Scenario Specification dialog and return to the main Market Simulator dialog. Choose a utility run to use for your simulation. At this point, you should only have one run in the list, so this is your only choice. If there were multiple utility runs (for example, another run from the CVA/HB system), you could select that run by highlighting it in the list of Utility Runs. To simulate shares for a scenario you have specified, place a check mark in the box next to the scenario name in the Simulation Scenarios list, and then click Compute!. SMRT examines each respondent's record and predicts which product(s) each respondent would likely choose based on the product specifications and each respondent's utilities. The following report is displayed (we've inserted commentary within brackets): Scenario: Base Case Utility Run: CVA OLS Run Product Simulation Settings Simulation Mode: Simulation Model: Randomized First Choice Total Sampling Iterations: 200000 Attribute Variability Multiplier: Autocalibrated Value (0.568351) Product Variability Multiplier: 0 Exponent: 1 <<Above are the settings that govern this run. It is not necessary at this point for you to know what these settings mean (iterations, Multipliers, Exponent). These are explained in other areas of this online help and also in the manual.>> Product Specifications Bronze Silver Gold Seating Location 1 1 2 2 3 3 Price 17500 22500 30000

CVA Help

673

<<Above are the product level codes you specified for the three products in this simulation. Below are the simulated shares of preference (choice) for these products, along with their standard errors.>> Product Shares of Preference Bronze Silver Gold Shares 22.41 36.41 41.17 50 Std Err 1.37 1.47 1.48

Total Respondents

Though a total is not shown, the shares of preference sum to 100%. Standard errors are presented next to each product's shares, which are useful for expressing confidence intervals around the share estimates.

Tuning the Market Simulator to Predict Choice

Recall that when respondents were asked to choose among these three alternative skyboxes in the Holdout question within the questionnaire, 18% preferred the Bronze option, 40% preferred the Silver option, and 42% preferred the Gold option. The market simulator's predicted shares as displayed at the end of the previous section show a very similar pattern of preference: the rank order is the same, but the magnitude of the share differences is a bit compressed for the predictions relative to the actual choices. When using a ratings scale with CVA questions (recall we used a 1 to 9 scale) and OLS estimation, there is no guarantee that the resulting utilities will be appropriately scaled so that the market simulations are closely linked to choice probabilities. However, the market simulator provides a way for us to tune the scaling of the shares to better match our target holdout choice probabilities. We tune the scaling of the shares using the "Exponent". Tuning the market simulator to best fit holdouts requires some trial and error. The first step is to compare the predicted results to the actual choice probabilities and determine a measure of fit. Let's again compare our predicted shares of preference with actual choice probabilities as determined through the Holdout question. The table below reflects the predicted shares when using the default settings for Randomized First Choice (Exponent = 1): (Exponent=1) Predicted Shares 22.41 36.41 41.17 Absolute Difference (Predicted vs. Actual) 4.41 3.59 0.83 -----Mean Absolute Error (MAE): 2.94 Actual Shares 18.00 40.00 42.00

Bronze Silver Gold

On average, our predictions are 2.94 share points different from the actual target holdout shares. There are many ways to determine fit; the use of MAE (Mean Absolute Error) is one common measure. Next, let's tune the exponent to achieve a better fit. From the Market Simulator dialog, highlight the Base Case scenario on the list of Simulation Scenarios, and click Edit. Then, click Advanced Settings and change the Exponent setting to 1.5. Choosing a higher exponent will accentuate the differences among the choice probabilities, which will move us in the right direction to achieve closer fit to the holdout probabilities. The new data are: (Exponent=1.5)

674

SSI Web v7 Predicted Shares 18.68 37.32 44.00 Absolute Difference (Predicted vs. Actual) 0.68 2.68 2.00 -----Mean Absolute Error (MAE): 1.79 Actual Shares 18.00 40.00 42.00

Bronze Silver Gold

By adjusting the Exponent from 1.0 to 1.5, we have made the resulting share predictions "steeper" and achieved a better overall fit to the holdouts. Our MAE has decreased from 2.94 to 1.79. By trial and error, you will find that the MAE is minimized when the Exponent is about 1.6 (MAE = 1.71). The data for this run are given below: (Exponent=1.6) Predicted Actual Shares Shares 18.10 18.00 37.43 40.00 44.47 42.00 Absolute Difference (Predicted vs. Actual) 0.10 2.57 2.47 -----Mean Absolute Error (MAE): 1.71

Bronze Silver Gold

For the remainder of this tutorial, set the Exponent to 1.6 so that your results match ours. If you were analyzing this data set in practice, you should use the Exponent of 1.6 for all subsequent market simulations. Note that in practice researchers usually include more than just one holdout choice task. The exponent is adjusted to reduce the MAE across all holdout tasks. One does not choose a separate Exponent to provide best fit to each separate holdout task. The Market Simulator is a powerful tool for testing nearly an unlimited number of possible market scenarios. It is important to remember that the shares of preference resulting from conjoint predictions are not equivalent to actual market shares, and often look quite different. Many other factors in the real world influence market shares and cannot be measured and reflected solely by conjoint data. Conjoint assumes perfect information, equal distribution and availability, and that each respondent is in the market and able to purchase. Conjoint results reflect the potential market acceptance, given proper promotion, distribution and time.

Sensitivity Analysis
Performing sensitivity analysis is one of the most common uses of the market simulator. Sensitivity analysis involves first simulating a base case (as we previously did), representing an array of products in competition with one another, then modifying one of the products and observing the change in share due to the modification. For example, the planning group at Wendell State may be interested to know how raising the price for the most expensive option (Gold) would alter the relative preference among the three proposed alternatives. You may recall from the previous sections in this tutorial that the base case scenario was defined as:

CVA Help

675

The shares of preference for this base case configuration are: Bronze Silver Gold 18.10% 37.43% 44.47%

The question is: how would the share of preference change if the Gold option were increased from $30,000 per year to $32,500 per year? We can create a new simulation scenario to test this possibility. You can save a lot of time in entering this scenario if you retain the settings for the first three products when you create the new scenario. SMRT lets you do this. Click Add..., and from the Add Scenario dialog use the drop-down control to choose Base Case as the starting template for the new scenario. You can name this second simulation scenario Gold to $32,500. The level codes/values for the three products for this simulation are: Gold to $32,500:

After altering Gold's price, click OK to return to the Market Simulator dialog. Before running the new Gold to $32,500 scenario, you may wish to clear the previous results from the report window by clicking Clear and un-checking the box for the Base Case scenario. Check the Gold to $32,500 scenario and click Compute!, and the new shares of preference are displayed. We've summarized the previous shares of preference for the base case (when Gold was at $30,000) and the new shares of preference (when Gold is increased to $32,500) in the table below: Product Shares of Preference Base Case Shares 18.10 37.43 44.47 Price Increase Gold 19.78 41.95 38.27

Bronze Silver Gold

How might the relative shares of preference change if seating capacity or position relative to the field were changed? You may wish to run additional scenarios to test those possibilities.

676

SSI Web v7

6.1.3

Is CVA the Appropriate Technique?

Many methods are available for collecting and analyzing conjoint data, and the researcher contemplating a conjoint study must choose among them. We at Sawtooth Software have had many years of direct experience with these methods, as well as the benefit of many conversations with users of our own and other software. Based on that experience, we offer the following suggestions: The Full Profile Method (such as used in CVA) was the first conjoint method introduced to the marketing research community, and it is considered the original standard. Green and Srinivasan (1990) recommended use of the full profile method when the number of attributes was six or fewer. Our opinion is that this is a useful guideline, but there isn't a hard limit, and the number of attributes should depend on the complexity of the attribute descriptions and respondent interest/knowledge of the product category. During the 1980s and 1990s, we tended to favor ACA for projects involving about six or more attributes. As choice-based approaches with HB estimation have captivated the industry and shown their worth, we have shifted our attention toward CBC and Adaptive CBC. Adaptive CBC combines some of the best aspects of ACA of CBC, and the early evidence suggests that it is a valuable new technique. We have not conducted nor seen research that directly compares ACA to Adaptive CBC, but we expect that ACBC should have important advantages when the number of attributes is between 5 and 12. When the number of attributes exceeds about 15, and pricing research isn't a goal, we don't know which method would prevail. CVA is a "main effects only" model, and assumes there are no interactions among attributes. Many conjoint practitioners agree that one must remain alert for the possibility of interactions, but that it is usually possible to choose attributes so that interactions will not present severe problems. Like other conjoint methods, CVA can deal with interactions in a limited way by collapsing two attributes into one. For example, we could deal with an interaction between car color and body style by cross-classifying the levels: Red Convertible Black Convertible Red Sedan Black Sedan However, if the attributes in question have many levels, or if an attribute (such as price, for example) is suspected of having interactions with many others, then collapsing attributes will not be enough. In that case too many parameters must be estimated to permit analysis at the individual level, and the most common solution is to evaluate interactions by pooling data from many respondents. When CVA is executed well, it provides solid results for applications such as product design, pricing, and segmentation work. It can work with the smallest of sample sizes, even in the case of revealing the preferences for a few or even one individual. Even though the bulk of interest lately is in favor of CBC and ACBC, those who have become expert in CVA methodology and CVA study execution will continue to use the method confidently in appropriate contexts for many years to come. For more information about selecting the appropriate conjoint method, see the article in the technical papers library on our website entitled: "Which Conjoint Method Should I Use?" or use the Interactive Advisor at http://www.sawtoothsoftware.com/products/advisor/.

CVA Help

677

6.2
6.2.1
6.2.1.1

Designing/Creating a CVA Study

Attributes and Levels
Defining Attributes and Levels
CVA projects allow up to 30 attributes, with up to 15 levels per attribute. (Hint: you can select a list of attributes or levels from Word and paste into SSI Web using the "paste from word processor or spreadsheet" icon. This can save a great deal of time.) An attribute is a characteristic of a product (e.g. color) which can take on various levels (e.g., red, yellow, blue). Every attribute must have at least two levels. The underlying theory of conjoint analysis holds that a buyer places a certain part-worth (or utility value) on each attribute level, and that the overall utility of any product is obtained by summing up the part-worth values of its specific attribute levels. In conjoint experiments, we show respondents product concepts that are described by different combinations of attribute levels and ask them to somehow express their preferences for those concepts. One way of thinking about conjoint analysis is that we are conducting a designed experiment for each respondent to see how his/her preferences vary as we change attribute levels. By observing how respondents evaluate products in response to changes in the underlying attribute levels, we can estimate the impact (part-worth) each attribute level has upon overall product preference. Once we learn respondents' preferences for the various attribute levels, we can predict how buyers might respond to any potential combination of levels in our study, whether or not that actual product was ever displayed during the interview.

Ranking (Applies only to ACA or CVA-type conjoint studies)

For each attribute, you can specify whether it has known (a priori) order. If you know ahead of time that respondents prefer low prices to high prices (all else equal) or fast speeds to slow speeds, you can avoid having to ask what might seem to be an obvious question. For CVA studies, that means avoiding showing products that are clearly superior on all aspects to another. We strongly encourage you to specify an order for attributes that have known rational order, like price and speed. When you specify a ranking order "worst to best" or "best to worst," you must be certain that the rational respondent would agree with you, and that you have specified the correct order. For example, the levels: Level 1: $5 Level 2: $10 Level 3: $15 are arranged from "Best to Worst," meaning, the best level is first and the worst level is last.

Guidelines
In generating attributes and levels, consider the following guidelines: 1. Attributes should be independent. It is therefore important to economize; including attributes with

678

SSI Web v7 overlapping meanings is wasteful and can lead to incorrect answers. Furthermore, levels for related attributes may not combine naturally with one another. Though it can lead to more realistic interviews, it is usually detrimental (and sometimes fatal) to prohibit levels from occurring with others. (There are some intriguing exceptions to this that we will not discuss in this documentation.) 2. Levels within each attribute should be mutually exclusive. This point becomes clear when you specify products using the market simulator (during the analysis phase) and are forced to choose only a single level from each attribute. Consider the following attributes for a study on optional automobile features: Optional Features: Sunroof GPS (Global Positioning System) Extended warranty This formulation doesn't permit simulating preference for a car that has both a Sunroof and a GPS. Similarly, we could not simulate preference for an automobile that had none of these features. There are two ways to resolve this quandary: a) Create an attribute with all potential combinations of these features. This results in an eight-level attribute, if you include the option that none of these features is available. b) Formulate three separate attributes each with two levels: (No Sunroof, Sunroof), (No GPS, GPS), (No Warranty, Warranty). Option (a) adds seven parameters to the model and forces the measurement of an explicit three-way interaction. With the more complex model definition, we can investigate whether there are diminishing returns by bundling the features. By splitting the options into three distinct binary attributes (Option b), only three parameters are added to the model; however, interactions are not measured. 3. Attribute levels should cover the full range of possibilities for existing products as well as products that may not yet exist, but that you want to investigate. Although the market simulator allows you to extrapolate and interpolate, only linear interpolation and extrapolation are possible. Although interpolation is likely to produce acceptable results, extrapolation is prone to error and should be avoided. One way to ensure that you are including the appropriate levels and ranges is to ask your client to specify ahead of time the market simulations to be run during the analysis phase of your study. That exercise can often reveal weaknesses in your attribute specifications. 4. Prohibitions, if at all possible, should be avoided. Specifying unnecessary or excessive prohibitions is one of the most common mistakes. The problem usually begins when either the analyst (or the analyst's client) notices that some product combinations displayed during the interview are not realistic, given what currently exists in the market. Sometimes a product is shown with all the best features at the lowest price; or two attribute levels that would not naturally occur in the real world are paired together. The inclination is simply to prohibit such combinations. We urge you to exercise restraint when considering prohibiting pairs. Too many prohibitions can lead to imprecise utility estimation and unresolvable (confounded) effects and the complete inability to calculate stable utilities. It is better to prompt respondents that they will see combinations during the interview that are not yet available in the market or that seem unlikely. You can urge respondents to answer as if these products were actually available today. There are other strategies for dealing with prohibitions. Consider the example below with brands of soda and package types:

CVA Help Brand: Sawtooth Spritz Kong Kola Splut Package Type: 2-liter bottle 6-pack of 12-oz cans

679

Suppose that Splut was only available in 6-packs of cans. Furthermore, you are displaying actual pictures of the products, and thus can only display actual (not potential) products. Rather than define a prohibition between Splut and the 2-liter bottle, it would make more sense to combine these two attributes as a single attribute with five levels: Sawtooth Spritz in a 2-liter bottle Sawtooth Spritz in a 6-pack of 12-oz cans Kong Kola in a 2-liter bottle Kong Kola in a 6-pack of 12-oz cans Splut in a 6-pack of 12-oz cans Under this strategy, no prohibitions are required. CBC and CVA are even more sensitive to too many attribute prohibitions than ACA. ACA tends to be more robust in the face of attribute prohibitions given that a significant portion of the information used to estimate part-worth utilities comes from the priors section, which is not affected by attribute level prohibitions. And, partial-profile designs generally are less affected by prohibitions than full-profile designs. 5. The number of levels you choose to define an attribute can have a significant bearing on the results. The first concern has been called the "Number-of-Levels Effect." All else equal, attributes defined on more levels tend to get more importance. The Number-of-Levels Effect is less problematic in ACA than full-profile conjoint methods. Even so, we suggest you at least approximately balance the number of levels across attributes. The second concern is that you limit the number of levels on which quantitative attributes are described. We suggest not including more than about five levels to describe attributes such as price or speed. It's usually better to have more data at each price point than to have thinner measurements at more price points. Measuring too many points along a quantitative function can result in troublesome reversals. If you cover the entire range of interest with fewer levels, you can interpolate between levels within the market simulator to get finer granularity if needed. 6. Attributes that cannot be adequately described in words should be represented in multimedia. But if attributes do not require multimedia to adequately communicate their properties, it would probably be a mistake to make them multimedia. Though the interview might appear more attractive, it might bias the results in favor of multimedia attributes. 7. In our experience, respondents have a difficult time dealing with more than about six or eight attributes in full-profile conjoint methods like CBC or CVA. When faced with too much information, respondents may resort to simplification strategies to deal with the difficulty of the task. Unless respondents employ the same sort of simplification strategies when making real-world decisions, CVA results may place too much emphasis on the few most important features. Traditional CBC or CVA will work well for relatively small conjoint designs, and we generally suggest other methods (such as Adaptive CBC or ACA) for larger problems.

680 6.2.1.2

SSI Web v7

Level Prohibitions and CVA Studies

Prohibitions should be used sparingly, or not at all. Specifying unnecessary or excessive prohibitions is one of the most common mistakes. The problem usually begins when either the analyst (or the analyst's client) notices that some product combinations displayed during the interview are not realistic, given what currently exists in the market or what the client is considering to offer. Sometimes a product is shown with all the best features at a relatively low price; or two attribute levels that would not naturally occur in the real world are paired together. The inclination is simply to prohibit such combinations. Newcomers to conjoint analysis commonly think that by specifying prohibitions they can concentrate the respondent's effort on the product combinations of most interest to the client, which should in turn improve utility estimation for those most relevant combinations. This unfortunately is not always the case with main-effects utility estimation, which CVA (like most conjoint approaches) employs. Prohibiting certain combinations of levels from occurring often decreases the overall quality of the utility estimates. In other words, by asking respondents to evaluate even those product combinations that the client is unlikely to offer, we often improve the estimates of the product combinations that the client is most likely to consider. We urge you to exercise restraint when considering prohibiting pairs. Please see the discussion of prohibitions in the CVA Tutorial and Example section of the CVA documentation for an example and guidance regarding using limited prohibitions to achieve specific objectives in CVA designs. Too many prohibitions can lead to imprecise utility estimation. It is better to prompt respondents that they will see combinations during the interview that are not yet available in the market or that seem unlikely. You can urge respondents to answer as if these products were actually available today. Users commonly ask, "how many prohibitions can I specify without seriously damaging the results of my study?" This cannot be answered without more information. It is not the sheer number of prohibitions but the specific pattern of prohibitions that more directly affects the degree of correlation among the attribute levels in the design matrix, and thus the design efficiency. Assume that the researcher wants to specify three prohibitions between a two-level attribute and a three-level attribute. There are just six possible combinations that can occur when combining those two attributes. If the researcher prohibits three of those combinations from occurring, this eliminates half of the possible combinations. There would probably be less damage to the efficiency of the design if three total prohibitions were allocated across three separate attribute combinations (one per pair of attributes). The researcher in this instance may decide to specify the three non-prohibited combinations of those two attributes as a single attribute, rather than as two separate (and not independent) attributes. The main drawback of this approach is that after combining these two attributes, the researcher cannot compute the attribute importance or the relative part-worths of each attribute independently.

CVA Help

681

6.2.2
6.2.2.1

Designing the CVA Tasks (Questions)

Introduction to CVA Questionnaires
CVA offers two main questionnaire layout options: single concept (ranking or rating), and pairwise comparison (ratings only). The approach you use is driven by which format best reflects decisionmaking for the market you are studying, and the kinds of data analysis you need to perform. Once you have entered your attributes and levels and specified a layout (single-concept or pairwise comparison) you are ready to generate an experimental design (the conjoint questions). CVA's designer uses a computer search algorithm that seeks to maximize D-efficiency, given the specifications for your study (i.e. attributes and levels, and number of requested tasks). Designs with high D-efficiency support independent and relatively efficient estimation of the part-worths.

Pairwise Presentation
CVA uses Pairwise Comparison as the default design and question presentation format. With pairwise presentation, respondents are asked to compare two products, as shown below:

With small numbers of attributes, the paired comparison format has desirable characteristics. Paired comparisons have a long history in the field of psychometrics, where they have been the method of choice for the precise measurement of small subjective differences. When respondents compare two products side-by-side, it can help them draw finer distinctions between products offering different features. However, if two products are shown per question, respondents must evaluate twice as much information as single concept designs (described below). All research we are aware of to date suggests that relative part-worths are very similar whether estimated from single concept or paired comparison conjoint questionnaire formats. We suggest the pairwise design for most conjoint projects. However, the pairwise questionnaire only captures the relative differences in a respondent's preferences for attribute levels. Pairwise questions never measure respondents' absolute level of interest in the product concepts. Comparative information is all that is necessary for estimating relative part-worths and running competitive market simulations. If you need to run "purchase likelihood" simulations within the market simulator, you should use the single concept presentation, described next.

682

SSI Web v7

Single Concept Presentation

Single concept presentation is commonly known as "card-sort." Respondents can be shown one product at a time and asked to rate how desirable each is or how likely they would be to purchase each. An example is shown below:

This method is particularly useful if the "purchase likelihood model" is to be used in the simulator. CVA's ordinary least squares (OLS) routine is appropriate for analyzing single concept ratings. Alternatively, each of a set of product profiles can be printed on individual cards, and respondents can be asked to sort the cards from most to least preferred. In this case, the responses are rank orders of preference, and CVA's monotone regression routine is suggested for developing utilities. Ranking tasks (like pairwise comparisons) only elicit comparative information.

CVA Help 6.2.2.2

683

Advantages and Challenges of CVA Questionnaires

Although CVA's question format has advantages in studies with small numbers of attributes, its task can be challenging for the respondent. To be sure that your CVA questionnaire is appropriate for the study, you must: keep the number of attributes small, pretest the questionnaire, and conclude that the resulting utilities are reasonable, before you proceed. Any conjoint questionnaire can be difficult for the respondent, but CVA questionnaires can be particularly so. Unlike our ACA System for Adaptive Conjoint Analysis, CVA does not use any strategy for simplifying the respondent's task. Every question involves consideration of all the conjoint attributes. CVA can create either single concept (card sort) or pairwise comparison (two concepts at a time). With pairwise questions, the amount of information the respondent must consider is twice as great as with single concept methods. Unless the number of attributes is quite small, (no more than about six to eight) there is risk that respondents may become overloaded, and that the quality of the data collected may suffer. For this reason, we recommend other types of questionnaires if many attributes must be studied. In summary, although the CVA question format can be very effective when the number of attributes is not too large, we are concerned that inappropriate use of CVA may produce questionnaires that are too difficult for respondents. As with any conjoint method, but particularly so in this case, it is essential to pretest the questionnaire: answer the questionnaire yourself, and analyze your own data to make sure the part-worth utilities mirror your own values. have others answer the questionnaire to report to you about whether it's too difficult. have a sample of relevant respondents answer the questionnaire, and analyze their data. Be particularly alert for "nonsense" results with respect to price. If many respondents have higher utilities for higher prices, then the data are suspect. If you do keep the number of attributes small, and if you do pretest the questionnaire and conclude that the resulting utilities are reasonable, then you can proceed with confidence.

684 6.2.2.3

SSI Web v7

Selecting the Number of Tasks

After you have defined attributes and levels and decided which questionnaire format is best for your study, you need to decide how many conjoint questions (tasks) to ask. This decision depends on how many separate part-worth parameters need to be estimated. The more attributes and levels in your study, the more part-worths to be estimated; and the more parameters to estimate, the longer the questionnaire. CVA examines the list of attributes and levels in your study and provides an initial recommendation regarding the number of tasks (conjoint questions) to ask. The recommended number of tasks provides three times the number of observations as the number of parameters to be estimated. The number of parameters to be estimated is determined by the formula: Total number of levels - number of attributes + 1 Although it is mathematically possible to estimate utilities by asking only as many questions as the number of parameters to be estimated, we strongly recommend against this practice when using standard estimation routines (OLS and Monotone Regression). Asking the recommended number of tasks helps ensure enough information to calculate stable estimates for each respondent. If your design includes many attributes and levels, you may notice that the recommended number of questions exceeds what you could reasonably expect respondents to complete. This leads to a difficult decision. Too few questions limit the degrees of freedom of the design (and the precision of the partworth estimates); but if you ask too many conjoint questions respondents may become fatigued or bored and provide data of poor quality. In our experience, asking more than about 30 conjoint questions may result in poor quality data. Testing your questionnaire with actual respondents can help you determine the optimal number of questions to ask for your project. CVA will not let you specify fewer questions than the number of parameters to be estimated, and will warn you if you don't ask at least 1.5x the number of parameters to be estimated. A number of studies in the literature suggest that some experienced conjoint analysts are willing to ask as few as 1.5x the number of questions as parameters to estimate. Indeed, the use of HB estimation may slightly reduce the number of questions you decide to ask respondents, as it estimates part-worths based on information from the current respondent plus information from the other respondents in the same data set. Note: if using HB estimation, it is possible to ask respondents fewer questions than parameters to be estimated, while still obtaining good aggregate results. (See Sparse Designs and Large Samples for more info.) In that case, the questionnaire design still contains more tasks than parameters to estimate, but each respondent is asked to complete just a random subset of the tasks. Please note that there is loss in information at the individual level by taking this approach (relative to asking respondents to complete all tasks). If having respondents skip tasks, you should use larger sample sizes than usual to compensate for the loss in information .

CVA Help 6.2.2.4

685

Throw out "Obvious" Tasks

CVA can use the a priori preference relationships (i.e. "best to worst") specified when you provided your attributes to avoid presenting conjoint profiles that would result in obvious answers. For example, questions including a product that has many good features at a low price or many bad features at a high price may be less efficient for deriving conjoint utilities. Questionnaires that avoid the most obvious comparisons are more challenging for respondents and provide more useful information. A moderate degree of balance in the expected utility of the concepts in CVA questionnaires is therefore desirable. However, we caution that too much balance can increase the difficulty of the questionnaire and the error in the responses, counteracting the benefits of improved utility balance. CVA's method of discarding "obvious" tasks takes a conservative approach that eliminates only the most extreme cases. Advanced researchers can further balance the design by specifying additional prohibitions. This may be a reasonable approach so long as the researcher conducts substantial quantitative pretests to ensure that the benefits from utility balance are not counteracted by increased response error. Recall that pairwise designs show a product concept on the left and another on the right. For each attribute for which CVA has a priori order information, if the concept on the left has a higher utility than the one on the right, a +1 is scored; if it has a lower utility, then it is scored -1. For single-concept designs, only extreme levels are counted. Highest utility levels are scored +1, and lowest levels -1. Non-a priori attributes are ignored, because CVA has no way of knowing which level is preferred. If the imbalance exceeds some threshold, then the task is considered "obvious" and discarded. The threshold used to determine "obvious" tasks is >75% of a priori attributes marked as +1. The following table below shows the threshold used for each number of attributes (up to 10) to determine "obvious" tasks: "Obvious" Tasks Threshold # TOTAL Attributes 2 3 4 5 6 7 8 9 10 Threshold 2 3 3 4 5 6 6 7 8

For example, for studies involving just 4 attributes, the threshold score must reach 3 before the task is considered "obvious." Because a modest degree of utility balance is a desirable quality, it may make sense in the design phase to set any attribute to a priori for which you have a reasonably good idea about which levels are preferred to others. For example, if some brands are generally preferred to others, you might specify them in order from generally least to most preferred and then set the a priori setting to "worst to best." If you take this approach, make sure to remove any a priori setting for all attributes that do not have a clear order of preference before calculating utilities (under Additional Utility Constraints), or you may impose constraints that are not consistent with some respondents' preferences.

686 6.2.2.5

SSI Web v7

CVA Design Strategy

After you have specified attributes, any a priori ordering, how many tasks to ask, and whether to use "single" or "pairwise" concept tasks, you are ready to generate a design. The design specifies the combination of attribute levels (profiles) shown in each conjoint question, for a single or multiple versions of the questionnaire. To generate a design using the defaults, click Generate Design. Number of Tasks (Questions) Based on the number of attributes and levels in your study, CVA provides an initial recommended number of tasks (conjoint questions). The recommendation is based on asking three times as many tasks as parameters to be estimated, where the number of parameters to estimate is equal to: Total number of levels - number of attributes + 1 The recommended number is an ideal number from a statistical standpoint that often is not used in practice. The recommended number is often more than respondents can reasonably complete. Use this as a guideline, then specify how many tasks you actually want to use within the software. One of the most important decisions with a conjoint design is how many questions to ask. If you don't ask enough questions, it may result in noisy part-worth estimates. If you ask too many questions, you may overtax your respondents, leading to decreased data quality and/or abandoned surveys. CVA warns you if you do not ask at least 1.5x as many tasks as parameters to be estimated, and does not let you ask fewer tasks than the number of parameters to estimate. See Selecting the Number of Tasks for more information. Randomize Attribute Position within Concepts Randomize Attribute Order Randomize Attribute Order specifies whether to present the attributes in random presentation order within a concept. If this is selected, the attribute list is randomized once per respondent, and all tasks within a respondent's interview will display the attributes in that given order. This can be useful to control order effects across respondents. First Randomized Attribute The first attribute in the range of attributes to be shown in random presentation order. Specify a "1" if you wish all attributes to be randomized. If, for example, you want the first and second attributes always to appear as the first two attributes in a product concept, specify a "2." Last Randomized Attribute The last attribute in the range of attributes to be shown in random presentation order. Specify the last attribute number if you wish all attributes to be randomized. If, for example, you had a total of five attributes in your study and you wanted the fourth and fifth attributes always to appear as the last two attributes in a product concept, specify a "3." Advanced Settings When you click Show Advanced Settings, the following fields are available: Number of Versions (default=10): A version of the questionnaire represents a single series (block) of conjoint questions. If you want respondents to receive different sets of questions, you can request multiple versions (up to 10). If you are conducting paper-and-pencil studies, you probably

CVA Help

687

do not want to manage more than a few different versions because of the increased hassle of dealing with unique questionnaire versions. However, when you are able, additional questionnaire versions decrease psychological order and context effects and thus improve your overall results. See the section below entitled "A Single Version or Multiple Version?" for further guidance. Design Seed (default=1): CVA's design generation algorithm requires a starting seed. You can use any integer from 1 to 9999. If you repeat the analysis using a different starting seed, you will usually obtain a slightly different (sometimes better) result. Throw out Obvious Tasks (default = yes): CVA can exclude product concepts that are clearly better or worse than others from the questionnaire. To use this, you need to specify that certain attributes have a priori preference order. Task Pool Multiplier (default=10): When generating a questionnaire version, the Task Pool Multiplier (multiplied by the number of requested tasks) controls how many unique tasks will be used in the pool of candidate tasks to include. For example, if you request 18 tasks with a Task Pool Multiplier of 10, 180 unique tasks will be searched among to find an optimal 18 tasks. Version Pool Multiplier (default=10): This defines how many tries (passes) will be attempted from different starting points. For example, if you are requesting 10 questionnaire versions and the Version Pool Multiplier is 10, then 100 attempts will be made to find optimal versions of the questionnaire. The top 10 versions (in terms of design efficiency) will be used in the final plan. Hints: The defaults we've specified tend to work well and quickly. But, you can often improve your questionnaire's design efficiency by asking CVA to try harder. You can increase the Task and Version Pool Multipliers to search deeper and longer for better solutions. You can also change the random design seed to repeat the process from different starting points.

CVA Design Procedure

The conjoint design is a critical component to the success of any conjoint project. Attributes must vary independently of each other to allow efficient estimation of utilities. A design with zero correlation between pairs of attributes is termed "orthogonal." Level balance occurs if each level within an attribute is shown an equal number of times. Designs that are orthogonal and balanced are optimally efficient. In the real world, it might not be possible to create a perfectly balanced, orthogonal design for a particular set of attributes and prohibitions consisting of a reasonable number of tasks. The CVA approach produces high quality designs automatically, although they probably will not be perfectly balanced or orthogonal. CVA lets you test the efficiency of a design before fielding your study. Testing the design also lets you study the impact of including prohibitions, or asking fewer than the recommended number of questions. CVA provides an easy-to-use tool for generating well-balanced, "nearly-orthogonal" designs. CVA generates a pool of potential conjoint questions (from which the final design will be chosen) using a relatively simple procedure. If the total number of concepts to generate in the pool is greater than or equal to the number of total possibilities, a complete enumeration is done of all non-prohibited possibilities. If the pool represents a subset of all possible combinations, for each question to be composed CVA does the following: For each attribute it picks a pair of levels randomly from among all permitted pairs that have been presented the fewest times. A random decision is made about which levels will appear on the left and on the right (for pairwise designs). No pair of levels will be repeated until all other permitted pairs have been shown. Each pair of levels will be shown approximately the same number of times, and each level from one attribute is equally likely to be shown with any level from another attribute.

688

SSI Web v7 CVA's designer uses the following steps to select efficient designs, given the questionnaire specifications: 1) CVA generates a pool of potential conjoint questions equal to, by default, 10 times the requested number of questions (assuming that many unique questions exist). 2) The D-efficiency of the design is calculated for the pool, excluding one conjoint question at a time. The one task that contributes least to the efficiency of the design is discarded, and the process repeated until the desired number of tasks remains. (See Technical Notes about the CVA Designer for more information about D-efficiency). 3) CVA then examines every potential 2-way swap of conjoint questions that remain with those that were discarded or are available in the pool of potential conjoint questions. CVA swaps any pairs of questions that result in an increased efficiency. 4) Next, CVA examines the frequency of level occurrences for each attribute. It investigates changing levels that are over-represented in the design to levels of the same attribute that are under-represented. Any changes that result in improved D-Efficiency (and are not prohibited) are retained. 5) For pairwise designs, CVA flips left and right concepts to improve the left/right balance of the design. CVA repeats steps 1 through 5 multiple times and selects the best n solutions (where n is the total number of versions of the questionnaire you'd like to use). CVA uses re-labeling (trading pairs of levels within attributes across all tasks within a design) to create approximate aggregate level balance across all versions. (Relabeling doesn't change the efficiency of each individual design.) An additional check is performed to ensure that no two versions of the questionnaire are identical.

A Single Version or Multiple Versions?

Previous versions of our CVA software employed a single version (block) of the questionnaire design. Each respondent received the same questionnaire (though the order of the tasks could be randomized). From a statistical standpoint, a single version of the plan was all that was typically needed to ensure precise estimates of main effects. However, researchers recognize that respondents are human, and therefore the quality of the estimated utilities will depend on controlling psychological/order/context effects. Randomizing the order of the tasks is a good way to control for context effects. However, these effects may be further reduced by using more than one version of the questionnaire. In addition, CVA/HB for hierarchical Bayesian analysis of part-worth utilities would seem to benefit from multiple versions. Because HB pools information across respondents, having greater variation in the design matrix (across all respondents) may improve individual estimates due to improved population estimates. Multiple versioning is nothing new for users of Sawtooth Software's tools. In our CBC software, the default (when using computer interviewing) is to use 300 versions of the questionnaire. In reality, this is probably overkill, since the vast majority of the benefits of multiple versioning are captured after the first few versions. But, since CBC's algorithms for developing designs are generally fast and computer interviewing automatic, it is as easy to develop and field 300 versions as 4. With CBC, there is another good reason for including multiple versions of the questionnaire: choice-based designs can be stronger than traditional conjoint for estimating interaction effects, and additional versions of the questionnaire help stabilize those effects (assuming pooled estimation). With CVA, there are benefits for including multiple versions of the questionnaire (to help reduce psychological effects), but there are reasons to believe that employing a very large number (such as

CVA Help 300) is not necessary.

689

1. CVA's search algorithm tries to find the optimally efficient set of questions given a requested number of tasks. In contrast, CBC uses a "build up" approach that while achieving very good design efficiency, doesn't attempt to find the optimal set of questions given a requested number of tasks. Therefore, it can be expected that CVA is better suited to choosing a smaller number of versions that are quite effective statistically. 2. CVA only estimates main effects (a single utility value for each level in the study). CBC can estimate both main effects and first-order interactions. The goal of estimating main effects suggests requiring fewer questionnaire versions. 3. CVA is generally a slower (and more thorough) way to generate designs than CBC. Generating a very large number of versions by default (such as 300) could take a significant amount of time. Because of these arguments, CVA permits up to 10 different versions of the design. In our opinion, the benefits for using more than 10 versions with CVA would be miniscule. For two decades, previous versions of CVA have used single-version plans, with generally good results. Allowing a few additional versions is convenient when interviewing via computer, and should lead to slight improvements. When fielding multiple versions of the questionnaire, it is desirable (but in nowise a requirement) that each version of the questionnaire be completed by approximately the same number of respondents. When fielding your study on the Web, this happens automatically. Each respondent starting a CVA survey is given the next available design. Once the last design is reached, the next respondent receives the first design, and the process repeats.

Design Suggestions
We have stressed the importance of including enough conjoint questions to ensure efficient designs. However, design efficiency is not the only reason for including two to three times as many questions as parameters to be estimated. All real-world respondents answer conjoint questions with some degree of error, so those observations beyond the minimum required to permit utility estimation are useful to refine and stabilize utility estimates. While there is generally a positive relationship between the number of conjoint tasks in the design and D-efficiency, there are many exceptions. It is possible for designs with few or no degrees of freedom to have 100% D-efficiency. This means that this design is optimally efficient for estimating main-effect parameters. But this assessment ignores human errors. By increasing the number of tasks, you provide more opportunities for respondent errors to cancel themselves out. Increasing the number of tasks in this case (beyond the saturated orthogonal plan) will slightly reduce the reported D-efficiency (from 100% to something slightly less than 100%), but the precision of the estimated parameters may be significantly improved due to a greater amount of information provided by each respondent. To summarize our point, don't focus solely on D-efficiency. Good CVA designs foremost should include enough conjoint questions relative to the number of parameters to yield relatively precise estimates of part-worths. Given an adequate number of conjoint questions, we next focus on selecting a design with high D-efficiency.

Importing and Exporting Designs

You may import or export designs, as described in the section of this documentation entitled Importing/Exporting CVA Designs. This is useful if you need to use a particular set of questions developed by a colleague, using a different piece of software, or even from a design catalog. Importing designs is also a way to add user-specified holdout questions.

690 6.2.2.6

SSI Web v7

Testing the CVA Design

After you have created a design (either through CVA's designer or by importing a design from a text file), you can review the D-efficiency and receive additional diagnostic information by using CVA's Test Design functionality. When you select this option, a report is shown on the screen:
CVA Design Efficiency Test Copyright Sawtooth Software Tasks are 'Single Concept' using a seed of 1. Based on 10 version(s). Includes 140 total tasks (14 per version). Each task includes 3 attributes. Design Version Efficiencies ------------------------------------------------------------Version Efficiency 1 0.97528 2 0.97528 3 0.97528 4 0.97528 5 0.97528 6 0.97528 7 0.97528 8 0.97528 9 0.97528 10 0.97528 Attribute Efficiency Coefficients (including all versions) ------------------------------------------------------------Attribute Efficiency 1 0.999 2 0.995 3 0.999 One-Way Frequencies (by version) ------------------------------------------------------------Level Total Ver1 Ver2 Ver3 Ver4 Ver5 Ver6 Ver7 1 70 7 7 7 7 7 7 7 2 70 7 7 7 7 7 7 7 1 2 3 1 2 3 4 46 48 46 35 35 35 35 4 4 6 3 4 3 4 4 4 6 4 4 3 3 4 4 6 3 3 4 4 6 4 4 3 4 3 4 6 4 4 4 4 3 3 6 4 4 4 3 4 3 4 6 4 3 3 4 4

Ver8 7 7 4 6 4 4 3 4 3

Ver9 7 7 4 6 4 4 4 3 3

Ver10 7 7 4 6 4 3 3 4 4

Two-Way Frequencies (all versions) ------------------------------------------------------------Att/Lev 1/1 1/2 2/1 2/2 2/3 3/1 1/1 70 0 23 24 23 17 1/2 0 70 23 24 23 18 2/1 23 23 46 0 0 12 2/2 24 24 0 48 0 12 2/3 23 23 0 0 46 11 3/1 17 18 12 12 11 35 3/2 18 17 12 11 12 0 3/3 18 17 11 13 11 0 3/4 17 18 11 12 12 0

3/2 18 17 12 11 12 0 35 0 0

3/3 18 17 11 13 11 0 0 35 0

3/4 17 18 11 12 12 0 0 0 35

The Test Design report shows you the D-efficiency of each design that was generated (CVA generates 10 design versions by default), which provides an indication of the overall quality of each design. The

CVA Help

691

highest possible number is 1.0, and the lowest is zero. For more information about D-efficiency, see Technical Details on the CVA Designer. Next, CVA reports attribute efficiencies (described in greater detail below), assuming all designs are appended as an aggregate design plan. Low efficiencies for some attributes can be an indication that some levels of that attribute might not be included enough times in the design, or that this attribute has too many prohibitions (and is thus correlated with another attribute(s)). These coefficients can be useful diagnostics for a conjoint questionnaire design during the pilot stages of a study. If the relative efficiency is too low for a particular attribute, then additional questions should be introduced, or the design should be modified to have less dependence among the attribute levels. If you have low D-efficiency for some or all designs, or if some attributes have particularly low efficiencies, you should consider removing constraints, asking more questions, or regenerating your design. Poor efficiencies can compromise the quality of your utility estimates. Next, the one-way frequencies are displayed, considering all designs together ("Total" column) and then for each design separately ("Ver 1" through "Ver 10" columns). The frequencies reflect the number of times each level occurs in the design. Following that are the joint frequencies, or the number of times two levels appear together within the same product concept, across all designs. Although not shown above (to conserve space), similar joint frequency tables follow for each separate design.

More Details on Attribute Efficiency Calculation

The Attribute Efficiency statistics describe the quality of the conjoint design. We are mainly interested in the precision of estimates of differences in utility among levels of the same attribute. The relative error variance of the difference is obtained from elements of the inverse of the correlation matrix among independent variables. The relative error variance of the difference between two utility estimates is obtained by summing those two diagonal elements of the inverse and subtracting twice the corresponding off-diagonal element. We compute the relative error variances of all within-attribute pairwise differences. Suppose there were an orthogonal design for the same attributes and levels, and all the levels of a particular attribute appeared the same number of times. (Such a design may or may not actually exist.) Then we could compute similar relative error variances for that "ideal" design. Our "Relative Efficiency" index is the ratio of the sum of theoretical relative error variances for such an ideal design, divided by corresponding sum of relative error variances for the actual design used. The overall relative efficiency index is reported, as is an index for each attribute. The best possible value is 1.000, although values that high may not actually be achievable for some combinations of numbers of attributes and levels, since the required orthogonal designs may not exist.

692 6.2.2.7

SSI Web v7

Importing/Exporting CVA Designs

One of the strengths of the CVA system is its ability to generate experimental designs (the combinations of attribute levels to show to respondents) in an efficient and easy manner. Also, multiple versions (up to 10) of the questionnaire may be generated, so different respondents see different combinations of attributes within the conjoint tasks. Some researchers may wish to export the design, to examine ahead of time the combinations that will be asked of respondents (and perhaps to perform additional analysis regarding the design). Other researchers may have developed questionnaire designs using other software and wish to import those designs for fielding within the CVA system. They can do this through CVA's Import Design capability. Another use of the Import feature is to add holdout conjoint questions ("fixed" CVA questions that are asked exactly the same way across all questionnaire versions) in your study design. For all purposes, the design file uses the same comma-separated (.CSV) layout. A .CSV file can be directly opened and saved with Excel, and the examples below assume you are working with the comma-separated file in Excel. (If using a text editor, the values in the file are separated by commas.) If you import a design, any existing design will be overwritten. Import/Export File Format The file format is as follows. Column 1 Indicates the questionnaire version. Column 2 Indicates the task (conjoint question) number. Column 3 Indicates the concept number. Designs showing one product concept at a time always have concept #1. Designs showing two product concepts at a time (pairwise presentation) have concept #1 and concept #2. Columns 4 through n Shows the attribute levels associated with the current concept, task, and version number. Single-Concept Presentation: In the example below, only two versions of the questionnaire are shown. There are three attributes in this study.

CVA Help

693

694

SSI Web v7 Pairwise (Two-Concept) Presentation: In the example below, only two versions of the questionnaire are shown. The design reflects pairwise presentation layout (two concepts per question). There are three attributes in this study.

CVA Help Additional Rules: The version must start with 1 and proceed in that order.

695

The number of versions and number of tasks must match the number of versions and tasks specified in the software. The number of concepts must be constant across all tasks. The number of concepts must match the number of concepts specified in the software. Any prohibitions specified in CVA will not be checked against the imported design.

696 6.2.2.8

SSI Web v7

Including Holdout Choice Tasks in CVA Questionnaires Holdout Choice Tasks

One of the reasons that Choice-Based Conjoint has gained in popularity over the years at the expense of traditional full-profile conjoint is that the resulting CBC utilities are scaled based on the choice data (rather than taking its scaling from the arbitrary choice of the scale chosen in CVA). When applied in market simulations, these choice-developed utilities lead to shares of preference for competitive products that are directly linked to probabilities of choice. In contrast, CVA utilities are estimated from rating scales or rank-order data. These scales do not automatically lead to market simulations with appropriate choice probability scaling. CVA users can achieve approximate choice-probability scaling for utilities by adding holdout choice tasks within their CVA surveys (as Select, Grid, or Free Format question types). By tuning the scale factor (exponent) within Share of Preference or Randomized First Choice simulations to best fit holdout choice probabilities, the simulator may be tuned to resemble choicebased scaling. The tutorial in this online help describes the use of a holdout choice task. Additional information on holdout choice tasks can also be obtained from a technical paper entitled "Including Holdout Choice Tasks in Conjoint Studies" at www.sawtoothsoftware.com in the Technical Papers library. Holdout choice tasks present a set of product profiles (usually three or four) described in full profile using the same attributes and levels used in the CVA questionnaire. Respondents are asked to choose the one alternative from the set that they most prefer. The analyst observes the probabilities of choice across the competing alternatives by simply tabulating the selections. Then, these same product profiles are specified as a scenario in the market simulator. Using a Share of Preference or Randomized First Choice model, the analyst uses the CVA utilities to predict the probabilities of selection for the same alternatives in the holdout task. If the conjoint data are working well and are scaled properly, the predictions should closely match the observed choice probabilities. The analyst tunes the exponent (scale factor) in the market simulator so that the predicted choice probabilities most closely fit the observed probabilities of choice from the questionnaire. Unless you have additional choice information by which to rescale your utilities for use in market simulations, the scaling of shares of preference may not be appropriate when using CVA. We strongly encourage you to use holdout choice tasks in every CVA study, and that you tune your market simulations using the exponent to fit these holdout choices.

CVA-Looking Holdout Tasks

Some researchers may wish to include ratings-based holdout questions within their CVA studies that look exactly like the other CVA questions. To do this, you can export your CVA design to a .csv file, and then modify the design file to insert a few additional holdout tasks. Then, modify the design settings to specify a new number of total questions (including the additional tasks you've added). Then, import the new design. When you conduct analysis, make sure not to include the holdout tasks within utility estimation. Within the Settings dialog, use the Tasks to Include filter to exclude the holdout tasks from utility estimation.

CVA Help

697

6.3
6.3.1

CVA/Web Paper and Pencil Studies

CVA/Web Paper and Pencil Studies
SSI Web lets you run your CVA studies via Web, over PCs (CAPI installation), or as paper-and-pencil questionnaires. The process of running paper-based CVA studies is very similar to computer-based CVA studies. You should set up your attributes, levels, and compose your conjoint tasks in the same manner as if you were going to run a computerized study. However, you should probably choose plain black (text) on white (background) for questionnaire colors, unless you plan to print the questionnaires in color. By default, CVA's questionnaire designer assumes computerized interviewing with 10 questionnaire versions (you can change the number of versions through the Advanced Settings). Because of the added costs for managing multiple versions of the questionnaire in the field (and the added possibility of introducing an error), you will probably not want to employ so many versions within paper-and-pencil interviewing. You will create one or just a few versions (design blocks) of the questionnaire and assign respondents (randomly) to take one of the different questionnaire versions. For more information about the choice of how many versions to use in the questionnaire, please see the section on CVA Design Strategies.

Saving the Questionnaire

When you click the Field | Create Paper & Pencil Interviews button, each version of your questionnaire is saved to a separate file within the Paper-and-Pencil folder within your study directory. The files are named STUDYNAME_EXERCISENAME_V#.htm, where STUDYNAME is the fourcharacter or less study name for your project, and # is the version number of the questionnaire (always beginning with version #1). These files are in HTML format and may be opened with most any recent word processing software. You should do any additional formatting you like to make the questionnaire appear as you wish prior to printing. Note: It is helpful to print the version# and task# with each task in a paper-and-pencil study. This gives an additional reference while performing data entry, helping ensure that you do not introduce errors when specifying the version numbers seen by respondents. An easy way to automatically label your choice tasks is to add a function [%CVAVersion( )%] in the header or footer of the choice task. When you do this, each task includes a label such as D - 1, T - CVA1, meaning "Design #1, Task CVA1."

Fielding the Questionnaire

Make sure when fielding the questionnaire that you are able to match the different questionnaire versions correctly with respondent answers. You should also try to randomly distribute the questionnaire versions among respondents, so that roughly equal numbers of respondents complete each version.

Data Entry
After you have collected the data, you prepare a .csv file containing the respondent answers. This file follows a required .csv (comma separate values) format, with specific labels in the first row of the file. To see that format, prepare a template for yourself by clicking Field | Create Accumulated Data Template File.... A file named Accumulated Data.csv is saved to your study folder. Use this

698

SSI Web v7 template to arrange your data, and make sure to use the prescribed header row within your data file. The layout is: Column 1: Respondent# Column 2: Version# Column 3: CVA answer #1, etc. Any missing data are coded as blank (meaning, if you viewed the data file using Excel, the cell would be empty; if you viewed the file with a text editor, the missing value would look like two consecutive commas with no text in between).

Ranking Data Formats

If you have used single-concept presentation and asked respondents to rank the cards from best to worst, there are two possible formats you can use for recording the data. You use a comma-separated file, as described directly above. However, you can either record the data as: Answers are ratings/rankings Answers are card numbers If the answers are ratings/rankings, the third field in your data file contains the ranking assigned to Card #1 in your design. The fourth field in your data file contains the ranking assigned to Card #2, etc. If the answers are card numbers, then the third field in your data file contains the card number for the card chosen as "best" by the respondent. The fourth field would contain the card number for the card chosen 2nd best, etc.

Accumulating the Data

When you have prepared the data as described above, you are ready to Accumulate Paper & Pencil Data. Click Field | Accumulate Conjoint Paper & Pencil Data.... Then, browse to your commaseparated data file (should carry a *.csv extension) that contains the respondent answers. Once you have accumulated the respondent data, you are ready to compute utilities using the standard dialogs used for any CVA study.

CVA Help

699

6.4
6.4.1

CVA Advanced Designs

Conditional Display for CVA
Conditional display may be used for showing a graphic to represent the occurrence of multiple attribute levels appearing together within a concept in a CVA task. For example, you may have two separate attributes for brand and package size, but a single picture reflects the joint brand/package size concept. Where the text for a particular brand and package size might appear in a CVA concept, you may wish to replace that with a graphic. But conditional display is not limited to just graphics, as it simply adds (or replaces, depending on your Attribute Visibility setting) the existing level text. Other possibilities include: Multi-media files Text JavaScript Attribute Label (Optional) If you are using Conditional Display and have chosen to show the graphic (or other conditional display "element") at the top or bottom of the conjoint question, you can optionally specify an attribute label to be shown in the conjoint question (on the left-hand side of the task). This attribute label is only used (and is only applicable, otherwise it is "grayed out") if you have specified that attribute labels are to appear in the choice task. You specify that attribute labels should appear within the choice task by clicking Compose | Conjoint Settings | CVA Settings and selecting the Format tab. Then, check the Include Attribute Label in Tasks box. Note: if in Step 3 of the Conditional Relationships Wizard, you specified that the conditional display element is to replace the text of a certain attribute within the task, then the attribute label used for the conditional element is the original attribute label for the replaced attribute text (that you originally typed within the Specify Attributes and Levels dialog).

700

SSI Web v7

6.4.2

Conditional Pricing for CVA

Warning: this is an advanced area. Users should pay special attention to the fact that CVA does not automatically support the estimation of interaction effects. To reduce the likelihood that interaction effects may be required, it is important to construct conditional pricing tables in a proportional manner. Even if using "proportional" lookup tables, CVA may still fail to capture interaction effects important to modeling buyer behavior. Conditional pricing increases the likelihood that significant interaction effects may occur in the data. CBC (with its similar conditional pricing capability) may be the better choice in such situations, because it is designed to detect and model interaction effects. Please also see the section entitled Modeling Interaction Effects with CVA for more information regarding these issues. In pricing research, it is sometimes very useful if prices for product concepts are made to depend on other attribute levels (such as brands). The first versions of CVA could handle this in a limited way by using prohibitions. For example, one could prohibit high prices from being shown for some products, and prohibit low prices from being shown with others. Such prohibitions, however, can lead to very inefficient designs. There are some work-arounds for dealing with situations where prohibitions with price seem to be needed. For example, if package size is an attribute, prices for the 64 oz package would be much higher than prices for the 16 oz size. One approach is to express price as a percentage above or below the "normal price" for that package size, such as "10% above the average price." Another is to show unit prices, rather than prices per package. For example, with detergent we might study 16, 32, and 64 ounce packages, but present prices in terms of "cents per ounce." These are ways of making a single set of price levels work for all package sizes without specifying any prohibitions, but it would be preferable to display actual prices appropriate for each package size. The conditional pricing option lets you create a look-up table to determine the prices to show for specific combinations of attributes. Drawing upon the example above, assume we have three attributes as follows to describe detergents: BRAND Brand A Brand B Brand C PACKAGE 16 oz. package 32 oz. package 64 oz. package PRICE "Low price" "Medium price" "High price" The text for the price levels above is not shown during the interview, and serves as a place-holder only. The price levels displayed during the interview vary according to the package sizes as follows:

CVA Help Conditional Pricing Lookup Table Low Price 16 oz. package 32 oz. package 64 oz. package $1.49 $2.79 $4.89 Medium Price $2.09 $3.99 $6.99 High Price $2.69 $5.19 $9.09

701

To construct this lookup table, we started with average prices for the three package sizes (middle column). To determine the prices in the "Low Price" column, we decreased the average price by 30% (and rounded to the nearest 9 cents). The "High Price" was calculated by increasing the average price by 30% (and again rounding to the nearest 9 cents). The example above illustrates how to make price dependent on a single variable (package size). CVA lets you make price conditional on up to 3 attributes. You will need to deal with an additional level of complexity when analyzing the results of studies that involved conditional pricing. In the example above, even though nine unique prices were shown to respondents, CVA's analysis programs by default still regard these as just three levels of price: Low, Medium and High. However, we interpret the results keeping in mind that larger package sizes were displayed with higher prices on average. We interpret the part-worth of the 32 oz. package taking into account that it was shown at an average price roughly $1.90 cents higher than the 16 oz. package. The main effect contains information not only about how desirable one package size is versus another, but also about the relative price levels at which they are offered. Therefore, it is perfectly legitimate if the part-worth utility for the 32 oz. package is lower than the 16 oz. package. This would indicate that respondents on average did not feel the larger size was worth the extra $1.90. The conditional pricing table must also be taken into account when using the market simulator. Unless the conditional pricing table is explicitly used in the market simulator, to simulate a 32 oz. package at $3.99, level 2 for price is specified, etc. However, with the most recent releases of SMRT, when you prepare the data for analysis in SMRT (the utility file and .ATT file), the conditional pricing table is also exported for use in the SMRT market simulator. The conditional prices are automatically read from a special footer section in the .ATT file into the Assign Level Values area of the SMRT market simulator (please double-check them to verify that the information was interpreted correctly, as we parse text strings to extract the price values). Once the conditional prices are within the SMRT market simulator, you simply specify the actual conditional price during simulations. For example, to simulate a 32 oz. package at $3.99, instead of specifying level 2, you specify a level value of 3.99. Users should note that the use of Randomized First Choice (or the historical Share of Preference with Correction for Product Similarity) can be problematic with conditional pricing. These methods assume that products sharing the same attribute levels are identical on those levels and require some downward correction in share. But, with conditional pricing tables, even though two brands may share the same "level" for price, the conditional pricing table may have actually resulted in unique prices for brands within the questionnaire. Therefore, RFC may implement a correction for product similarity that isn't actually warranted, assuming that the brands' prices were identical when in fact they were not. To avoid this outcome, we suggest turning off the correction for product similarity with respect to price within the RFC Method Settings dialog. In our previous example, we used (near) constant deviations from average prices to build a proportional conditional table. Some researchers choose to use constant absolute price differences between levels of the conditional attribute(s) instead of proportional changes. In either case, the tables have symmetric properties, which are desirable from an analysis standpoint. Proportional or symmetric conditional pricing tables make it more likely that main effects utilities will fit the data well. Since CVA can only use main effects models, this is a very desirable outcome. When using conditional pricing (especially with asymmetric price tables), specification of interactions may be

702

SSI Web v7 necessary to fit the data properly, and CVA only can manage this in a limited manner (see Modeling Interaction Effects with CVA).

Specifying Conditional Pricing

Before you can specify conditional prices, you first must have defined your list of attributes and levels. Instead of specifying the text for actual prices for your price levels, you should have used placeholder text such as "low price," "medium price," and "high price." When the questionnaire is administered, the placeholder text you assigned is overwritten by the text string you specify in the conditional pricing table. When you create a conditional price lookup table in CVA, you must specify the number of attributes that participate in the relationship (there should be at least two). If price is dependent on one other attribute (such as package size), two attributes participate in the relationship: package size and price. Click Compose | Conjoint Settings | Conditional Relationships (or click the Conditional Relationships button from the CVA Settings dialog). A dialog appears in which you specify attributes involved in the relationship. First, select your price attribute. The level text for this attribute is overwritten by the prices that you specify in the conditional pricing table. Then, select the attribute(s) upon which price is dependent. The conditional pricing table is displayed, with as many rows in the table as required to completely define the prices for all combinations of price and the dependent attribute(s). You can cut and paste conditional pricing information directly from a spreadsheet program into the conditional pricing grid. There can be a maximum of one conditional pricing table per study. However, some users have discovered that they can use conditional display tables just as they would use conditional pricing tables. In that case, more than one conditional pricing variable may be included. The only difference is that the conditional prices specified as conditional display variables are not automatically carried forward to the SMRT market simulator. In that case, you may need to use relative indices (such as 0.8, 1.0, 1.2, etc.) when defining products in the market simulator, rather than having SMRT automatically use the conditional pricing table within the simulator.

CVA Help

703

6.4.3

Modeling Interaction Effects with CVA

This is an advanced area. We assume the reader understands experimental design principles, is familiar with dummy-coding for part-worth attributes, is familiar with tuning simulators via the Exponent to best fit holdout data, and is able to do some additional data processing.

Background
As a traditional full-profile conjoint approach, CVA is a "main effects only" model, and assumes there are no interactions among attributes. Many conjoint practitioners agree that one must remain alert for the possibility of interactions, but that it is usually possible to choose attributes so that interactions will not present severe problems. Like other conjoint methods, CVA can deal with interactions in a limited way by defining composite variables. For example, we could deal with an interaction between car color and body style by cross-classifying the levels: Red Convertible Black Convertible Red Sedan Black Sedan However, if the attributes in question have many levels, or if an attribute (such as price, for example) is suspected of having interactions with many others, then composite attributes will not be enough. In that case too many parameters must be estimated to permit analysis with a purely individual-level model, and the most common solution is to evaluate interactions by pooling data from many respondents. For example, CBC (Choice-Based Conjoint) is considered a stronger approach for estimating interaction terms. It can automatically estimate interaction effects between all attributes, not necessarily just those that the analyst identifies prior to designing the study. CVA can accommodate up to 15 levels for each attribute. Therefore, you can use the composite attribute approach to model interactions between two attributes having as many as 5 and 3 levels, respectively (such that the product of the levels between the attributes does not exceed 15).

Designing CVA Studies for Limited Interactions

In the previous section of this documentation, we introduced the advanced design strategy of conditional pricing tables. We warned that these should be built with near-proportional pricing across the price tiers. We also warned that specifying conditional pricing tables might increase the likelihood that interactions would be needed to properly model the data. The following instructions provide a framework for advanced analysts to develop designs that formally support estimation of limited interactions and also examine whether those interaction terms are useful for a given data set. Rather than use the Conditional Pricing facility in CVA (which assumes main effects), this example also assumes customization of price ranges for each brand. For this example, we will assume a conjoint study where two of the attributes are brand (4 levels) and price (3 levels). (Note that the steps below may be used for any two attributes, whether involving price or not.) 1. When designing the study, create a single composite attribute reflecting all combinations of brand and price (4 x 3 = 12 levels). This will increase the number of parameters that need to be estimated relative to the main effects plan. With main effects, there are (4-1) + (3-1) = 5 parameters to fit to account for the main effects of brand and price. With the composite factor accounting for all combinations of brand and price, there are 12 - 1 = 11 parameters to fit. Thus, the interaction design increases the number of parameters to estimate by 11 - 5 = 6. Given the standard rule-of-thumb for designing CVA questionnaires (field twice as many questions as parameters to estimate), this will increase the number of cards that each respondent should evaluate by 2 x 6 = 12 cards. Of course, with CVA/HB estimation, you may do just as well with fewer cards than this recommendation.

704

SSI Web v7 2. Add CBC-looking fixed holdout questions at the end of the questionnaire (as Select-type or Grid questions). Three or four scenarios, each having 4 concepts (one for each brand) should be sufficient. Don't include a "None" option. These holdouts are useful not only for tuning the CVA part-worths to have the appropriate scale factor for predicting choice probabilities, but for investigating whether interaction effects improve the accuracy of the model. 3. Estimate the part-worth utilities (preferably with CVA/HB). 4. Plot the mean part-worths for the brand x price composite factor (pseudo demand curves). If reversals are present, you may consider re-running the estimation with constraints in place. Examine the resulting curves for face validity. 5. Specify the holdout scenarios each as a separate simulation scenario within the market simulator. Tune the Exponent setting (not independently for each simulation scenario, but using the same scale factor across all simulation scenarios) so that the simulated shares most closely fit the actual observed choices for the holdouts (Mean Absolute Error, or Mean Squared Error). 6. Repeat the analysis using main effects only. To do this, save a copy of the study within SSI Web and work from this duplicate study. Export the original design to a .csv file. Modify the design file to separate the brand x price composite factor into two attributes. Recode the level numbers accordingly to reflect main effects. Modify the attribute list to separate brand and price as attributes, to match your modified design. Import the recoded design. 7. Import the respondent data as if the study were conducted via paper-and-pencil. 8. Repeat steps 3 through 5 with the main effects model. Compare the plot of utilities and also the fit to holdouts. If interaction effects have greater face validity (visual plot) and noticeably stronger fit to holdouts, then this is evidence that interaction effects are useful. If the two models appear equally useful, we suggest using the more parsimonious model (main effects).

CVA Help

705

6.4.4

Sparse Designs and Large Samples

With typical CVA studies, one assumes OLS estimation at the individual level and therefore must ask each respondent to complete as many questions as parameters to be estimated, equal to (N-n+1), where N is the total number of levels in the study and n is the total number of attributes. However, with HB estimation, it is possible to achieve better results than OLS with the same number of (or even fewer) questions. Because HB borrows information across the sample, it stabilizes parameters (partworth utility estimates) even when the data are very sparse at the individual level (perhaps involving fewer questions than parameters to estimate). Sometimes, researchers have the luxury of very large sample sizes (such as 1000+), and may willing to sacrifice precision at the individual level to dramatically shorten the length of the survey. For example, if a researcher was comfortable using a standard CVA questionnaire with 18 choice tasks and 500 respondents, it would seem reasonable that the researcher could obtain quite similar overall results by using 1000 respondents where each respondent answered 9 questions, or 1500 respondents where each respondent answered 6 tasks. In all three cases, there are 9,000 answers to conjoint tasks, and sampling error is reduced if increasing the sample sizes. And, HB has proven to do a creditable job with estimating useful part-worth utilities for conjoint studies with as few as 6 tasks per person. CVA allows you to randomly skip a subset of the conjoint questions for each respondent. Of course, if the number of tasks answered by the respondent is fewer than the number of parameters to be estimated, then OLS estimation cannot be used. We only recommend this approach for situations where you have very large sample sizes and where market simulation results are the key output, rather than individual-level classification. If you want to purposefully use sparse designs (within each individual), we suggest developing the questionnaire using standard rules (generate 2x to 3x as many tasks as parameters to estimate, and multiple design versions). Aggregate Estimation via OLS: The simplest method of analysis is to compute the mean ratings for conjoint questions across the population (or subgroups of the population). Then, use the paper-andpencil methodology to import those means from a text (.csv) file into CVA for OLS estimation. Rather than each record in the .csv file representing a unique person, format just a single record to represent the total population (or, for subgroup analysis, format a record for each subgroup to be analyzed). The rating for each card is the mean rating for the population (the paper-and-pencil import supports decimal places of precision). HB Estimation: HB estimation can be used even with sparse data. If the data are quite sparse at the individual level relative to the number of parameters to be estimated, we would recommend decreasing the Prior Variance assumption (and potentially increasing the strength of the prior assumptions by increasing the Degrees of Freedom) in the Advanced Settings so that much more "shrinkage" to the population mean occurs. Otherwise, individual respondents may display wild deviations from reasonable individual-level utilities. You may also need to significantly increase the number of initial and used iterations to obtain convergence.

Example: Assume each version of your questionnaire has 21 CVA questions. Further assume you wanted each respondent to complete a random 6 out of 21 tasks (you plan to use HB estimation). First, make sure each of your CVA questions (CVA1 to CVA21) is located on its own page. Randomize the pages, by clicking Randomize... | Pages, and by selecting your first and last CVA questions (CVA1 and CVA21) as the range. To randomly skip a subset of questions, add a new HTML/Filler question directly beneath CVA1 (on the same page as CVA1). Call that new question something like Skip1. Within Skip1, click the Skip Logic

706

SSI Web v7 tab, add a skip (Post Skip, with a destination question being the next question directly following the last CVA question), and specify the following skip logic:
Begin Unverified Perl if ( SHOWN("CVA1")+ SHOWN("CVA2")+ SHOWN("CVA3")+ SHOWN("CVA4")+ SHOWN("CVA5")+ SHOWN("CVA6")+ SHOWN("CVA7")+ SHOWN("CVA8")+ SHOWN("CVA9")+ SHOWN("CVA10")+ SHOWN("CVA11")+ SHOWN("CVA12")+ SHOWN("CVA13")+ SHOWN("CVA14")+ SHOWN("CVA15")+ SHOWN("CVA16")+ SHOWN("CVA17")+ SHOWN("CVA18")+ SHOWN("CVA19")+ SHOWN("CVA20")+ SHOWN("CVA21") >=6 ) { return 1; } else { return 0; } End Unverified Finally, copy the Skip1 question directly beneath CVA2, CVA3, etc. It will automatically be renamed Skip2, Skip3, etc. Test your survey to ensure it works properly.

CVA Help

707

6.5
6.5.1

Data Analysis
Estimating Part-Worth Utilities
Once you have downloaded your data (saving the STUDYNAMEdat.dat file within your study folder) or read your paper-and-pencil data into SSI Web, you are ready to estimate part-worth utilities. CVA employs different statistical methods for estimating the separate part-worths for the attribute levels in your study. A unique set of part-worths is estimated for each individual, and the utility run is saved to a .UTL file for importing into the SMRT Market Simulator.

Ordinary Least Squares and Monotone Regression

When you click the Analysis menu, you can select whether to estimate utilities via Ordinary Least Squares (OLS) or Monotone Regression . (If your system includes the CVA/HB module, you can additionally choose HB estimation.) OLS is the method of calculation traditionally used in most conjoint studies. However, OLS is not appropriate for conjoint data consisting of rank orders. For OLS to be appropriate, we must assume the data are "scaled at the interval level." By this, we mean that the data are scaled so that real differences in the things being measured are communicated by the arithmetic differences in their values. Fahrenheit temperature, for instance, has an interval scale. The difference between 70 and 80 degrees is exactly as large as the difference between 80 and 90 degrees. In the social sciences and in marketing research we are usually willing to assume that rating scale values possess this kind of scaling. However, it is usually not reasonable to make such an assumption about rank order data. Suppose a respondent were to rank 30 concept statements in terms of his likelihood of buying each concept. In the absence of other information, we would probably expect the concepts to have a normal distribution of buying likelihood. If so, then we would expect there to be larger "real" differences between concepts with extreme ranks (such as 1 versus 2, or 29 versus 30) than those in the center of the distribution (such as ranks 14 and 15). When the data are rank orders, it is more appropriate to use a method of calculation that does not assume that the data represent anything more than rank orders. That is the case with nonmetric methods, and in particular with the monotone regression method provided in CVA. There is also another reason why we have provided a nonmetric method of calculation within CVA: With such methods it is easier to constrain calculated utilities to conform to the researcher's expectations. Conjoint utilities are often observed to violate principles of common sense. For example, in pricing studies it sometimes turns out that respondents seem eager to pay higher prices rather than lower prices. This may accurately reflect some respondents' behavior; price is sometimes taken as an indicator of product quality, and respondents may suspect that a low price reflects poor quality. However, it is customary to explain to the respondent that "everything else is equal," and that the attributes are to be considered independently of one another. Under those conditions, if a high price level receives a higher utility then a low price level, we are likely to conclude that the respondent was simply confused. Rather than discard data for that respondent, it is often useful to provide additional information to the calculating program in the form of "constraints." For example, we may tell the calculating program that the utility for $1.00 must be no lower than the utility for $1.25. With nonmetric methods it is easy to enforce such constraints. Since these methods are iterative, all that is necessary is to insure that the successive estimates at each stage obey the specified

708

SSI Web v7 inequalities. With OLS it is much more difficult to enforce such constraints. CVA provides that capability in a limited way: after the least squares solution is computed, then it is adjusted by "tying" values that violate specified inequalities. However, this is an inelegant way of solving the problem. When the data contain many violations of common sense relationships, then the nonmetric method provides a better way of enforcing desired constraints. However, more recent research has shown that HB estimation provides yet an even better way for estimating utilities that are less prone to reversals and for additionally enforcing utility constraints.

Additional Information
We have not provided details regarding utility estimation in this section. Please see How CVA Calculates Utilities for more information about OLS and Monotone Regression. Also see an introductory paper available at www.sawtoothsoftware.com in the Technical Papers library entitled "Analysis of Traditional Conjoint Using Excel: An Introductory Example." For information regarding HB estimation, please see two papers also available in the on-line Technical papers library: "The CVA/HB Technical Paper" and "Monotonicity Constraints in Choice-Based Conjoint with Hierarchical Bayes."

Settings for Utility Estimation

From Analysis | Calculate CVA Utilities using <OLS/Monotone>... if you click the Settings button, you can review or change the settings for the estimation. The controls depend slightly whether you are using Ordinary Least Squares or Monotone Regression Estimation: OLS Settings Monotone Regression Settings Additional Utility Constraints: When you click this button, a new dialog opens in which you can specify Utility Constraints or override previously defined a priori orders for attributes. Utility constraints are useful if you have attributes that have a specific order such as price. It usually doesn't make sense that a higher price would be preferred to a lower price. However, utilities (especially when calculated at the individual level) sometimes include such "reversals." The researcher can smooth some of the noise out of conjoint data by providing additional information about the order of attribute levels. However, researchers often find that constraining utilities will slightly hurt share prediction accuracy. The benefit of adding constraints is that they can improve individual-level classification and prediction. Important Note: If you specified a priori order when you specified attributes and levels, CVA will automatically constrain utilities of these ordered attributes to have no reversals. If you do not wish to apply constraints, you should override these a priori specifications by editing the Constraints. When you are satisfied with your settings, close the dialog and click the Prepare for CVA OLS Computation... (or Prepare for CVA Monotone Regression Computation...) button. CVA automatically calculates part-worths for each respondent and provides a summary report on the screen. Respondents with no variation in the dependent variable or too many missing answers are not included in the utility run. The utility run is saved to a .UTL file for use in the SMRT Market Simulator.

Moving the Data into SMRT

SMRT stands for "Sawtooth Software Market Research Tools" and contains the market simulation tool used to analyze data resulting from any of Sawtooth Software's conjoint analysis systems. We'll now

CVA Help describe how you move data from your CVA project within the SSI Web Platform into SMRT.

709

When you calculated part-worth utilities using Analysis | Calculate CVA Utilities Using Ordinary Least Squares... (or via Monotone regression or HB) this process creates a text-only format file containing case IDs and part-worth estimates called STUDYNAME OLS.utl. To move the CVA part-worth utility data into SMRT for analysis: 1. Start the SMRT software by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 2. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname, making sure to specify that this is a CVA study type). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 3. Import the STUDYNAME OLS.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CVA utilities (selecting CVA (Traditional Conjoint) Utilities (*.utl) as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations. Click close to return to the main menu. Now that you have imported the utilities, the Market Simulator menu item becomes active. From the main menu, click Analysis | Market Simulator. You will note that there is a list of available Utility Runs (your recently computed run is available), and a list of Simulation Scenarios (currently empty). Click Compute!, and a report of average utilities and importances for your data set is displayed in the report window (these data should match the similar report that was earlier displayed within SSI Web, directly following utility estimation).

710

SSI Web v7

6.5.2

CVA Ordinary Least Squares Settings

This dialog controls the settings used for estimating part-worths using OLS (Ordinary Least Squares) Regression. OLS is recommended for CVA questionnaires involving ratings data, where respondents rated individual conjoint cards or compared two concepts at a time using a sliding rating scale. Please see the section entitled How CVA Calculates Utilities for more information about OLS Regression. Minimum Scale Value: If you chose either Logit or Zero-centered recoding methods, you must specify the minimum scale value. If you used a 0 to 100 purchase likelihood scale, the minimum is 0. If you used a 1 to 9 scale with a pairwise questionnaire and used 0 as the missing value, the Minimum Scale Value is 1. Maximum Scale Value: If you chose either Logit or Zero-centered recoding, you must specify the maximum value. Scale Direction: We assume that most users field ratings-based CVA questionnaires. For singleconcept designs, we have made "Highest Number Best" the default, meaning that higher numbers indicate higher preference. However, for ranking tasks, lower numbers frequently imply higher utility. In that case, "Lowest Number Best" should be chosen. For pairwise comparison designs, "Highest Number on Right" should be chosen if the values run from lower numbers on the left to higher numbers on the right. If you specify this field incorrectly, the part-worths will appear opposite to how you expect them (i.e. higher prices will be preferred, etc.). Recode Method: Sometimes it is desirable or even necessary to recode the CVA responses. CVA provides three options: None, Logit (default) and Zero-centered. Each of these is described below. If you only plan to use the First Choice model in the CVA market simulator, whether you center or not does not impact your results. If you plan to use the Purchase Likelihood model, you must use single concept presentation and specify a Logit recoding to ensure meaningful results. If you plan to use the Share of Preference option or Randomized First Choice, it is probably helpful to specify a zero-centered or a logit rescaling. Scaling of part-worths can have a significant impact upon Share of Preference results. The scaling can be adjusted later in the Market Simulator using the Exponent setting. None. This setting uses the responses in the data file as-is. If you only plan to use the First Choice model in the CVA market simulator, you may choose this option. Logit. It is common in "single concept at a time" questionnaires for respondents to rate concepts on "Level of interest" or "likelihood of buying" scales. The scale could be three digit (from 0 to 100) or single digit (from 1 to 5). In those cases it is standard procedure to calculate part-worths using the actual ratings as dependent variables. Consider a 0 to 10 purchase likelihood scale. If we calculate part-worths using the original scale values, we will be assuming that the difference between ratings of say, 5 and 6, is as meaningful as the difference between ratings of 9 and 10. Yet it would seem reasonable to believe that the difference between 9 and 10 is a larger subjective difference for the respondent than the difference between 5 and 6. When data are probabilities, this problem is often handled by transforming the data to a new scale that shrinks the differences in the center of the scale and stretches differences at the extremes of the scale. This is called the "logit" transformation, given by the formula: Recoded value = ln[p/(1-p)] where p is equal to the probability of purchase. When using the Purchase Likelihood model, the Market Simulator requires that utilities be scaled so that antilogs of their sums are equal to expected likelihoods of purchase. Thus, the data should be

CVA Help recoded into logits. Here is an example of a logit transformation for a 0 to 10 scale:
Scale Value 0 1 2 3 4 5 6 7 8 9 10 Interpreted Probability .0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Logit Value -3.000 -2.197 -1.386 -0.847 -0.405 0.000 0.405 0.847 1.386 2.197 3.000

711

Notice that in the middle of the scale the difference between adjacent logit values is only 0.405, but the difference between logit values for ratings of 8 and 9 is 0.811. Logit values are undefined for probabilities of 0 and 1, but approach plus and minus infinity. To keep things under control, we arbitrarily limit the logit values to the range of plus or minus 3. The example above shows a simple way for converting numeric responses on a 1 to 9 scale first to probabilities, and then to logits. The method CVA actually employs for the logit recode is only slightly different. Because the logit transform is very sensitive at high and low probabilities, we first map numeric responses to probabilities using the following formula: p = (y-min+1)/(max-min+2) where: p=probability y=answer min=minimum scale value max=maximum scale value After calculating the probability p using the above formula, we again use the formula: Recoded value = ln[p/(1-p)] Researchers who wish to use other recoding schemes can do so by using the paper-and-pencil method of importing respondent answers and directly modifying the values in the respondent answer file. Logit recodes can also be useful in "pairwise" questionnaires, where the rating scale may run from 1 to 9 with 5 meaning "no preference." Zero-centered. Many researchers find that zero-centering the response scale yields more interpretable utilities than when the response scale features only positive values. Whether you adopt this option or use no recoding method has no impact in the reported average utilities from the simulator or the competitive simulation models. But, if you plan to export and use the raw utilities in another way, you may want the resulting utilities to be based on a zero-centered response scale We suggest zero-centered or logit scaling for paired-comparison questionnaires. If we asked pairedcomparison questions using CVA's default 1 to 9 point scale, the recode values would be: -4 -3 -2 -1 0 1 2 3 4. Notice that these recoded values fall roughly in the same zero-centered range as the logit transformation for purchase likelihood models, but that the zero-centered rescaling assumes equal increments instead of "stretching" the scale at the ends.

712

SSI Web v7 Tasks to include: This lets you specify which tasks (conjoint questions) should be included when estimating part-worths. By default, all tasks are included. Generally, you should include all conjoint tasks in estimation. However, if you have placed holdout tasks within the survey (by exporting the design, modifying it to include additional holdout tasks, and re-importing the design), you may decide to omit those holdouts from the part-worth estimation. Please see the section entitled Importing Designs into CVA for more information about including holdout cards in your CVA studies. Constraints: The Edit... button lets you specify that certain levels within certain attributes have known a priori preference order and that the part-worths should be constrained accordingly. All rankorder relationships you originally specified when entering your list of attributes and levels are carried forward to the Additional Utility Constraints dialog. But, you can modify those selections.

CVA Help

713

6.5.3

CVA Monotone Regression Settings

This dialog controls the settings used for estimating part-worths using Monotone Regression. Monotone Regression is recommended for card-sort designs involving rank-order data, where respondents sorted individual conjoint cards from best to worst. Please see the section entitled How CVA Calculates Utilities for more information about Monotone Regression. Minimum Scale Value: Specify the lowest value used in the rank-order scale. This is usually 1 when using card-sort data. If using pairwise comparison presentation, it is the lowest value used in the response scale. Maximum Scale Value: Specify the highest value used in the rank-order scale. This is usually equal to the number of cards (conjoint questions) in your study. For example, if you have 18 cards in your study, then specify 18. If using pairwise comparison presentation, it is the highest value used in the response scale. Scale Direction: Single-Concept (Card-Sort) Designs: If using card-sort data, there are two choices: "Lowest Number Best" and "Highest Number Best." With rank-order data, if a "1" in your data set indicates the best card, you should choose "Lowest Number Best." If you used CVA to import rank-order data from a text file and specified that the values were card numbers, CVA internally coded the values so that a "1" is associated with the best card, a "2" with the nextbest card, etc. If higher numbers indicate greater preference, choose "Highest Number Best." Pairwise Comparison Designs: It is unusual to use Monotone Regression in pairwise comparison CVA designs, but if you do wish to try it, you should specify the correct scale direction. The selections are: "Highest Number on Right" and "Lowest Number on Right." Monotone regression only retains the ordinal information about which concepts are preferred to others. The magnitudes of the differences are ignored. For this reason, part-worth estimation from pairwise rating questionnaires using monotone regression may result in noisier estimates than when preserving the additional information of the ratings scale under OLS. Tasks to Include: This lets you specify which tasks (conjoint questions) should be included when estimating partworths. By default, all tasks are included. Generally, you should include all conjoint tasks in estimation. However, if you have placed holdout tasks within the survey (by exporting the design, modifying it to include additional holdout tasks, and re-importing the design), you may decide to omit those holdouts from the part-worth estimation. Please see the section entitled Importing Designs into CVA for more information about including holdout cards in your CVA studies. Constraints: The Edit... button lets you specify that certain levels within certain attributes have known a priori preference order and that the part-worths should be constrained accordingly. All rankorder relationships you originally specified when entering your list of attributes and levels are carried forward to the Additional Utility Constraints dialog. But, you can modify those selections.

714

SSI Web v7

6.6
6.6.1

Technical Details
The CVA Designer
The CVA designer uses a computer search technique to find D-efficient designs, given a certain conjoint design, any prohibitions (for level by level combinations or due to utility balance constraints), and a requested number of conjoint profiles. D-efficiency is described in the article by Kuhfeld, Tobias, and Garratt (1994), "Efficient Experimental Design with Marketing Research Applications," Journal of Marketing Research, 31 (November), 545557. Paraphrasing the article, D-efficiency is a function of the determinant of the X'X matrix, given by the formula:

where ND p X

= = =

number of tasks number of attributes the design matrix using orthogonal coding

If a design is orthogonal and balanced (each level within an attribute shown an equal number of times), then it has optimum efficiency. The D-efficiency measures the goodness of the design relative to the hypothetical orthogonal design. A perfect design will be both orthogonal and balanced and will result in an efficiency of 1.0. However, an orthogonal design is not always possible, given the number of attribute levels and requested number of tasks. A final efficiency of less than 1.0 may still indicate a satisfactory design.

User-Specified Designer Controls

CVA generates an initial pool of conjoint profiles equal to ten times the requested number. It repeats the search process within that candidate pool equal to ten times as many questionnaire versions that are requested. Some users may wish to over-ride these limits to either speed up the process, or to let CVA spend an even greater amount of time searching for the most efficient design. You can override the defaults by clicking the Show Advanced Settings button and modifying the fields that appear. For larger CVA designs, a task pool multiplier of 100 or more often will lead to finding more efficient designs faster.

CVA Help

715

6.6.2

How CVA Calculates Utilities

Ordinary Least Squares (OLS)
This section assumes the reader has basic knowledge of linear regression. However, understanding of the technical details in this section is not essential for using CVA. The OLS calculator does ordinary least squares linear regression using a "dummy variable" approach. A vector of independent variables for each observation (conjoint question) is built using information from the experimental design. The vector has elements of 1, 0, or -1, depending on whether respective attribute levels appear in that question, and whether they appear on the left-hand (-1) or right-hand side (+1) of the pairwise questions. The dependent variable for each observation is obtained by applying the indicated recoding transformation to the corresponding data value. If the data value has the code reserved for missing data, then that observation (conjoint question) is not included in the calculation. When using regression to estimate conjoint utilities, it is customary to delete one level of each attribute from the computation. Otherwise, there is a linear dependence among the variables describing levels of each attribute, which leads to indeterminacy in the computation. Omitting one level of each attribute from the computation is equivalent to setting its utility at zero, with the other levels measured as contrasts with respect to zero. The OLS calculator omits the first level of each attribute from the regression computation. Thus, if there are k attributes with a total of n levels, the regression is done with only n-k independent variables. The indeterminacy could also be handled by adding side conditions, such as requiring that the utilities for each attribute sum to some constant. However, our approach has the advantage of greater computational speed. An intercept term is also computed by CVA, but it is not reported separately. Since the utilities will most often be used by adding up sums consisting of one value from each attribute, the intercept has been divided by the number of attributes and that fraction has been added to every utility value. Thus the first level for each attribute, which would otherwise be zero, will be equal to the intercept divided by the number of attributes. The "r squared" value for each respondent does not contain any correction for degrees of freedom. If the number of observations is equal to the number of parameters being estimated (levels - attributes +1), then the r squared value will be unity. If the design is deficient -- containing either too few observations to permit estimation or insufficient information for a particular attribute level -- then a message to that effect will appear on the screen and utilities will not be estimated for that respondent. If there are degrees of freedom available for error, then descriptive data will be written to a log file with information about the precision of estimation. A statistic ("rms cor") is provided for each respondent, which describes the amount of correlation among the independent variables. It is the "root mean square" of off-diagonal elements of the correlation matrix for the n-k independent variables. Subtle relationships among variables can easily lead to faulty designs that would not be detected, and therefore we caution against paying much attention to this statistic. We include it only because our users may be accustomed to similar statistics in other software packages. In an orthogonal design with no missing data, this value will be either zero or a small positive number. (Orthogonal designs have correlations within attributes.) Orthogonal designs are sometimes altered to eliminate "nonsense" questions, and this compromises orthogonality. Also, some design procedures (CVA's questionnaire design module, for example) produce well-balanced but not perfectly orthogonal designs.

716

SSI Web v7 For each individual, standard errors of utility estimates are provided (within a log file), except for the first level of each attribute, which is assumed to have utility of zero. These standard errors may also be regarded as standard errors of differences between each level and the first level of that same attribute. These standard errors can be of diagnostic value. Attribute levels with large standard errors should be given more attention in questionnaire design. They may appear in too few questions, or they may occur in patterns that compromise the level of independence necessary for good estimation. For more information on utility estimation, see Avoiding Linear Dependency.

Monotone (Nonmetric) Regression

This option for calculating utilities uses a method similar to that described by Richard M. Johnson in "A Simple Method of Pairwise Monotone Regression", Psychometrika, 1975, pp 163-168. The method is iterative, finding successive solutions for utility values that fit the data increasingly well. An initial solution is developed, either randomly or using information in the experimental design. Two measures of goodness of fit are reported: theta and tau. Tau Suppose the conjoint questionnaire presented concepts one at a time and asked for a rank order of preference. Although there would have been many concepts in the questionnaire, consider just four of them, concepts P, Q, R, and S. Suppose the respondent ranked these concepts 7, 9, 13, and 17, respectively, and at some intermediate stage in the computation, utilities for these concepts are estimated as follows: Concept P Q R S Estimated Utility 4.5 5.6 1.2 -2.3 Preference Rank 7 9 13 17

We want to measure "how close" the utilities are to the rank orders of preference. One way we could measure would be to consider all of the possible pairs of concepts, and to ask for each pair whether the member with the more favorable rank also has the higher utility. Since these are rank orders of preference, smaller ranks indicate preference, so we know that: Preference P P P Q Q R is is is is is is preferred preferred preferred preferred preferred preferred to to to to to to Q R S R S S Utility Squared Difference Difference -1.1 1.21 3.3 10.89 6.8 46.24 4.4 19.36 7.9 62.41 3.5 12.25 ------Total 152.36

Of the six pairs, five have utility differences with the correct signs (the preferred product has the higher utility), and one pair has a utility difference with the wrong sign. Kendall's tau is a way of expressing the amount of agreement between the preferences and the estimated utilities. It is obtained by subtracting the number of "wrong" pairs from the number of "right" pairs, and then dividing this difference by the total number of pairs. In this case,

CVA Help tau = (5 - 1) / 6 = .667

717

A tau value of 1.000 would indicate perfect agreement in a rank order sense. A tau of 0 would indicate complete lack of correspondence, and a tau of -1.000 would indicate a perfect reverse relationship. Tau is a convenient way to express the amount of agreement between a set of rank orders and other numbers, such as utilities for concepts. However, it is not very useful as a measure on which to base an optimization algorithm. As a solution is modified to fit increasingly well, its tau value will remain constant and then suddenly jump to a higher value. Some other measure is required that is a continuous function of the utility values. Theta For this purpose we use the statistic "theta." Theta is obtained from the squared utility differences in the last column of the table above. We sum the squares of those utility differences that are in the "wrong order," divide by the total of all the squared utility differences, and then take the square root of the quotient. Since there is only one difference in the wrong direction, theta = square root(1.21/152.36) = .089 Theta can be regarded as the percentage of information in the utility differences that is incorrect, given the data. The best possible value of theta is zero, and the worst possible value is 1.000. Now that we have defined theta, we can describe the nature of the computation. The process is iterative. It starts with random values as estimates of the partworths. In each iteration a direction of change (a gradient vector) is found which is most likely to yield an improvement in the partworths. A number of small changes are made in that direction, which continue as long as theta improves. Each iteration has these steps: 1. Obtain the value of theta for the current estimates of partworths and a direction (gradient) in which the solution should be modified to decrease theta most rapidly. 2. Try a small change of the partworths in the indicated direction, which is done by subtracting the gradient vector from the partworth vector and renormalizing the partworth estimates so as to have a sum of zero within each attribute and a total sum of squares equal to unity. Each successive estimate of utilities is constrained as indicated by the a priori settings or additional utility constraints. 3. Re-evaluate theta. If theta is smaller than before, the step was successful, so we accept the improved estimates and try to obtain further improvement using the same procedure again, by returning to step (2). If theta is larger than before, we have gone too far, so we revert to the previous estimate of partworths and begin a new iteration by returning to step (1). If any iteration fails to improve theta from the previous iteration, or if theta becomes as small as 1e-10, the algorithm terminates. A maximum of 50 iterations are permitted, and within any iteration a maximum of 50 attempts at improvement are permitted. In theory, the iterations could continue almost indefinitely with a long series of very small improvements in theta. For this reason it is useful to place a limit on the number of iterations. To avoid the possibility of stumbling into a bad solution due to a poor starting point, the process is repeated 5 separate times from different starting points. For each respondent, the weighted average of the five resulting vectors of part-worths is computed (weighted by Tau, where any negative Tau is set to an arbitrarily small positive number). A weighted Tau is also reported with this final estimate of part-worth utilities.

718

SSI Web v7

How CVA Utilities Are Scaled

Monotone regression CVA's monotone regression utility calculator scales utilities in a way that is easy to describe and to understand. For each respondent, the values for each attribute have a mean of zero, and their sum of squares across all attributes is unity. Here is an example, assuming two attributes, one with 3 levels and one with 2 levels: utility Attribute One Level 1 Level 2 Level 3 .50 .10 -.60 --0.00 .44 -.44 ---0.00 square .25 .01 .36 --.19 .19 ----1.00

Attribute Two Level 1 Level 2

OLS regression CVA's OLS utility calculator scales utilities in a way that depends upon the data, and upon the researcher's use of the recode capabilities. The calculation has these steps: 1. If automatic recoding was specified, then the data are automatically recoded. If no recode was specified, the values in the data file are used without modification. 2. An array of "dummy" variables is constructed with a row for each conjoint question and a column for each attribute level. Each cell of this array has a value of 1, 0, or -1, depending on the experimental design. For single-concept presentation, the values are either 1 or 0. If the level appears in the concept it is coded as 1, and if absent it is coded as 0. For pairwise presentation, if a level appears in the left-hand concept it is coded as -1, or 1 if in the righthand concept. If an attribute level does not appear in either profile, then the corresponding array element is 0. 3. The first level (column) for each attribute is omitted temporarily from the design, which avoids technical problems of indeterminacy in the solution. (See Avoiding Linear Dependency) 4. OLS regression is used to predict the transformed data values from the surviving columns of the array (variables). A regression coefficient is computed for each variable, as well as a single intercept. The regression coefficients for the omitted variables are assumed to be zero. 5. The intercept is divided by the number of attributes, and the quotient is added to every regression coefficient, including those previously assumed to be zero. The resulting values are reported as utilities for the attribute levels. (The intercept is handled in this way to make it easy to calculate total utilities for products during simulations. Since each product to be simulated will have exactly one level from each attribute, the simulator will be able to include the intercept automatically just by adding the utilities of its attribute levels.) As can be seen from this explanation, with the OLS calculator the scaling of the utilities is completely under the control of the researcher. Other things being equal, if the data are collected on a 100-point scale, the utilities will be about ten times the magnitude as they would be if the data were collected on a 10-point scale.

CVA Help

719

6.6.3

Avoiding Linear Dependency

CVA's OLS regression utility calculator deletes one level of each attribute from the utility computation. That is done to avoid linear dependence among attribute levels, which would result in indeterminacy in the solution. The purpose of this section is to explain why that precaution is required, when it is appropriate, and when it is useful but not essential. We consider two popular types of conjoint designs: Full-profile single concept Paired comparison In each case we will consider a design matrix with a row for each question and a column for each attribute level. Suppose that the total number of attributes is n and the total number of levels (columns) is N.

Full-profile Single Concept

In these designs, each question presents a single concept. Each concept has a specified level of each attribute. Each row of the design matrix has 1's in the columns corresponding to the attribute levels in that concept, and other elements of 0 (zero). Since each row contains exactly n 1's and N -1 zeroes, the sum of each row is n. CVA computes an intercept term, which is equivalent to adding an imaginary N + first column to the design matrix in which every element is a 1. That column would be linearly dependent on the sum of the N existing columns, which would cause the solution to be indeterminate. However, the problem is greater than that. Consider just the subset of columns corresponding to levels of one attribute: Every row will have a single 1 and other elements of 0. Therefore, within that attribute the sum of every row is equal to 1. This problem generalized to every attribute. The most popular solution is to delete one level from each attribute, which is equivalent to assuming that that level has a utility of 0. If this is done, then the rows no longer have equal sums and the indeterminacy is resolved. CVA arbitrarily deletes the first level of each attribute.

Paired Comparisons
In these designs each question presents two concepts and the respondent is asked to express a preference. Suppose one concept is presented on the "left" and one on the "right" and that the response scale is arranged so that larger responses indicate greater preference for the concept on the right. Also, we assume that if a particular attribute appears in one of the concepts, it will also appear (probably with a different level) in the other concept. The design matrix has a row for each concept and N columns. Each element is 1 if that attribute level appears in the concept on the right, -1 if it appears in the concept on the left, or 0 if that attribute level does not appear in either concept. (If an attribute level should happen to appear in both concepts it should not have an effect on respondent preference, and is given a value of zero.) Consider just those columns corresponding to levels of one attribute. If an attribute appears in either concept, it will also be represented in the other concept. Therefore, each value of 1 has a corresponding value of -1, and the sum of values will be 0 for every row within the columns corresponding to each attribute. Thus paired comparison designs (even those that present concepts specified on only subsets of attributes) are subject to the same indeterminacy as full-profile single concept designs. CVA handles the problem of indeterminacy in exactly the same way as with fullprofile single concept designs, by deleting the first level of each attribute and assuming its utility to be

720

SSI Web v7 0.

CVA Help

721

6.7
6.7.1

The CVA/HB Module for HB Estimation

What is CVA/HB (hierarchical Bayes for CVA)?
Introduction
The CVA/HB Module is an add-on component within SSI Web that uses hierarchical Bayes (HB) to estimate part-worths for ratings-based full-profile conjoint analysis (CVA) studies. HB considers each individual to be a sample from a population of similar individuals, and "borrows" information from the population in estimating part-worths for each respondent. With HB, CVA users can often achieve equivalent results (relative to OLS) using fewer tasks per person and/or fewer total respondents. Precisely how much improvement HB estimation offers over the standard estimation techniques depends on the project. If using the CVA/HB module, it is possible to obtain good part-worth utility estimates even though respondents have evaluated a random subset of the questions within a questionnaire version (even fewer questions than parameters to be estimated). This approach can significantly reduce the number of questions each respondent is asked. However, we recommend you not do this unless sample sizes are quite robust. HB estimation takes considerably longer than OLS or Monotone regression. Computation time will usually vary from about 3 to 20 minutes for most CVA data sets. Although the CVA/HB module will produce results for very small sample sizes, we caution against using it in those instances. CVA/HB will produce results even with data from a single respondent, but with very small sample sizes it will have difficulty distinguishing between heterogeneity and error. How many respondents are required for robust HB estimation depends on the study design and the nature of the sample. We have seen HB perform well with samples as small as 80 respondents for ratingsbased conjoint studies. HB may perform consistently well with even smaller sample sizes, though we know of no series of studies to substantiate that claim. In any case, we suggest not using HB blindly. It is prudent to design holdout choices within your CVA questionnaires so that you can assess the performance of alternative part-worth estimation methods.

Background
The first traditional conjoint analysis applications in the early- to mid-1970s used non-metric estimation or OLS to derive part-worths. These techniques served the industry well over the first few decades of conjoint analysis practice. Even so, conjoint researchers have always faced a degrees of freedom problem. We usually find ourselves estimating many parameters (part-worths) at the individual level from relatively few observations (conjoint questions). It is often challenging to get respondents to complete many conjoint tasks, so researchers may sacrifice precision in the part-worth estimates by reducing the number of conjoint profiles. It is precisely in those cases that HB can be most useful. HB became available in about the mid 1990s to marketing researchers. HB significantly improves partworth estimates and produces robust results when there are very few or even no degrees of freedom. Several articles (see for example Lenk, et al. 1996 and Allenby, et al.1998) have shown that hierarchical Bayes can do a creditable job of estimating individual parameters even when there are more parameters than observations per individual. It is possible using CVA/HB to estimate useful part-worths for an individual even though that respondent has answered fewer tasks than parameters to estimate. This can occur if respondents quit the survey early. However, the researcher may choose this approach by design. The researcher might randomly assign respondents a subset of a larger CVA design, so that across all respondents each task has roughly equal representation.

722

SSI Web v7 The CVA/HB Module estimates a hierarchical random coefficients model using a Monte Carlo Markov Chain algorithm. In the material that follows we describe the hierarchical model and the Bayesian estimation process. It is not necessary to understand the statistics of HB estimation to use this module effectively. The defaults we have provided make it simple for researchers who may not understand the statistics behind HB to run the module with consistently good results. We at Sawtooth Software are not experts in Bayesian data analysis. In producing this software we have been helped by several sources listed in the References. We have benefited particularly from the materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing Association's Advanced Research Techniques Forum.

The Basic Idea behind HB

CVA/HB uses Bayes methods to estimate the parameters of a randomized coefficients regression model. In this section we provide a non-technical description of the underlying model and the algorithm used for estimation. The model underlying CVA/HB is called "hierarchical" because it has two levels. At the upper level, respondents are considered as members of a population of similar individuals. Their part-worths are assumed to have a multivariate normal distribution described by a vector of means and a matrix of variances and covariances. At the lower level, each individual's part-worths are assumed to be related to his ratings of the overall product profiles within the conjoint survey by a linear regression model. That is to say, when deciding on his preference for a product profile, he is assumed to consider the various attribute levels that compose that product, and add the value of each level to come up with an overall rating for the product concept. Discrepancies between actual and predicted ratings are assumed to be distributed normally and independently of one another. Suppose there are N individuals, each of whom has rated conjoint profiles on n attribute levels. If we were to do ordinary regression analysis separately for each respondent, we would be estimating N*n part-worths. With the hierarchical model we estimate those same N*n part-worths, and we further estimate n mean part-worths for the population as well as an n x n matrix of variances and covariances for the distribution of individuals' part-worths. Because the hierarchical model requires that we estimate a larger number of parameters, one might expect it would work less well than ordinary regression analysis. However, because each individual is assumed to be drawn from a population of similar individuals, information can be "borrowed" from other individuals in estimating parameters for each one, with the result that estimation is usually enhanced. For more information, please read the CVA/HB Technical Paper, available for download from our technical papers library at www.sawtoothsoftware.com.

CVA Help

723

6.7.2

Running CVA/HB
Once you have collected conjoint data for a CVA project, you are ready to estimate part-worths with HB. Click Analysis | Calculate CVA Utilities Using CVA/HB.... Click the Settings... button to review or change the default settings:

Estimation Parameters: Number of Iterations before Using Results: This describes how many "burn-in" iterations will be used, prior to assuming convergence. 10,000 iterations usually provides enough initial iterations so that the part-worth utilities "converge." Convergence is assessed typically by viewing the visual display of the history of part-worth estimates. If the lines in the display tend to be wobbling horizontally, with no noticeable trend, then this is evidence of convergence. If the parameters do not seem to converge, then you can increase the number of initial iterations. Number of Draws to be Used for Each Respondent: This describes how many "draws" or candidate betas will be used (averaged to form a point estimate) for each respondent. 10,000 draws tends to give quite good precision for each respondent. However, you can obtain slightly better precision by increasing the number of used draws.

724

SSI Web v7 Scale: The Scale Settings (Minimum Scale, Scale Direction, and Recode Method) are similar to those described for OLS estimation Advanced Estimation Parameters: Prior Degrees of Freedom: This value is the additional degrees of freedom for the prior covariance matrix (not including the # parameters to be estimated). The higher the value, the greater the influence of the prior variance and more data are needed to change that prior. The scaling for degrees of freedom is relative to the sample size. If you use 50 and you only have 100 subjects, then the prior will have a big impact on the results. If you have 1000 subjects, you will get about the same result if you use a prior of 5 or 50. As an example of an extreme case, with 100 respondents and a prior variance of 0.1 with prior degrees of freedom set to the number of parameters estimated plus 50, each respondent's resulting part-worths will vary relatively little from the population means. We urge users to be careful when setting the prior degrees of freedom, as large values (relative to sample size) can make the prior exert considerable influence on the results. Prior Variance: The default is 2 for the prior variance for each parameter, but users can modify this value. Increasing the prior variance tends to place more weight on fitting each individual's data, and places less emphasis on "borrowing" information from the population parameters. The resulting posterior estimates are relatively insensitive to the prior variance, except 1) when there is very little information available within the unit of analysis relative to the number of estimated parameters, and 2) the prior degrees of freedom for the covariance matrix (described above) is relatively large. Random Starting Seed: HB requires a random number generator, and therefore a starting seed is required. By default, the random seed is 1, but users can specify a specific seed to use (integers from 1 to 32000), so that results are repeatable. When using different random seeds, the posterior estimates will vary, but insignificantly, assuming convergence has been reached and many draws have been used. If you use a seed of "0", a seed is drawn based on the computer's clock, so a different seed will be chosen if you repeat the analysis.

Results
After the computation finishes, the new HB run is automatically saved to a STUDYNAME.hbu file within your study director. That file may be imported within SMRT for use in the market simulator. To move the CVA part-worth utility data into SMRT for analysis: 1. Start the SMRT software by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT. 2. Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname, making sure to specify that this is a CVA study type). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study. 3. Import the STUDYNAME.hbu file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import and following the prompts for importing CVA utilities (selecting HB Utilities (*.hbu, *.hbc) as the file type, and browsing to your .hbu file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

CVA Help

725

Click close to return to the main menu. Now that you have computed utilities, the Market Simulator menu item becomes active. From the main menu, click Analysis | Market Simulator.

726

SSI Web v7

6.7.3

Using Constraints in CVA/HB

Conjoint studies frequently include product attributes for which almost everyone would be expected to prefer one level to another. However, estimated part-worths sometimes turn out not to have those expected orders. This can be a problem, since part-worths with the wrong relationships (especially if observed at the summarized group level) are likely to yield nonsense results and can undermine users' confidence. CVA/HB provides the capability of enforcing constraints on orders of part-worths within attributes. The same constraints are applied for all respondents, so constraints should only be used for attributes that have unambiguous a priori preference orders, such as quality, speed, price, etc. Evidence to date suggests that constraints can be useful when the researcher is primarily interested in individual-level classification or the prediction of individual choices, as measured by hit rates for holdout choice tasks. However, constraints appear to be less useful, and indeed can be harmful, if the researcher is primarily interested in making aggregate predictions, such as predictions of shares of preference. Most research is principally concerned with the latter. Another concern is that constraints can bias the apparent importances of constrained attributes in market simulations, relative to unconstrained attributes. If you check the Use Constraints box on the CVA/HB Settings dialog, CVA/HB automatically constrains the part-worths if you have specified a priori orders (e.g. Best to Worst or Worst to Best) when you specified your attributes, or if you specify additional utility constraints by clicking Edit... and supplying additional constraints using the Custom Constraints dialog. CVA/HB employs a technique called Simultaneous Tying. In a paper available on the Sawtooth Software Web site (Johnson, 2000), the author explored different ways of enforcing constraints in the HB context. He found the method of simultaneous tying to perform best among the techniques investigated. Simultaneous tying features a change of variables between the "upper" and "lower" parts of the HB model. For the upper model, we assume that each individual has a vector of (unconstrained) partworths, with distribution: i ~ Normal(, D) where: i = unconstrained part-worths for the ith individual = means of the distribution of unconstrained part-worths D = variances and covariances of the distribution of unconstrained part-worths For the lower model, we assume each individual has a set of constrained part-worths, bi where bi is obtained by recursively tying each pair of elements of i that violate the specified order constraints. With this model, we consider two sets of part-worths for each respondent: unconstrained and constrained. The unconstrained part-worths are assumed to be distributed normally in the population, and are used in the upper model. However, the constrained part-worths are used in the lower model to evaluate likelihoods. We speak of "recursively tying" because, if there are several levels within an attribute, tying two values to satisfy one constraint may lead to the violation of another. The algorithm cycles through the constraints repeatedly until they are all satisfied. When constraints are in force, the estimates of population means and covariances are based on the unconstrained part-worths. However, since the constrained part-worths are of primary interest, we plot the constrained part-worths to the screen. Only the constrained part-worths are saved to the utility run

CVA Help for use in the market simulator.

727

When constraints are in place, measures of fit (average r-squared) are decreased. Constraints always decrease the goodness-of-fit for the sample in which estimation is done. This is accepted in the hope that the constrained solution will work better for predictions in new choice situations. Measures of scale (Avg. Variance and Parameter RMS), which are based on unconstrained part-worths, will be increased.

728

SSI Web v7

Appendices for SSI Web

729

7
7.1

Appendices for SSI Web

A: Interpreting Conjoint Analysis Data
Conjoint analysis provides a number of outputs for analysis including: part-worth utilities (or counts), importances, shares of preference and purchase likelihood simulations. This appendix discusses these measures and gives guidelines for interpreting results and presenting findings to management. Before focusing on conjoint data, we'll review some fundamentals for interpreting quantitative data. The definitions below are adapted from Statistics for Modern Business Decisions, Fourth Edition, by Lawrence L. Lapin.

The Nature of Quantitative Data:

There are four general types of quantitative data: 1) Nominal data are those wherein the numbers represent categories, such as 1=Male, 2=Female; or 20=Italy, 21=Canada, 22=Mexico. It is not appropriate to perform mathematical operations such as addition or subtraction with nominal data, or to interpret the relative size of the numbers. 2) Ordinal data commonly occur in market research in the form of rankings. If a respondent ranks five brands from best "1" to worst "5," we know that a 1 is preferred to a 2. An example of an ordinal scale is the classification of the strength of tornados. A category 3 tornado is stronger and more damaging than a category 2 tornado. It is generally not appropriate to apply arithmetic operations to ordinal data. The difference in strength between a category 1 and 2 tornado is not necessarily equal to the difference in strength between a category 2 and a 3. Nor can we say that a category 2 is twice as strong as a category 1 tornado. 3) Interval data permit the simple operations of addition and subtraction. The rating scales so common to market research provide interval data. The Celsius scale also is interval scaled. Each degree of temperature represents an equal heat increment. It takes the same amount of heat to raise the temperature of a cup of water from 10 to 20 degrees as from 20 to 30 degrees. The zero point is arbitrarily tied to the freezing point of distilled water. Sixty degrees is not twice as hot as 30 degrees, and the ratio 60/30 has no meaning. 4) Ratio data permit all basic arithmetic operations, including division and multiplication. Examples of ratio data include weight, height, time increments, revenue and profit. The zero point is meaningful in ratio scales. The difference between 20 and 30 kilograms is the same as the difference between 30 and 40 kilograms, and 40 kilograms is twice as heavy as 20 kilograms.

Conjoint Utilities (Part-Worths): Conjoint part-worths are scaled to an arbitrary additive constant within each attribute and are interval data. The arbitrary origin on the scaling within each attribute results from dummy coding in the design matrix. When using a specific kind of dummy coding called "effects coding," utilities are scaled to sum to 0 within each attribute. A plausible set of part-worth utilities for miles per gallon might look like:
30 MPG 40 MPG 50 MPG -1.0 0.0 1.0

730

SSI Web v7 Just because 30 MPG received a negative utility value does not mean that this level was unattractive. In fact, 30 MPG may have been very acceptable to all respondents. But (all else being equal) 40 MPG and 50 MPG are better. The utilities are scaled to sum to 0 within each attribute, so 30 MPG must receive a negative utility value. Other kinds of dummy coding arbitrarily set the part-worth of one level within each attribute to zero and estimate the remaining levels as contrasts with respect to zero. Again, the same cautions regarding interpretation apply. Whether we multiply all the part-worth utilities by a positive constant or add a constant to each level within a study, the interpretation is the same. Suppose we have two attributes with the following utilities:
Blue Red Green 30 20 10 Brand A Brand B Brand C 20 40 10

The increase in preference from Green to Blue (20 points) is equal to the increase in preference between Brand A and Brand B (also 20 points). However, due to the arbitrary origin within each attribute, we cannot directly compare values between attributes to say that Red (20 utiles) is preferred equally to Brand A (20 utiles). And even though we are comparing utilities within the same attribute, we cannot say that Blue is three times as preferred as Green (30/10). Interval data do not support ratio operations.

Counts:
When using Choice-Based Conjoint (CBC), the researcher can analyze the data by counting the number of times an attribute level was chosen relative to the number of times it was available for choice. In the absence of prohibitions (orthogonal plans), counts proportions are closely related to conjoint utilities. If prohibitions were used, counts are biased. Counts are ratio data when compared within the same attribute. Consider the following counts proportions:
Blue Red Green 0.50 0.30 0.20 Brand A Brand B Brand C 0.40 0.50 0.10

We can say that Brand A was chosen 4 times as often as Brand C (.40/.10). As with conjoint utilities, we cannot report that Brand A is preferred to Red.

Conjoint Importances:
Sometimes we want to characterize the relative importance of each attribute. We do this by considering how much difference each attribute could make in the total utility of a product. That difference is the range in the attribute's utility values. We percentage those ranges, obtaining a set of attribute importance values that add to 100, as follows:
Percent Range Importance Brand (B - C) 60 - 20 = 40 26.7 Color (Red - Pink) 20 - 0 = 20 13.3 Price ($50 - $100) 90 - 0 = 90 60.0 ------150 100.0

For this respondent, the importance of Brand is 26.7%, the importance of Color is 13.3%, and the importance of Price is 60%. Importances depend on the particular attribute levels chosen for the study. For example, with a narrower range of prices, Price would have been less important. When summarizing attribute importances for groups, it is best to compute importances for respondents individually and then average them, rather than computing importances from average utilities. For

Appendices for SSI Web

731

example, suppose we were studying two brands, Coke and Pepsi. If half of the respondents preferred each brand, the average utilities for Coke and Pepsi would be tied, and the importance of Brand would appear to be zero! When calculating importances from CBC data, we suggest using utilities resulting from Latent Class (with multiple segments) or HB estimation, if there are attributes on which respondents disagree about preference order. Importances are ratio data. An attribute with an importance of 20 (20%) is twice as important as an attribute with an importance of 10.

Shares of Preference:
Conjoint part-worths and importances are often very difficult for non-researchers to understand. Many presentations to management have gone awry when the focus of the conversation turned to explaining how part-worths were estimated and, given the scaling resulting from dummy coding, how one can or cannot interpret them. We suggest using market simulators to make the most of your data and for communicating the results of the conjoint analysis project to non-researchers. When two or more products are specified in the market simulator, we can estimate what percent of the respondents would prefer each product. Shares of preference are ratio data. Even so, we recognize that noise inherent in the data, the exponent (scaling multiplier if using logit simulations) and the simulation model used can dramatically affect the scaling of shares of preference. A product that captures twice as much share as another in a first choice simulation (or using a large exponent) may capture considerably less than twice the share using the share of preference (probabilistic) model.

Purchase Likelihoods:
During some conjoint interviews such as ACA or traditional full-profile ratings-based conjoint (CVA), respondents may be asked to rate individual products on a 0 to 100 point purchase likelihood scale. This is very helpful to gauge respondent interest in the product, and for scaling the data for use in purchase likelihood simulations. Once we have scaled conjoint data to reflect purchase likelihoods, we can predict how respondents would have rated any combination of attributes included in the study in terms of purchase likelihood. Purchase likelihoods should not be considered as strictly ratio data. A respondent may not truly be twice as likely to purchase a product he rated a 50 versus another he rated a 25. Even so, it is quite common to state that a product with a purchase likelihood of 55 represents a 10% relative increase in purchase likelihood over a product that received a 50.

732

SSI Web v7

7.2

B: Holdout Choice Tasks in Conjoint Studies

We think it is wise to include holdout choice tasks in conjoint interviews, even though they may not appear to be needed for the main purpose of the study. They almost always turn out to be useful, for these reasons: They provide a proximal indication of validity, measured by the utilities' ability to predict choices not used in their estimation. They provide a check on the scaling of the utilities. If the most popular concepts are overpredicted, then the scale parameter should be reduced. If the predictions are too flat, then the scale parameter should be increased. They permit identification and removal of inconsistent respondents. They can be used for testing specific product configurations under consideration. Much value can be added by direct measurement of these concepts. It's hard to design good holdout concepts without some prior idea of respondent preferences. There's no point in asking people to choose among concepts where one dominates in the sense that everyone agrees which is best. And, similarly, it's good to avoid presenting concepts that are equally attractive, since equal shares of preference would be predicted by a completely random simulator. If you present triples of concepts, it's probably best if their shares of choices are somewhere in the neighborhood of 50/30/20. When conducting CBC studies, if you plan to do segmentation with latent class analysis, it's wise to consider the kinds of groups you expect to get, and to design products in holdout choice sets so that one alternative will be much more preferred by each group. This maximizes your ability to confirm the validity of the multi-group Latent Class simulator. It isn't necessary to have many holdout sets to check the validity of your utilities, or their scaling. However, if you want to use those choices to identify and eliminate inconsistent respondents, you need several choice sets. For ACA studies, holdout concepts can be included in the Web-based interview using CiW question types. For CBC, they can be included by adding "Fixed" choice tasks to the design. We've shown an example of a holdout choice task below: If you were shopping for a credit card and these were your only options, which would you choose? MasterCard Discover $30 Annual Fee 12% Interest Rate No Frequent Flier Program $6,000 Credit Line $60 Annual Fee 9% Interest Rate Frequent Flier Program $2,000 Credit Line

Visa

No Annual Fee 15% Interest Rate Frequent Flier Program $4,000 Credit Line

It is probably not very useful to include a "None" option in holdout choice tasks, particularly when these are paired with traditional conjoint exercises which don't have a "None" option. The exception to that rule would be if modeling "None" and using it in the market simulator was of interest. Finally, if you do have several choice sets, it's useful to repeat at least one of them so you can obtain a

Appendices for SSI Web

733

measure of the reliability of the holdout choices. Suppose your conjoint utilities are able to predict only 50% of the respondents' holdout choices. Lacking data about reliability, you might conclude that the conjoint exercise had been a failure. But if you were to learn that repeat holdout tasks had reliability of only 50%, you might conclude that the conjoint utilities were doing about as well as they possibly could, and that the problem lies in the reliability of the holdout judgements themselves.

734

SSI Web v7

SMRT (Market Simulator) Help

735

8
8.1

SMRT (Market Simulator) Help

What Is SMRT?
SMRT stands for "Sawtooth Software Market Research Tools" and is a separate program from the SSI Web system. The SMRT Market Simulator is used for estimating the likely acceptance or interest in different product concepts defined using the attributes and levels included in your conjoint analysis study. You install SMRT separately from the CD-ROM or from our website. SMRT was released as a software system prior to SSI Web. It is our older Windows-based software for conducting ACA, CBC and traditional conjoint (CVA) analysis. The system includes market simulation capabilities (supporting all three of our conjoint approaches) and additional tools for data management and analyzing CBC data, including Counts, logit analysis, and latent class estimation. However, SMRT cannot create web-based projects. Interviewing with SMRT is done via floppy disks, from a hard drive on a local or networked machine, or via paper-and-pencil (in the case of CBC and CVA). When the internet increased in popularity for market research interviewing, Sawtooth Software created a new platform for Web-based surveys called SSI Web rather than try to retrofit the older SMRT platform for web-based interviewing. Since the market simulator (and other analytical tools for CBC) already existed under SMRT, Sawtooth Software elected not to rebuild those same tools within the new SSI Web platform. The SMRT platform can import data and utility runs developed within the SSI Web system. You can also export segmentation data from SSI Web to a text-only format and import those data into SMRT for segmentation purposes. When you analyze data using SMRT, you need to create a new project within the SMRT software system. You may store this new project in the same folder as your SSI Web project, or you may install it in a separate folder. SSI Web and SMRT are independent software systems, but they can "speak" to one another through Import, Export and Merge processes. A special version of SMRT may be installed by clients (the Client Conjoint Simulator). This way, clients can perform market simulations independent of consultants. Sawtooth Software also has a web-based marketing simulator, if you need a less feature-filled and more user-friendly market simulator than SMRT.

736

SSI Web v7

8.2
8.2.1

Data and Utility Run Management

Importing Utility Runs
Overview
SMRT can automatically import conjoint part-worth data generated by any of Sawtooth Software's current conjoint software systems by clicking Analysis | Run Manager | Import. (The number of attributes and levels per attribute to be imported cannot exceed the limitations of your system). It can also import part-worths that you have generated in most any other standard way, as long as you format those data as an .HBU file and optionally create a file describing the coded linear terms ( .VAL) if they apply. The formats automatically imported by the Market Simulator are: HB Utilities (*.hbu, *.hbc) ACA or ACA v4.x (*.utl) CVA (Traditional Conjoint) Utilities (*.utl) Latent Class Utilities (*.pxx) Generic Conjoint File (*.hbu) Latent Class, HB Utilities, and the Generic Conjoint *.hbu formats are the most complex, allowing for a great deal of flexibility in the inputs, including first-order interactions and linear term specifications of utilities.

Importing Utilities into the Market Simulator

The first step in importing utilities into the Market Simulator is to create a new study or open an existing study. Open the SMRT software by clicking Start | Programs | Sawtooth Software | Sawtooth Software SMRT. To create a new study, select File | New, and browse to the directory in which you want to save all study-related files. Remember: The SMRT and SSI Web systems work independently, so you need to establish a new project name and project file for the SMRT system to analyze your conjoint data. Specify a study name, and the extension ".SMT" will automatically be added. If you own more than one conjoint analysis license, you are asked to specify the study type (i.e. ACA, CVA or CBC). To open an existing SMRT study (not an SSI Web study), select File | Open, and browse to the directory containing the studyname.SMT file. Double-click the studyname.SMT file. If attributes and levels definitions exist in the study, they must match the attributes and levels for the conjoint data to be imported. After you have opened the study, choose Analysis | Run Manager. The Import option on the Run Manager dialog lets you import conjoint utility (part-worth) data from text-only files for use within the Market Simulator. You can import multiple sets of conjoint utilities which become separate selectable runs. The utility data can come from a number of formats or sources: HB Utilities (*.hbu) Hierarchical Bayes utilities generated by CBC/HB, ACA/HB, or Adaptive CBC (see the HB manuals for data layout). If importing an .HBU file, the attribute labels are read from the header section of the .HBU file. part-worth utilities from the ACA system (see the ACA manual for data layout) or ACA v4.x. If you import a studyname.UTL file, labels for the

ACA (*.utl)

SMRT (Market Simulator) Help

737

attribute levels are imported from the studyname.ACA file. If a studyname.ACA file does not exist, default attribute and level labels are assigned. CVA Utilities (*.utl) part-worth utilities from the CVA v2.x system. If you import a studyname.UTL file, labels for the attribute levels are imported from the studyname.CVA file. If a studyname.CVA file does not exist, default attribute and level labels are assigned. Latent Class utilities generated by the CBC Latent Class v2 Module (see the Latent Class manual for data layout). When importing Latent Class utilities, the data are read from two files: *.Pxx (e.g. studyname.P05 for a 5-group solution), which includes information about respondents' probabilities of membership in each group; and the *.LCU file which contains the average utility estimates for each group. Note that the .Pxx file is not generated by default, but must be requested by a parameter in the .PAR file prior to generating the Latent Class run. During the import process, individual-level utility data are generated by multiplying the probabilities of membership by the vector of utilities for each class. If importing Latent Class data, the attribute levels are read from the header section of the .LCU file. If linear specification of quantitative attributes has been used, there must be an accompanying .VAL file in the same directory. You can also merge the latent class membership for use as banner points and/or filters. To merge group (class) membership, select File | Merge Variables | Add... and then choose Latent Class 2.x Group Membership (*.Pxx). SMRT assigns each respondent to the group for which he/she has the highest probability of membership (ties broken randomly). To use group membership from latent class as a banner point, you must first specify the merged variable as a Custom Segment. Generic Conjoint (*.hbu) With the Generic Conjoint .HBU format, you can import conjoint utility data generated in any other way or software program, including main effects, first-order interactions or linear term specifications, as long as the attributes and levels do not exceed the capacity of your system and the data are formatted as an .HBU file. The generic .HBU format assumes one case per respondent (not "Draws" as can be an option for Hierarchical Bayes estimation). Data from text-only files are read with 15 significant digits of precision. When utility run data are imported, four files are modified or created: Studyname.UCS (a binary data file containing respondent numbers and conjoint utilities) Studyname.QNR (a binary data file containing attribute level labels and questionnaire settings) Studyname.DAT (a binary data file containing respondent numbers and any segmentation information) Studyname.IDX (a binary index file describing the layout of the Studyname.DAT file). If your study had no respondent data prior to importing a utility run, a list of respondent numbers (corresponding to your utility run) is created in a Studyname.DAT file. If your study already included respondent numbers and data, then the conjoint part-worths are written to the .UCS file and it is assumed that corresponding respondent records are available in the .DAT file (if applying banner points, filters or weights). If no attribute labels are provided, SMRT provides default labels. If conjoint attributes and level labels exist in your study prior to importing conjoint utility data, those definitions are

Latent Class (*.pxx)

738

SSI Web v7 preserved and the utilities to be imported must match that layout. If there is a mis-match between the number of imported attribute levels and the number of existing levels in the study, SMRT will issue a warning message. If your text-only conjoint data file included segmentation information you want to import, you must merge those data separately using File | Merge Variables.

Exporting Utilities into Text-Only Format

The Export option lets you export conjoint utility data from the binary .UCS file into a text-only .HBU format. Data are written with up to 5 decimal places of precision. Text-only conjoint data in an .HBU format can be imported for use in the Market Simulator.

SMRT (Market Simulator) Help

739

8.2.2

Merging Segmentation Data

Introduction
SMRT includes a Merge Wizard to lead you through the process of merging additional descriptive or segmentation variables into that same data file. These variables might be demographics, such as gender, household size or age, or information about a firm such as company size, volume purchased, or industry classification. You can also merge continuous variables to be used as respondent weights. The type of file the typical user will merge will probably be in text-only (DOS Text) format as created by SSI Web. The data file to be merged must include a respondent classification (Case ID) number. All merged data must be numeric only. After you have merged additional variables into your data set, these are automatically available to you as "stub variables" for use in the Tables program or as respondent weights in either the Tables or the Market Simulator. If you undertake the additional step of defining Custom Segmentation Variables based on these merged variables, you can also use the merged information as filters or "banner points" during simulations if using individual-level utilities. The next few sections include basic information about merging segmentation information and defining Custom Segmentation Variables. Additional help and details are available in the on-line help within SMRT, accessed by pressing F1 at any time during the merge process.

Merging Segmentation Data from Text-Only Files

Often users merge segmentation information (demographics, usage data, firmographics, etc.) into their market simulators for use as segmentation, filters or weighting variables. These data can come from any source (such as SSI Web), as long as the data have been formatted as a text-only file (either fixed-column or delimited). The respondent numbers in the file(s) to be merged must be numeric and must match the respondent numbers in the conjoint data file. The cases do not necessarily need to be in the same sort order. The Merge program in the Market Simulator has a "Merge Wizard" that leads you through the entire process. For the Merge program to work properly, you must have unique respondent numbers that match in both the source and destination data files (numeric data only). You should not have duplicate respondent numbers in either file. Any respondent in the file of conjoint part-worths not found in the file to be merged will receive a "missing" value for the selected merge variables. Delimited Files When you specify to merge information from a delimited text-only file into your data file (*.dat), you need to instruct the software on how to interpret the data file. You do so by specifying the delimiter (the character that separates data fields, such as a space, tab or comma) and specifying what value (if any) is used to designate missing values. This value may be any string of alphanumeric characters or symbols. The Merge Wizard asks you to specify whether respondent data is on one line (example below, with three respondents numbered 1001 to 1003) or otherwise.
1001 1002 1003 3 2 1 2 3 3 5 1 3 7 5 7

In the example above, each respondent record occupies a single line, and the data are separated by

740

SSI Web v7 blank spaces. This layout is an example of "one line per respondent record." Below is an example in which each respondent's data span more than one line:
1001 3 2 1002 2 3 1003 1 3 5 1 3 7 5 7

In the example above, each respondent record is spread across two lines. In order to understand how to read this layout, the Merge Wizard asks you how many fields per respondent record there are. In this case, there are five (note that this number includes the respondent number field). The Merge Wizard asks you to indicate in which field the respondent number is located. Note that this is not the "column number" where each column is a single character, but the field in which fields are separated by the delimiter. In the examples above, respondent number is located in field #1. To avoid misinterpretation of text strings (which may contain delimiters), text in the data file may be enclosed in quotation marks. Single (') or double (") quotation marks are supported. The following is an example of how Merge reads data depending on the user-specified parameters for handling string variables: Text in Delimiter Comma Comma Space Space Fixed Text-Only Files When you are merging data from a fixed column text-only file, the Merge Wizard asks you to specify in which columns the data are located and the length of each field. A column is the width of a single character. Many text editors (such as Microsoft's DOS editor, which you can access from the DOS prompt by typing EDIT and pressing the ENTER key) provide a column counter at the top or bottom of the edit window. With fixed column ASCII files, the data are aligned in columns when viewed with a non-proportional font (such as Courier) such as follows (the first two lines below are a reference for counting columns, they would not appear in the data file):
1234567890123456789012345678901234567890 ---------------------------------------6997 3 25 2 25283 23 1 234 9 6 4

Quotes? Yes No Yes No

Data 5,6,"B K O",32 5,6,"B K O",32 5 6 "B K O" 32 5 6 "B K O" 32

Read as 4 fields 4 fields 4 fields 6 fields

String read as B K O (1 string) "B K O" (1 string) B K O (1 string) "B K O" (3 strings)

Assume that the first variable is the respondent number (though it is not necessary that it be the first variable in the file). It is right-justified within an area that begins in column 1 and has a length of 6. The next variable starts in column 15 and has a length of 2. The third variable starts in column 28 and has a length of two (note that the second case has a missing value). The final variable starts in column 34 and has a length of 1.

SMRT (Market Simulator) Help 8.2.2.1

741

Custom Segmentation Variables

The variables you merge into your data set (i.e. from SSI Web or a Text-Only source) though valid automatically as weights are not available as banner points and respondent filters until you first create new variables based on those data called Custom Segmentation Variables. A Custom Segmentation Variable is one you create based on the values of one or more other merged variables. Custom Segmentation Variables can be used as banner points and respondent filters in the Tables and the Market Simulator programs. You can create Custom Segmentation Variables by selecting Analysis | Custom Segments. The details for specifying these variables is available in the on-line help by pressing F1 when you are at the Custom Segments dialog. As an example, we could create a Custom Segmentation Variable based on gender and income. Suppose we merged a variable called Q31 from an SSI Web questionnaire that classified a respondent's gender (Male=1, Female=2), and Q32 that reported a respondent's income (continuous variable). These variables would not be available for choosing as banner points or respondent filters in the simulator until we had created a custom segmentation variable. We might call this custom variable Gender_Income, and indicate the following definitions: Definitions for Custom Segmentation Variable: Gender_Income Segment Label Female Low Income Female High Income Male Low Income Male High Income Logical Definition (Q31 = 2) & (Q32 <= 25000) (Q31 = 2) & (Q32 > 25000) (Q31 = 1) & (Q32 <= 25000) (Q31 = 1) & (Q32 > 25000)

After creating a new Custom Segmentation Variable called Gender_Income, these segments are available for selection as banner points and respondent filters. Though our example above used two variables to define a Custom Segmentation Variable, you could use only one variable, or many more variables. The Market Simulator supports sophisticated ways for defining Custom Segmentation Variables, including the following operators: = , >, >=, <, <=, != (not equal to), & (and), | (or), ! (not), $MISSING (missing value). You can include complex statements that include parentheses for controlling the flow of logical operations.

742 8.2.2.2

SSI Web v7

Weighting Respondents
Weighting lets some respondents have more impact on summary statistics than others. For example, a respondent with a weight of 2.0 is counted twice as much as another respondent with a weight of 1.0. Weighting is useful for adjusting a sample to reflect known population characteristics. Weights can be applied during all stages of analysis. There are two ways to define weights: 1. Use a merged variable value as a weight. You may have calculated a weight for each respondent and placed those weights along with the respondent numbers in a text-only file for merging with your part-worth data. 2. Assign weights to segmentation categories. In this case, the weighting variable must be defined as a Custom Segmentation Variable. For example, assume we wanted to weight our sample based on Gender. The table below shows the weight that needs to be given to respondents to achieve the target gender representation. Actual Proportion Male Female 0.35 0.65 Target Proportion 0.49 0.51

Weight 0.49/0.35 = 1.4000 0.51/0.65 = 0.7846

If we apply these weights, the average weight will be 1.0, and the unweighted number of respondents will be equal to the weighted number of respondents. Whether you use a merged variable value as the weight or assign weights to categories of a segmentation variable, we strongly suggest your weights have this property. If not, some statistics in Tables will be incorrect.

SMRT (Market Simulator) Help

743

8.2.3

Data Management Tools

Introduction
The Market Simulator includes some data management tools that can be useful for some situations. These tools are accessed from the Tools menu, and include: View/Edit Data Index Data Doctor Optimize Study files This section describes the purpose and use for these data management tools.

View/Edit Data
The Market Simulator lets you view the respondent numbers and any segmentation variables that have been merged into the conjoint part-worth data. (You cannot view the part-worths themselves through this option.) When you click Tools | View/Edit Data, a dialog is displayed with three separate tabs: Respondents, View Respondent Data and Edit Respondent Data. Respondents Tab When you click the Respondents tab, SMRT reads the .IDX file located in the study directory, which provides an index of the .DAT file (pointers to the location of data segments within a .DAT file) including a list of the respondent numbers. Respondent numbers are initially displayed in sorted order (by default) and an asterisk following a number indicates a gap in the sequence. Note that SMRT does not read the studyname.DAT file to generate the report on this tab. Therefore, if your .IDX file for some reason has become corrupted or for some reason does not match (pertain to) the .DAT file, it is possible to see information on this tab that is really not in the .DAT file. To ensure that the information in the .IDX file matches the .DAT file, you should run either Tools | Index, or Tools | Data Doctor. Close Print New Disk Filter Closes the current dialog. Prints the information displayed on the screen. Lets you view or edit data in another directory or floppy disk. (Drop down list). This drop-down list lets you filter the respondents to be displayed by complete/incompletes or a particular disposition. It also lets you view summary counts of the number of respondents by disposition or status (these disposition and status codes are a carryover from Sawtooth Software's Ci3 system, and are not relevant to the Market Simulator). By default, the respondent numbers in this window have been sorted in ascending order. Note that the actual order of the records in the file will likely be different. If you uncheck the Sort box, the actual order of the respondent records is displayed. Note that this check box only changes the order in which the records are displayed on the screen, it doesn't actually change their order in the data file. Displays the disposition codes (applies to Ci3 data onlysee the Ci3 manual for more details) for each respondent record.

Sort

Disposition

View Respondent Data Tab This dialog reads the information from the studyname.DAT file and displays any segmentation data

744

SSI Web v7 that you may have merged with the conjoint part-worths. If not all of the merged information can be displayed within the window, you can use the scroll bar at the right of the dialog box to see values for later variables. You can scroll through data for different respondents by clicking on the double left-or-right arrows at the bottom of the dialog box. Alternatively, you can select a different respondent by clicking on that respondent's number at the left of the screen. Close Print << and >> New Disk Closes the current dialog. Prints the information displayed on the screen. These buttons let you move between adjacent records in your data file. Lets you view or edit data in another directory or floppy disk.

Edit Respondent Data This dialog shows detailed information on each respondent, permitting editing the values for merged information or changing the respondent number. The Answer: field lets you change the value of any response. You can make tentative editing changes to answers that will not become permanent until you say that you want to retain them. If you click the Change Respondent Number button in the upper right hand corner, a box appears in which you can provide a different respondent number. You change a respondent number by typing in a different respondent number, and then clicking on OK. Unlike segmentation variable values that you change, a respondent number change is made immediately, and you will not have the opportunity to decide that this change should be ignored. You can also delete this respondent if you like. You do that by clicking the box labeled Delete This Respondent. A check mark appears in the box, indicating that the respondent will be deleted. If individual-level utility runs have been saved, the utility data will also be deleted for this respondent. If you change your mind you can click that box a second time and remove the check mark. You can move from one merged segmentation variable to the next by scrolling with the double arrows at the bottom of the dialog box, or scrolling in the box in which the question (variable) name is displayed. When you have finished with this respondent, click the Respondents Tab at the top of the dialog box to again see the list of respondents with data in this file, and select another respondent number. Then click again on the Edit Respondent Data tab, and you will be able to edit that respondent's data. When you make changes to the data file, they are recorded in a file named questionnaire.log. That file is in text-only format and can be viewed by any editor or word processor. Close When you click the Close button, any changes you made to the data are made permanent. You can make modifications to multiple records before clicking the Close button. These buttons let you move between adjacent records in your data file. Lets you view or edit data in another directory or floppy disk.

<< and >> New Disk

Delete This Respondent Deletes the current respondent (when you click the Close button). Change Respondent Number Lets you change the case identification number. This change is made immediately rather than when you click the Close button.

SMRT (Market Simulator) Help

745

Index
The data file (studyname.DAT) used by SMRT is always accompanied by an Index file (studyname.IDX). This file contains information about where each respondent's data are located within the file. If your studyname.IDX file gets lost or corrupted, you can easily regenerate it by clicking this option. By default, the software will re-index the file within your study directory. You can also regenerate lost or corrupted .IDX files in other directories (or your floppy drive) by specifying a different path.

Data Doctor
The Data Doctor is a utility for repairing data files (studyname.dat) that have become corrupted. Occasionally, sectors on disks fail or hardware doesn't properly record the data. With very large data sets and tens of thousands of respondents, data corruption or loss becomes increasingly likely. Sometimes a single corrupted record in the middle of a data file makes it impossible for the Market Simulator to understand how to deal with the subsequent respondents. Other times, the information at the top of the data file is corrupted so that the Market Simulator doesn't even know how to begin. The Data Doctor analyzes a .DAT file and checks it for validity, skipping non-critical sections of the file if it finds errors in them. The user is prompted as to what action to take for each error encountered. This process can be slow with large data files. The Data Doctor can operate on the accumulated data file in the study directory, or on any other drive/directory.

Optimize Study Files

This tool is used to optimize the study files (.SMT, .QNR and .UCS), removing information no longer needed. For example, when you delete a utility run (from the .UCS), the section is only marked for deletion until you choose Optimize Study Files. Optimizing study files will help speed access to information in those files.

746

SSI Web v7

8.2.4

Tables Program
Overview
The functionality and output of the Tables program resembles the cross-tab capabilities found in many statistical and cross-tab packages today. However, the SMRT Market Simulator is primarily concerned with conjoint analysis; the Tables program is basic and relatively inflexible. If you need to analyze the results with a more sophisticated package, SMRT or SSI Web can export data for use in other software systems.

Analyzing Nominal/Ordinal Data

Nominal data are numeric values, where the numbers refer to categories or classifications. For example, one might code gender as: Male = 1 Female = 2 An example of a type of survey question that might be analyzed is shown below: Which of the following best describes the highest level of education you achieved? o Some high school o Graduated high school o Some college o Graduated college o Some post-graduate studies o Post-graduate degree o Doctoral degree If we assign the following codes: Some high school = 1 Graduated high school = 2 Some college = 3 Graduated college = 4 Some post-graduate studies = 5 Post-graduate degree = 6 Doctoral degree = 7 it is true that larger numbers reflect greater education (most would agree on this point). In contrast to the previous example where the coded value really had no quantitative meaning other than nominal classification, these data reflect ordinal scaling. However, as with nominal data, it still doesn't make sense to apply mathematical operations such as addition, subtraction, multiplication or division to ordinal data. For example, a person with a "post-graduate degree" (code 6) doesn't necessarily have twice as much education as a person who completed "some college" (code 3).

SMRT (Market Simulator) Help The Table Display

747

When you analyze survey responses using Tables, a table of results (in this case, frequencies) is generated and displayed in the report window:
Total Education Some high school 3 Graduated high school 35 Some college 68 Graduated college 73 Some post-graduate studies 18 Post-graduate degree 11 Doctoral degree 4 Total Missing 212 -

This simple table displays summary frequencies for the entire sample. Frequencies represent the number of times respondents answered (or were classified) in a particular way. Every time you click Compute!, a new table (or set of tables if you have selected multiple variables) is appended to the report window. You clear the window by clicking Clear. In addition to frequencies, there are other statistics you can include (by clicking Statistics) in the table appropriate for nominal or ordinal variables: Column percent Row percent Table percent Chi-Square These statistics are described in any good statistics textbook. You can also analyze the results by another variable, such as Gender. In the example below, Gender is the banner (column) variable, and Education the stub (row) variable. This display is often referred to as a cross-tabulation.
Education by Gender Male Education Some high school Graduated high school Some college Graduated college Some post-graduate studies Post-graduate degree Doctoral degree Total Missing 3 18 38 33 8 6 3 109 Gender Female 17 30 40 10 5 1 103 Total 3 35 68 73 18 11 4 212 -

748

SSI Web v7 This display lets us compare educational achievement by gender. For example, 33 males went as far as graduating college (but no farther) vs. 40 females. If we include column percentages, we make better sense of the results, since there are an unequal number of males and females in the sample.
Education by Gender Gender Male Female Education Some high school Graduated high school Some college Graduated college Some post-graduate studies Post-graduate degree Doctoral degree Total Missing 3 3% 18 17% 38 35% 33 30% 8 7% 6 6% 3 3% 109 100% 17 17% 30 29% 40 39% 10 10% 5 5% 1 1% 103 100% Total 3 1% 35 17% 68 32% 73 34% 18 8% 11 5% 4 2% 212 100% -

Analyzing Continuous Variables

You can also analyze continuous variables with the Tables program. The following statistics are available in Tables and appropriate for analyzing continuous data: Mean Standard deviation Variance Minimum Maximum Standard Error of Mean These statistics are described in any good statistics textbook. Continuous variables can only be specified as stub variables; they cannot be used as a banner variable unless you recode them to discrete categories by creating a custom segment. If a continuous variable you are analyzing has many unique values, the table can become extremely large. A simple remedy is to turn off frequencies and only request summary statistics, such as the mean, minimum, maximum, standard deviation and standard error.

SMRT (Market Simulator) Help

749

Some Notes on Respondent Weighting

The Tables program lets you weight respondents differentially. Weighting lets some respondents have more impact on the statistical summaries than others. For example, a respondent with a weight of 2.0 is counted twice as much as another respondent with a weight of 1.0. Weighting is useful for adjusting a sample to reflect known population characteristics. Weights affect every statistic mentioned earlier in this section, except for Minimum and Maximum. When you request weighted tables, both weighted and unweighted totals are reported. We strongly suggest you assign weights so that the average weight is equal to 1.0. If the weights do not average 1.0, the total unweighted number of respondents will not equal the total weighted respondents. More critically, many statistics are incorrect if the average weight is not equal to 1.0, including: Standard error Chi Square You will also see some inaccuracy in the following statistics because of the division by (n-1), especially if you have small sample sizes: Standard deviation Variance

750

SSI Web v7

8.3
8.3.1

Using Simulators to Answer Strategic Questions

Overview
The Market Simulator is usually considered the most important tool resulting from a conjoint project. The simulator is used to convert raw conjoint (part-worth utility) data into something much more managerially useful: simulated market choices. Products can be introduced within a simulated market scenario and the simulator reports the percent of respondents projected to choose each. A market simulator lets an analyst or manager conduct what-if games to investigate issues such as new product design, product positioning, and pricing strategy.

A Warning about Interpreting the Output of Market Simulators

Under very controlled conditions (such as markets with equal information and distribution), market simulators often report results that closely match long-range equilibrium market shares. However, conjoint part-worth utilities cannot account for many real-world factors that shape market shares, such as length of time on the market, distribution, out-of-stock conditions, advertising, effectiveness of sales force, and awareness. Conjoint analysis predictions also assume that all relevant attributes that influence share have been measured. Therefore, the share of preference predictions usually should not be interpreted as market shares, but as relative indications of preference. Divorcing oneself from the idea that conjoint simulations predict market shares is one of the most important steps to getting value from a conjoint analysis study and the resulting simulator. While "external effect" factors can be built into the simulation model to tune conjoint shares of preference to match market shares, we suggest avoiding this temptation if at all possible. No matter how carefully conjoint predictions are calibrated to the market, the researcher may one day be embarrassed by differences that remain.

SMRT (Market Simulator) Help

751

8.3.2

What Is a Market Simulation?

A conjoint study leads to a set of utilities (part-worths) that quantify respondents' preferences for each level of each attribute. These part-worths can be analyzed in a number of ways. You can examine each respondent's part-worths (but this task could become overwhelming). You might summarize the average part-worth utilities, or compute average importances. You could create graphs and charts to display that information, but to many it might seem somewhat abstract and difficult to grasp. Examining average responses could also fail to detect important segments of the market that have unique and targetable preferences. A good market simulator is like having all of your respondents gathered in one room for the sole purpose of voting on product concepts and competitive scenarios (defined in terms of the attribute levels you measured) you show them. You walk into the room, show them a market scenario (i.e. products A, B and C), and they vote for the one(s) they prefer. Millions of potential products and market situations could be evaluated, and your captive audience would never get tired, ask for lunch breaks, or require you to pay them by the hour. How does a market simulator work? Let's suppose we had a way (such as through conjoint or choice analysis) to quantify how much people liked the different qualities of ice cream cones. Let's refer to those preferences as part-worth utilities, and assume the following values for a given respondent:
Utility 0 30 40 50 25 0

Chocolate Vanilla Strawberry $0.60 $0.80 $1.00

Using those utility values, we could predict how he would choose between a vanilla cone for $0.80 or a strawberry cone for $1.00.
Vanilla (30 utiles) + $0.80 (25 utiles) = Strawberry (40 utiles) + $1.00 (0 utiles) = 55 utiles 40 utiles

We'd predict he would prefer the vanilla cone. If we had data for 500 respondents, we could count the number of times each of the two cones was preferred, and compute a "Share of Preference," also referred to as a "Share of Choice":
Vanilla @ $0.80 Strawberry @ $1.00 Share of Choice 300/500 = 0.60 200/500 = 0.40

In our hypothetical market simulation, 60% of the respondents preferred the vanilla, and 40% the strawberry cone. This illustrates the most simple simulation approach, referred to as the First Choice model.

752

SSI Web v7

8.3.3

Why Conduct Market Simulations?

Looking only at average preferences (part-worth utilities) can mask important market forces caused by patterns of preference at the segment or individual level. Marketers are often not interested in averages, but in the targetable, idiosyncratic behavior of segments or individuals. For example, consider the following three respondents, and their preferences (utilities) for color:
Utilities for Color Blue 50 0 40 30 Red 40 65 30 45 Yellow 10 75 20 35

Respondent A Respondent B Respondent C Average:

Looking only at average preferences, we would pronounce that red is the most preferred color, followed by yellow. However, if one of each color was offered to each respondent, red would never be chosen under the First Choice model, yellow would be chosen once, and blue twice the exact opposite of what aggregate part-worth utilities suggest. While this is a hypothetical example, it demonstrates that average part-worth utilities do not always tell the whole story. Many similar, complex effects can be discovered only through conducting simulations. Some reasons for conducting conjoint simulations include: 1. Conjoint simulations transform raw utility data into a managerially useful and appealing model: that of predicting market choice (Share of Preference) for different products. Under the proper conditions, shares of preference quite closely track with the idea of market share something most every marketer cares about. 2. As demonstrated earlier, conjoint simulations can capture idiosyncratic preferences occurring at the individual or group level. These "hidden" effects can have a significant impact on preference for products in market scenarios. When multiple product offerings have been designed to appeal to unique segments of the market, capturing such effects is especially important for accurately predicting preference. 3. Conjoint simulations can reveal differential substitutability (cannibalism/cross-elasticity effects) between different brands or product features. If two brands are valued highly by the same respondents (have correlated preferences), these brands will tend to compete more closely. Product enhancements by one of these brands will result in more relative share being lost by the correlated brand than by other less similar brands within the same simulation. Examining aggregate part-worth utilities cannot reveal these important relationships. 4. Conjoint simulations can reveal interaction effects between attributes. If the same respondents that strongly prefer the premium brand are also less price sensitive than those who are more likely to gravitate toward a discount brand, sensitivity simulations will reflect a lower price elasticity for the premium relative to the discount brand. A similar interaction effect can occur between many other types of attributes: such as model style and color. Note when using CBC data: It is important to note that complex effects other than two-way interactions such as cross-effects cannot be reflected using the model of aggregate-level logit offered by our CBC system. Latent Class is a technique for estimating part-worth utilities and reflecting respondent differences at the group/segment level, and CBC/HB (Hierarchical Bayes) is a way to estimate utilities at the individual level for CBC data. Because they are built on individual-level preferences, simulators based on HB models are able to reflect the important and complex behaviors mentioned earlier. It is not surprising that Latent Class and CBC/HB have been shown to outperform similarly-defined aggregate level logit models in terms of predictive validity. ACA (Adaptive Conjoint Analysis) and CVA (Full-Profile Conjoint Analysis) capture respondent-by-

SMRT (Market Simulator) Help

753

respondent preferences and are thus very useful inputs to this and other market simulation models.

754

SSI Web v7

8.3.4

Typical Questions for Conjoint Simulators

There are many marketing strategies that can be investigated with the conjoint simulator. Here are three of the most common: 1. Given a current competitive environment, what product should I offer to maximize interest in my offering? How can I modify an existing product to capture more relative demand? A Market Simulator lets you input multiple products and place them in simulated competition one with another. Each product is defined using the attribute levels measured in the conjoint study (brands, colors, prices, speeds, warrantees, etc.). Therefore, if you have measured the relevant brands and features offered in the market, you can simulate a realistic market scenario within the Market Simulator. Within that market scenario, you can add a new product and see how well it competes. If the goal is to maximize share, offering the best features at the lowest price is often the trivial solution. The Market Simulator focuses on the demand side of the marketing equation; but it is also important to pay attention to the supply side and take the costs of producing different products/services into consideration. If you have cost information available to you, the Market Simulator permits you to investigate the incremental benefits of different features of a product relative to the cost of offering them. (SMRT does not provide an automatic way to input cost information, but you can divide the incremental gain in share of preference by the incremental cost to produce a profitability index.) 2. What is the relative price sensitivity of different brands? If I raise my price by 10%, how will it affect my brand? How will it affect competitors' brands? You can conduct "sensitivity analysis" for attributes such as price using the Market Simulator to generate relative demand curves. The approach involves holding all other brands at a constant price and changing the price of a single brand, recording the relative share at each point for that brand along the price continuum. If your conjoint data reflect respondent differences (all models offered by Sawtooth Software except for aggregate CBC logit), differential cross-elasticities can be investigated through this approach. 3. What portfolio of products can I offer to appeal to different market segments and maximize overall share? The market simulator lets you merge additional segmentation variables (such as demographics or firmographics) with your conjoint part-worths. If you have segmentation information, you can investigate product formulations that appeal to different groups of respondents. It is likely that by designing products that appeal uniquely to targetable segments that you can increase overall share for your product line or occupy a niche that is not currently being served. These three strategic questions and the simulation strategies for responding to them are illustrated later within this documentation.

SMRT (Market Simulator) Help

755

8.4
8.4.1

Market Simulation Theory and Models

Base Case Scenario
Introduction
Market Simulations provide a very useful way to use conjoint/choice data. Simulations provide an intuitive tool to move from the esoteric realm of part-worth estimates/effects toward the practical world of predicting buyer behavior for specific market situations. Before using the Market Simulator within SMRT, you should become familiar with some terminology, issues and theory. This section will discuss setting up a base case scenario, the scaling of simulation results, the RedBus/Blue-Bus problem, and the five simulation models offered in the Market Simulator. Finally, External Effects and the use of the None weight are introduced.

Base Case Scenario

Usually the first step in using the market simulator is to define a "Base Case" scenario. A base case typically reflects a current (or future) market scenario: your brand vs. the relevant competition. If there is no relevant competition, or your conjoint study was designed to model only your product, the base case may be a single product, reflecting a likely configuration. The Market Simulator lets you input the market scenario in a grid format, where the products are rows (you can have up to 100), and the attributes are columns (you can have up to 30). Attribute 1 (Brand) 1 2 3 Attribute 2 (Package) 2 1 3 Attribute 3 (Color) 3 2 1

Product Product A Product B Product C

You provide text labels in the first column to identify the products. In the attribute cells, you type the numeric value associated with different attributes. For example, Product B is defined by level 2 of Brand, level 1 of Package and level 2 of Color. (The Market Simulator displays the list of attributes and the codes associated with each level as you enter values in the grid.) After defining the market scenario, you should decide which simulation method is appropriate for your data and the types of strategic questions you intend to answer. Later, we'll describe all five available models in the Sawtooth Software market simulator and provide some notes and recommendations for each. After you have chosen the appropriate simulation technique, you can begin conducting simulations. Typically, one first examines the shares of preference (or choice) given to the products in the base case. Then, modifications to the base case are investigated by altering the base case itself and rerunning the analysis, or by adding additional "scenarios." A scenario is just another name for a defined set of competitive products, and setting up each subsequent scenario feels just like defining the first base case scenario. The market simulator lets you input many simulation scenarios, and stores these for your convenience. Prior to introducing the different models of choice offered by the Market Simulator, it is instructive to cover two topics: the Exponent and the Red-Bus/Blue-Bus problem.

756

SSI Web v7

8.4.2

The Exponent (Scale Factor)

Assume that you set up a simulation as defined in the previous section with three products: A, B and C. Also assume that you are simulating the projected choice for just one individual under a Share of Preference model (described in greater detail later). After clicking Compute!, simulation results for this individual might come back as follows:
Product A B C Total Share of Choice 10.8% 24.0% 65.2% 100.0%

Note that in conjoint simulations, the resulting shares are normalized to sum to 100%. We interpret these results to mean that if this respondent was faced with the choice of A, B, or C, he would have a 10.8% probability of choosing A, a 24.0% probability of choosing B, and a 65.2% probability of choosing C. Note that B is more than twice as likely to be selected as A, and C is more than twice as likely to be chosen as B. Let's suppose, however, that the differences in share seen in this simulation are really greater than what we would observe in the real world. Suppose that random forces come to bear in the actual market (e.g. out-of-stock conditions, buyer confusion or apathy) and the shares (probabilities of choice) are really flatter. We can often tune the results of market simulations using an adjustment factor called the Exponent. The table below shows results for the previous simulation under different settings for the Exponent:
Share of Choice under Different Exponent Values 0.01 0.5 1.0 33.0% 20.2% 10.8% 33.3% 30.1% 24.0% 33.7% 49.7% 65.2% 100% 100% 100%

Product A Product B Product C Total

2.0 2.4% 11.6% 86.0% 100%

5.0 0.0% 0.7% 99.3% 100%

The exponent is applied as a multiplicative factor to all of the utility part-worths prior to computing shares. As the exponent approaches 0, the differences in share are minimized, and preference is divided equally among the various product offerings. As the exponent becomes large, the differences in share are maximized, with nearly all the share allocated to the single best product. (Given a large enough multiplier, the approach is identical to the First Choice model, with all of the share given to a single product.) If you have solid external information (such as existing market share data) and have reason to expect that conjoint shares should resemble market shares (see earlier assumptions), you may want to tune the exponent within simulations. Or perhaps you have choice data from a holdout choice scenario included in your conjoint survey. You may decide to tune the exponent so that simulated shares resemble these holdout shares. If you do not have solid external information, you probably should not change the exponent from the default value of 1. Once the exponent is "set" for a data set, one typically does not change it from one simulation to the next.

SMRT (Market Simulator) Help

757

8.4.3

The Red-Bus/Blue-Bus Problem

While market simulators have proven eminently useful for simulating buyer behavior, one of the most common simulation models (the Logit or Share of Preference model) has displayed a problematic result as characterized by the oft-cited Red-Bus/Blue-Bus problem. The underlying property leading to this problem is termed IIA, which is shorthand for "Independence from Irrelevant Alternatives." The basic idea of IIA is that the ratio of any two products' shares should be independent of all other products. This sounds like a good thing, and at first, IIA was regarded as a beneficial property. However, another way to say the same thing is that an improved product gains share from all other products in proportion to their shares; and when a product loses share, it loses to others in proportion to their shares. Stated that way, it is easy to see that IIA implies an unrealistically simple model. In the real world, products compete unequally with one another, and when an existing product is improved, it usually gains most from a subset of products with which it competes most directly. Imagine a transportation market with two products, cars and red busses, each having a market share of 50%. Suppose we add a second bus, colored blue. An IIA simulator would predict that the blue bus would take share equally from the car and red bus, so that the total bus share would become 67%. But it's clearly more reasonable to expect that the blue bus would take share mostly from the red bus, and that total bus share would remain close to 50%. It is important to note that some degree of IIA is appropriate and useful within market simulations. In many markets, there is some degree of randomness to buyer behavior. It is not that people are irrational, but that buyers must balance the costs of making a utility maximizing decision against the costs of taking the time to make perfect decisions. It is quite reasonable for rational buyers to make what on the surface may seem as haphazard decisions especially for low-involvement purchases. A similar or even duplicate offering could thus be expected to capture more share in the real world than a rational simulation model might suggest. In general, market simulation models based on disaggregate models of preference (utilities estimated at the individual level) are more immune to IIA difficulties than aggregate models of preference (aggregate logit, as offered by our CBC System). In addition to modeling respondent preferences at the individual level, there are market simulation methods that help deal with IIA. These are described in the next sections.

758

SSI Web v7

8.4.4

Market Simulator Models

The Market Simulator offers five models: 1. 2. 3. 4. 5. First Choice Share of Preference Share of Preference with Correction for Similarity Purchase Likelihood Randomized First Choice

This chapter provides a brief introduction to the models used in SMRT's market simulator. More detail is provided in the section entitled: "Technical Details for Simulations."

First Choice
This option is the simplest and is sometimes referred to as the "Maximum Utility Rule." It assumes the respondent chooses the product with the highest overall utility. The results for this option are invariant over many kinds of rescalings of the utilities. In particular, one could add any constant to all the levels for an attribute and/or multiply all part-worth utilities by any positive constant without affecting the shares for the simulated products. The First Choice model requires individual-level utilities, such as those generated by ACA, CVA, ACBC, or CBC/HB. The First Choice model cannot be used with Latent Class or Logit runs for CBC. The First Choice model is very intuitive and simple to implement. Its principal strength is its immunity to IIA difficulties (red-bus/blue-bus problem). In other words, the First Choice rule does not artificially inflate share for similar (or identical products). This property is especially important for product line simulations or situations in which some product offerings are quite similar to others in the competitive set. Its principal weakness is that the share of preference results are generally more extreme than the other simulation models and one cannot adjust the steepness of the model using the exponent multiplier. We have seen evidence that the First Choice model's predictions can often be more extreme (especially when using CVA or ACA utilities) than market shares in the real world especially for low involvement purchases. Another weakness is that it reflects information only about the respondent's first choice. Information about the relative preference for the remaining products in the simulation is lost. As a result, standard errors for the First Choice model are generally higher than with the other models offered in the Market Simulator. Sample sizes need to be larger for First Choice modeling than the other approaches to achieve equal precision of estimates. We recommend using the First Choice model with ACA or CVA utilities if you have large sample sizes and have determined through holdout choice validation or, preferably, through validation versus actual market choices that the First Choice model accurately predicts shares better than the other approaches.

Share of Preference Models

The Share of Preference models (both with and without correction for product similarity) use the logit rule for estimating shares. The product utilities are exponentiated and shares are normalized to sum to 100%. The Share of Preference models result in "flatter" scaling of share predictions than the First Choice

SMRT (Market Simulator) Help

759

model. In general, we expect that this flatter scaling more closely matches what occurs in the real world. The Share of Preference models capture more information about each respondent's preferences for products than the First Choice method. Not only do we learn what product is preferred, but we learn the relative desirability of the remaining products. This means that standard errors of share predictions are lower than the First Choice shares. The Share of Preference model (without correction for product similarity) is subject to IIA, and can perform poorly when very similar products are placed in competitive scenarios (e.g. line extension simulations) relative to other less similar items within the same set. If using CBC under aggregate logit simulations, the IIA problem is intensified. Under Latent class, the problem is somewhat reduced. With individual-level utility models (ACA, CVA, ACBC, or CBC/HB), the problem is greatly reduced, but nonetheless can still be an issue. The Share of Preference with Correction for Product Similarity model can result in more valid predictions when the competitive set includes products that have significant differences in similarities. However, this model is not as theoretically complete as the Randomized First Choice method and can give unexpected results, especially when conducting sensitivity simulations. The Randomized First Choice method has been shown to handle product similarity issues in conjoint simulations better. For this reason, we generally don't suggest using the Share of Preference with Correction for Product Similarity. It remains an option in the Sawtooth Software simulator mainly for historical purposes.

Purchase Likelihood Model

The purchase likelihood model estimates the stated purchase likelihood for products you specify in the simulator, where each product is considered independently. The likelihood of purchase projection is given on a 0 to 100 scale. If you intend to use the Likelihood of Purchase option in the Market Simulator, your data must be appropriately scaled. The following estimation methods result in data appropriate for the purchase likelihood option: 1. ACA, if calibration concepts have been asked and used in utility estimation. 2. CVA, if single-concept presentation was used, and the logit rescaling option used with OLS regression. 3. CBC/HB, if calibration concepts have been asked and the CALIB program used to rescale the utilities. Any other procedure will result in simulations that are not an accurate prediction of stated purchase likelihood. Also keep in mind that the results from the Purchase Likelihood model are only as accurate as respondents' ability to predict their own purchase likelihoods for conjoint profiles. Experience has shown that respondents on average exaggerate their own purchase likelihood. You may use the Purchase Likelihood model even if you didn't scale the data using calibration concepts, but the results must only be interpreted as a relative desirability index. Meaning: a value of "80" is higher (more desirable) than a value of "60," but it doesn't mean that respondents on average would have provided an 80% self-reported likelihood of purchase for that particular product. The purchase likelihoods that the model produces are not to be interpreted literally: They are meant to serve as a gauge or "barometer" for purchase intent. Under the appropriate conditions and discount adjustments (calibration), stated intentions often translate into reasonable estimates of market acceptance for new products.

760

SSI Web v7

Randomized First Choice

The Randomized First Choice (RFC) method combines many of the desirable elements of the First Choice and Share of Preference models. As the name implies, the method is based on the First Choice rule, and can be made to be immune to IIA difficulties. As with the Share of Preference model, the overall scaling (flatness or steepness) of the shares of preference can be tuned with the Exponent. Most of the theory and mathematics behind the RFC model are nothing new. However, to the best of our knowledge, those principles have never been synthesized into a generalized conjoint/choice market simulation model. RFC, suggested by Orme (1998) and later refined by Huber, Orme and Miller (1999), was shown to outperform all other Sawtooth Software simulation models in predicting holdout choice shares for a data set they examined. The holdout choice sets for that study were designed specifically to include product concepts that differed greatly in terms of similarity within each set. Rather than use the part-worth utilities as point estimates of preference, RFC recognizes that there is some degree of error around these points. The RFC model adds unique random error (variation) to the part-worth utilities and computes shares of choice in the same manner as the First Choice method. Each respondent is sampled many times to stabilize the share estimates. The RFC model results in a correction for product similarity due to correlated sums of errors among products defined on many of the same attributes. The RFC model is very computationally intensive, but with today's fast computers speed is not much of an issue. With the suggested minimum of 100,000 total sampling iterations for a conjoint data set, it takes only a few moments longer than the faster methods to perform a single simulation. According to the evidence gathered so far on this model, we think it is worth the wait. The RFC model is appropriate for all types of conjoint simulations, based on either aggregate- or individual-level utilities. The most complete use of the RFC model requires tuning the appropriate amount of attribute- and product-level error. By default, only attribute-level error is used in the simulator. This setting assumes no product share inflation for identical offerings. If you have questions regarding tuning the RFC model read the section covering the details of RFC or read the technical paper entitled "Dealing with Product Similarity in Choice Simulations," available for downloading from our home page: http://www.sawtoothsoftware.com. Note: By default, a correction for similarity (correlated attribute error) is applied to all attributes; but starting in SMRT v4.14, the user can specify that certain attributes should not involve a correction for similarity. We recommend you remove the correction for similarity for any Price attribute. You do that under the Method Settings... button on the Scenario Specification dialog.

SMRT (Market Simulator) Help

761

8.4.5

External Effects
The factors evaluated in a conjoint analysis study usually focus on product attributes, ignoring other factors that affect market share. Consequently, the Market Simulator calculates a "preference share" rather than a "market share," emphasizing that important factors for calculating market share are missing from the model. Such missing factors include: the level and effectiveness of advertising, the size and effectiveness of the sales force, the number of outlets where the product is sold, the length of time the product has been on the market, and whether the product was first on the market. The principal value of conjoint simulators is to indicate the kinds of changes that would make the most favorable differences in customer preferences. There is no real need for them to have the appearance of market shares. All the same, it is sometimes disconcerting for those viewing results to see shares of choice that are vastly different from known market shares. The External Effect option of the Market Simulator helps to account for factors outside the model. When used properly, External Effects can lead to more realistic predictions. Even so, we recognize that the method used in our simulator to adjust for external effects is a simple approach with certain weaknesses. External Effects are introduced into the Market Simulator by applying a multiplicative External Effects factor to each product's preference share. The factor ranges from 0 to 9999. A value of 1 introduces no effect, a value greater than 1 increases a product's preference share above what the model would otherwise predict, a value less than 1 decreases the predicted preference below this value, and a value of 0 eliminates the product from the model. You set External Effect factors from the Scenario Specification dialog, by checking Apply External Effects. When the Apply External Effects box is checked, an additional column in the product specification grid appears (at the far right) in which you can type external effects. Setting the factor for each product is subjective. The following procedure minimizes the level of subjectivity: 1. Start by running a simulation on products currently on the market for which market shares are known or can be estimated. Set the External Effects factor to 1 for all products in this simulation. 2. Divide the actual market share for each product by the preference share predicted by the model. This number becomes the External Effects factor for that product. 3. Re-run the Market Simulator with these factors applied. Check that the results now reproduce the actual market shares for the products. 4. Set the External Effects factor for new products, using the External Effects factor for the current products as benchmarks. A new product should have an External Effects factor close to the ones for similar existing products (similar with respect to the "missing" factors listed above). 5. The Market Simulator produces shares that new products are expected to achieve when they have fully penetrated the market. The External Effects factor can be used to estimate shares before full penetration is achieved. One way to do this is to estimate, from past experience, the extent to which new products have reached full penetration at a time t after their introduction. When you want a simulation for time t, multiply this penetration factor by the one you have calculated using the steps above, and use this result as the External Effects factor.

762

SSI Web v7 For example, suppose you are a car manufacturer and you are introducing a new sedan. And suppose from your past sales you know the following penetration rate (fraction of expected monthly sales) for a sedan:
Time Market Penetration Months After Introduction 0 3 6 9 12 15 18 0.05 0.20 0.50 0.80 0.90 0.95 1.00

Then, if the External Effects factor at full market penetration is expected to be 1.2, the External Effects factor used as a function of time t would be:
Time Market Penetration Months After Introduction 0 3 6 9 12 15 18 0.06 0.24 0.60 0.96 1.08 1.14 1.20

As a final note, if you do need to apply external effect factors, we suggest you first investigate tuning the Exponent to best fit target shares prior to invoking external effect adjustments.

SMRT (Market Simulator) Help

763

8.4.6

Interpolating between Levels

Sometimes you may want to simulate a product that has a level in between two levels you measured. This is called interpolation, and it is usually a reasonable thing to do when dealing with attributes that are quantitative in nature (price, speed, etc.). The market simulator uses a straight-line interpolation between adjacent part-worths. Consider the following attribute:
Level # 1 2 3 4 Level Text 5 pages per 10 pages per 15 pages per 20 pages per minute minute minute minute

The simulator lets you specify a level code between two levels you measured. For instance, level 2.5 represents the point half-way between 10 and 15 pages per minute, or 12.5 pages per minute. Twelve pages per minute represents a value 40% of the way between 10 and 15 pages per minute. Therefore, level 2.4 corresponds to 12 pages per minute. But there is a much easier way to interpolate between levels. You may find it more convenient to refer to levels of quantitative attributes in the Market Simulator using values other than the original level numbers. When you click Assign Level Values, you can assign values to represent levels, such as:
Level # 1 2 3 4 Level Value 5 10 15 20 Level Text 5 pages per 10 pages per 15 pages per 20 pages per minute minute minute minute

It is much easier to specify products in the market simulator after recoding level numbers to values that match the pages per minute. To simulate a product at 12 pages per minute, we would now specify "12" for speed. Three additional points are worth mentioning regarding recoding attribute levels for use in the simulator: 1. Interpolation only works correctly if all values are either ascending or descending for an attribute. 2. If you want to omit an attribute from simulations, you must omit it for all products in the simulation. You omit a level by specifying "N/A" (you type these characters) instead of a numeric code. 3. If linear terms (coefficients) were estimated instead of part-worths for a quantitative function such as price, you should make sure not to change the level values from those that your consultant provided. For example, if the coded values for a linear price term were -2, 0 and +2 (corresponding to zero-coded values representing $2, $4, and $6), it would not be appropriate to change those to 200, 400, 600, as a utility effect 100 times the appropriate effect would be applied in simulations. The Market Simulator zero-centers (but doesn't standardize) all coded values for linear terms before multiplying by the utility coefficients. Thus, values under Assign Level Values... of 2, 4, 6 would be applied the same as 100, 102, 104 or 52, 54, 56. After zero-centering, these three different coding schemes would be applied as -2, 0, +2 in simulations. For more information about how the Market Simulator interpolates between levels when interaction terms are involved, please see the Appendix entitled: " How the Simulator Interpolates between Levels."

764

SSI Web v7

8.4.7

The "None" Weight

If you are conducting simulations for a CBC study and the questionnaire contains choice tasks including a "None" option, then the option of None can be included in simulations. However, the share of respondents predicted to choose None will only be correct if the number of products in the simulation is the same as the number of products in the CBC questionnaire. This is another problematic outcome of the IIA rule and especially affects simulators built upon aggregate logit runs. With individual-level modeling under CBC/HB, we have seen that the share of "None" is approximately correct irrespective of the number of products used in the simulator (within a reasonable range of product alternatives). The market simulator lets you specify a "None" weight from Scenario Specification dialog by clicking the Advanced Settings... button. By default, the None weight is set to "0," which means that we do not report a None percentage and we assume that all respondents are in the market and must "buy" a product. If you are using an aggregate logit run in your market simulations and are using a different number of products than were reflected in your CBC questionnaire, you may need to consider the suggestions below to deal with the problems of IIA and the "None" weight. If you are using individual-level utilities from CBC/HB, then you these considerations are less an issue for you.

None Calibration for Aggregate Logit Models

Logit (Share of Preference) models tend to give too much share to products that are similar to one another, and to penalize products that are unique (this is especially the case with aggregate logit solutions). The None option does not have specified levels on any of the conjoint attributes, and is therefore unique as compared to the products in the simulation. If you do a simulation with a few products plus None, and then try another that includes those same products plus others, you will find that the share predicted for None will be smaller when there are more products. That may not be an accurate reflection of reality. If the respondents who chose None did so because they would never buy a product in that category, then it would clearly be incorrect to assume that fewer would choose none just because they are offered an array of more products. On the other hand, if respondents are really candidates for those products, then one would expect the share choosing None to decrease when respondents are offered a richer set of choices. We know of no way to tell how the share choosing None should vary as the number of products in the simulation changes, although we think some allowance should be made in those cases where more products are in the simulation than in the original choice tasks. For that reason, we provide the user with the capability of adjusting the percentage choosing None, through multiplication by a constant supplied by the user. In general, we think the None weight should be somewhere between unity and the fraction n/t, where n is the number of products in the simulation and t is the number of products in the average choice task. With individual-level utilities, less (or no) adjustment for the None may be needed. With aggregate utilities, more adjustment may be appropriate.

SMRT (Market Simulator) Help

765

8.5
8.5.1

Using the Market Simulator

The Market Simulator Dialog
Introduction The previous two sections have dealt mainly with theory. This section discusses how to use the software to simulate market choices after you have computed part-worth utilities or have imported utilities from another system. We should note that context-sensitive on-line help for all screens and dialogs within the software is available by pressing F1. You can access general help within SMRT by clicking Help | Help Topics.

The Market Simulator Dialog

To start SMRT, click Start | Programs | Sawtooth Software | Sawtooth Software SMRT. The first step in running SMRT's Market Simulator is to open a study. (We'll assume you have already created a study by importing part-worth data into SMRT, as described earlier.) To open an existing study, select File | Open, and browse to the directory containing the studyname.SMT file. Double-click the studyname.SMT file. The main program menu is displayed. To open the market simulator, click Analysis | Market Simulator. The Market Simulator dialog is displayed:

766

SSI Web v7 This dialog contains two main lists: the list of available Utility Runs (which you cannot modify) and the list of Simulation Scenarios (which you can add to and modify). A description of each of the settings and buttons is provided below: Banner If you are using part-worth utilities from a method other than logit, you can specify the banner variable to use in reporting the results of the simulation. For example, if you want to see the results split out between large and small business, you can choose that variable as the banner point. A utility run contains a set of part-worths utilities estimated in a particular way for a given number of respondents. For example, perhaps you imported part-worth data from two separate HB runs for use in the Market Simulator. The first set involved main effects only and the second set additionally involved some interaction effects. When you imported these two separate data sets, you provided a label for each utility run by which it is referenced and selected during simulation analysis. For example, you may have referred to the first run as "Main Effects Only" and the second run as "With Interactions." By default, the first utility run on the list is highlighted. Highlight the utility run you want to use during simulations by clicking with the mouse. Only one utility run can be selected at a time. Simulation Scenarios When you first set up a study, the list of Simulation Scenarios is blank. To specify a new scenario, click Add. To choose an existing scenario, check the scenario box with the mouse. You can check more than one simulation scenario at a time. If you choose multiple scenarios, all of the simulations will be conducted at the same time (in batch mode). To edit an existing scenario, highlight it on the list and click Edit.... To delete an existing scenario, highlight it on the list and click Delete. Output Precision This setting controls how many decimal places of precision are used when reporting results.

Utility Runs

Assign Level Values This button lets you define the numeric code to be applied to different levels, such as for levels of price, speed or quantity. Assigning level codes can make it easier to define products in simulation scenarios and interpolate between measured levels. Compute! Click this to perform a simulation for the selected Utility run and Simulation Scenarios. The simulation results are displayed in the report window. If no products have been specified in a Simulation Scenario, average part-worth utilities and importances are displayed. Click this to clear the results from the report window. Prints the results displayed in the report window. Saves the results displayed in the report window to a text file. Exits the Market Simulator dialog.

Clear Print Save As Close

SMRT (Market Simulator) Help

767

8.5.2

Adding a Simulation Scenario

Usually the first step in using the market simulator is to define a "Base Case" scenario. A base case typically reflects a current (or future) market scenario: your brand vs. the relevant competition. If there is no relevant competition, or your conjoint study was designed to model only your product, the base case may be a single product. If you have defined a base case scenario or other scenarios, when you open the Market Simulator dialog, these will appear in the list of Simulation Scenarios. You can view or edit an existing simulation scenario by highlighting that scenario and clicking Edit. To add a new scenario, click Add. If a scenario or scenarios have already been defined, you are next asked whether the new scenario should be a copy of an existing scenario (used as a starting point, so that you don't have to re-enter common information again) or whether you want to start with a blank template. The Scenario Specification dialog is then displayed:

The cursor is initially active in the Name: field. You must provide a name to identify the scenario. This name will appear in the Simulation Scenarios list in the Market Simulator dialog. The Market Simulator lets you input the market scenario in a grid format, where the products are rows (you can have up to 100), and the attributes are columns (you can have up to 30). For example, you may have in mind to specify the following competitive scenario for a study that has three total attributes with three levels each:

768

SSI Web v7

Product Product A Product B Product C

Attribute 1 (Brand) 1 2 3

Attribute 2 (Performance) 2 1 3

Attribute 3 (Price) 3 2 1

If you started with a blank template, only one line in the grid is displayed. A default label of "Product 1" is provided, which you can edit by clicking this field with the mouse. To specify the attribute levels that make up a product, click each cell within the grid and specify a level code. The attribute level labels and level codes are displayed automatically in the Attribute Level View window as an aid. You must provide a level value or an "N/A" for each cell in the grid under the attribute columns. After you have fully defined the first product (row) of the grid, you are ready to add another product. You can add and delete products from the simulation scenario using the Insert Product and Delete Product buttons. Insert Product The products in the market scenario are specified in the grid central to this dialog. Each row of the grid represents a product concept. When you click Insert Product, a new row is added to the grid. You define a product concept by typing in level values directly into the grid for the attributes describing that product. Deletes the highlighted product concept. Saves any modifications you've made to the current dialog and returns you to the main market simulation dialog. Returns you to the main market simulation dialog without saving any changes.

Delete Product OK Cancel

SMRT (Market Simulator) Help

769

8.5.3

Other Controls in Simulation Scenarios

Besides formulating the different competing products in the simulation scenario, there are a number of other important decisions to be made. You must decide on the simulation method (the mathematical model that converts respondent preferences into simulated shares of choice or purchase likelihood estimates), respondent settings (if using individual-level data), the scaling of shares and output display options. Simulation Method Select the method used for simulations for the current scenario. The method is Randomized First Choice by default. (See the section entitled, " Market Simulator Models" for more details about the different simulation methods.) The default mode is Simulation. You can specify Sensitivity mode, which is a way of processing in batch multiple simulation runs that are variations of the current scenario. For example, you may wish to see how varying all levels of price for Product 1 affects its share relative to a fixed set of competitors. Rather than run the same simulation over and over again for all levels of price for Product 1, you can specify Sensitivity mode, and choose Product 1 as the Sensitivity Product, and Price as the Sensitivity Attribute.

Operating Mode

Respondents to Include If you are using part-worth utilities from a method other than logit, you can specify the respondents to include in the simulation. If you are using aggregate utilities (from Logit), this option is not available. Respondent Weights If you are using individual-level utilities (from ACA, CVA, ACBC or CBC/HB), you can specify the respondent weights to be applied in the simulation. If you are using aggregate utilities (from Logit or Latent Class), this option is not available. In some situations, you may want to adjust the share given to a particular product by some external factor. For example, perhaps that product is only available in half of the markets relative to the other products. In that case, you can check the Apply External Effects box. When this box is chosen, a new column appears in the product specification grid labeled "External Effect." All external effects are initially set to unity (1). To adjust the shares to account for a given product only being available in half of the markets, you might specify an external effect of 0.5 for that product, and leave all other products with external effects of unity (1). When you click this button, a dialog appears that lets you define additional settings: If you are using individual-level utilities generated by ACA, CVA, ACBC or CBC/HB, a measure of fit is provided indicating the relative consistency of each respondent's utilities. You can select a value used as a cutoff for excluding respondents that are at or below that threshold for consistency. The exponent tunes the overall "flatness" or "steepness" of the share results (see discussion earlier regarding the Exponent). It has effect only for the Share of Preference models and Randomized First Choice. If you are analyzing data for a CBC study that included a "None"

Apply External Effects

Advanced Settings

Correlation Cutoff

Exponent

"None" Weight

770

SSI Web v7 option, this lets you specify the weight applied to the None utility. By default, it is zero (no "None" share computed). (See discussion earlier regarding the None weight for more direction.) Unacceptable Extrapolation If you are using data from an ACA study and if you asked the "Unacceptables" section, you can tune how low the utility should be for unacceptable levels. This percentage determines the degree of extrapolation when assigning the utility of "Unacceptable" levels. The extrapolation is customized for each respondent as a projection from the worst level of each attribute. The constant subtracted from the worst level within that attribute is determined as a percentage of the average differences between best and worst levels across all attributes for that respondent. The default is 20%. To use this setting properly, you should tune the extrapolation to best fit holdout choices/observations. Please note that the Unacceptable Extrapolation you set has a direct bearing on the importance of each attribute, and also alters the part-worth utilities.

Output Options Individual Results to File If you are running simulations using part-worth utilities generated by a method other than logit, you can save the estimated shares of preference for each individual to a text-only file for analysis in another software package. This can be helpful, for instance, if you want to identify particular respondents that prefer (or do not prefer) a certain product. Display Utilities Controls whether the utilities for each attribute level (or interaction terms) are displayed in the output. The utilities displayed are rescaled by a method called Zero-Centered Diffs. The diffs method rescales utilities so that the total sum of the utility differences between the worst and best levels of each attribute across attributes (main effects) is equal to the number of attributes times 100. Note: the average part-worth attribute utilities are influenced by the number of respondents in the simulation and respondent weighting, but are not affected by the product specifications you enter. Please see an earlier section entitled, " Interpeting Conjoint Analysis Data" for more information about interpreting utilities. Display Importances Checking this option tells the Market Simulator to include a summary of attribute importances in the simulation output. The importance of an attribute is defined as its weight, or the maximum influence it can have on product choice, given the range of attribute levels defined in the study. In other words, an attribute's importance is an indicator of the amount of influence an attribute may have in simulations. Attribute importances are only accurate when individual- or segment-level utility data are available (such as computed under Latent Class, ACA, CVA, ACBC or Hierarchical Bayes). Aggregate importances from logit can be misleading if respondents disagree about the order of preference of levels within an attribute. Please see an earlier section entitled, " Interpeting Conjoint Analysis Data" for more information about interpreting importances.

SMRT (Market Simulator) Help

771

8.5.4

Running the Simulation

After you have specified the products and chosen the desired settings for your simulation scenario, you are ready to compute the results. From the Market Simulator dialog, click Compute! Depending on the model you choose and the number of respondents, it can take from a few moments to a few minutes to compute. The results are displayed in the text report window. You can scroll up and down across the report within this window. You can also cut-and-paste results into a word processor or spreadsheet package. Every time you click Compute!, results are appended to any existing data in the report window. Click Clear to delete the contents of the window.

772

SSI Web v7

8.6
8.6.1

Practical Simulation Examples

Introduction
In this section, we'll discuss how to use the market simulator to answer some common strategic questions introduced earlier: Given a current competitive environment, what product should I offer to maximize interest in my offering? What is the relative price sensitivity of different brands? If I raise my price by 10%, how will it affect my brand? What portfolio of products can I offer to appeal to different market segments and maximize overall share?

For the examples in this section, we'll assume the following three attributes, measured using three levels each: Brand: 1 BrandA 2 BrandB 3 BrandC Style: 1 StyleA 2 StyleB 3 StyleC Price: 1 $100 2 $150 3 $200 The results reported in this section are fictitious and not based on any particular data set.

SMRT (Market Simulator) Help

773

8.6.2

New Product Introductions

Let's assume that your company is interested in entering a market that currently consists of just two competitors. There are just three attributes that adequately describe the products and account for preference in the market: Brand, Style and Price. The two players are:
1) BrandA, StyleA, $100 2) BrandB, StyleB, $200

Your company has developed a new style (StyleC) that you think may appeal to buyers, and you want to investigate its potential with respect to the two existing products. The first step, typically, is to simulate the existing market scenario. You use the market simulator to define the two existing products:
Product Specifications: Product Name Brand Style "BrandA" 1 1 "BrandB" 2 2 Price 1 3

For this simulation scenario, there are no similarities (in terms of common shared attribute levels) between the products in the simulation. Therefore, the Share of Preference model is appropriate (though it would also be appropriate to use the Randomized First Choice method). When you click Compute!, the following shares of preference are displayed:
Shares of Preference for Products: BrandA 64.3 BrandB 35.7

Note that the buyers in the simulation are all assumed to choose a product, so the shares of preference across products in the simulation sum to 100%. Let's assume that you have actual market share information about these two brands. You note that the shares reported above do not necessarily match the actual market shares. You accept this, however, recognizing that many factors influence market shares in the real world that cannot be captured through conjoint analysis. You are principally interested in relative preferences, assuming that the marketplace is an equal playing field: equal distribution, awareness, effectiveness of sales force, and equilibrium long-range demand. In the second stage of this simulation example, we'll define a new scenario that includes your company's proposed product: BrandC, StyleC, $150. You add another row to the simulation specification grid:
Product Specifications: Product Name Brand Style "BrandA" 1 1 "BrandB" 2 2 "BrandC" 3 3 Price 1 3 2

After clicking Compute!, the following shares are displayed:

Shares of Preference for Products: BrandA 42.5 BrandB 21.3 BrandC 36.2

You note that BrandA is still the most preferred product, but that your brand is preferred to BrandB. Like any market research statistic computed from samples, shares of preference are not estimated without error. It is common to estimate a confidence interval, to gain a feel for the degree of

774

SSI Web v7 uncertainty due to sampling and measurement error associated with a given share of preference. If your simulator is based upon part-worth utilities generated by a method other than logit, the Market Simulator lets you add the standard errors of the shares of preference to the simulation report. Let's assume that the standard error reported for BrandC for the simulation above was 1.53. The 95% confidence interval is computed by adding plus and minus 1.96 times the standard error to the estimated share of preference. In this example, the 95% confidence interval is 36.2 plus and minus (1.96)(1.53) = 3.0 share points, or the interval [33.2, 39.2]. You next may ask yourself what price you would need to charge to capture the same relative preference as BrandA. To simulate this, you lower the price slightly for your brand. The market simulator lets you interpolate between levels, so you can investigate even the smallest of price changes. As a first step, you decide to lower the price to $130 for BrandC (while holding the specifications for BrandB and BrandA constant). The new simulated shares are:
Shares of Preference for Products: BrandA 39.2 BrandB 19.0 BrandC 41.8

You have overshot the mark (BrandC's share exceeds BrandA's share), so you try a slightly higher price than $130 and click Compute! once more. You make repeated attempts until BrandA and BrandC's shares are equal. Let's assume that after a few more attempts, you discover that the price that makes your company's offering match the share of preference of the market leader is $136. Another way of thinking about this finding is that your proposed product commands a $136 - $100 = $36 premium over BrandA's product. (Respondents are indifferent between BrandA, StyleA at $100 and BrandC, StyleC at $136).

SMRT (Market Simulator) Help

775

8.6.3

Estimating Demand Curves and Elasticities

Before outlining simulation strategies for pricing research, we should give a few notes of caution. The ACA system is often not a very good pricing research tool; CVA and particularly the CBC system are considered stronger approaches for most markets. If using CBC for pricing research, we recommend the use of Latent Class or especially individual-level computation methods, such as the CBC/HB (Hierarchical Bayes) module. For estimating demand curves using market simulations, the Share of Preference method is appropriate. Randomized First Choice may also be selected, if you believe some correction for product similarity should result if products carry the same prices (however, the use of Randomized First Choice for generating demand curves can often result in strange shifts in the curves around the "average" price, if brands are initially set at the same average price). We will build upon the previous example during this section. We previously computed shares of preference for three products defined using the following attribute level codes:
Product Specifications: Product Name Brand Style "BrandA" 1 1 "BrandB" 2 2 "BrandC" 3 3 Price 1 3 2

You may recall that price levels 1, 2 and 3 correspond with the following prices $100, $150, and $200. The shares of preference for the products as defined above were:
Shares of Preference for Products: BrandA 42.5 BrandB 21.3 BrandC 36.2

Let's assume we wanted to estimate a demand curve for your company's offering: BrandC, in the context of the current competition and prices. We do this through "sensitivity analysis." The Market Simulator offers an automatic way to conduct sensitivity analysis, but so you understand the process, we'll describe it in separate steps. Recall that we measured three distinct levels of price $100, $150 and $200. Note that we've already computed the share of preference for BrandC when it is offered at $150 (21.3). To estimate the demand curve for BrandC, we'll need to conduct two additional simulations: a simulation with BrandC at the lowest price ($100), and a simulation with BrandC at the highest price ($200). For each of these simulations, we'll hold the BrandA and BrandB product specifications constant. To estimate BrandC's share at the lowest price ($100), we use the following product specifications:
Product Specifications: Product Name Brand Style "BrandA" 1 1 "BrandB" 2 2 "BrandC" 3 3 Price 1 3 1

We click Compute!, and the following shares are reported:

Shares of Preference for Products: BrandA 33.9 BrandB 15.6 BrandC 50.5

776

SSI Web v7 We record BrandC's share (50.5), and proceed to the next step. To estimate BrandC's share at the highest price ($200), we use the following product specifications:
Product Specifications: Product Name Brand Style "BrandA" 1 1 "BrandB" 2 2 "BrandC" 3 3 Price 1 3 3

We click Compute!, and the following shares are reported: Shares of Preference for Products
BrandA BrandB BrandC 49.2 26.9 23.9

From these three separate simulation runs, we now have all the information we need to plot a demand curve for BrandC, relative to the existing competitors and prices. Assuming that BrandA and BrandB are held constant at current market prices, the relative shares of preference for BrandC at each of the price points within the measured price range are:
Price $100 $150 $200 Share of Preference 50.5 36.2 23.9

We have demonstrated how to estimate a demand curve for BrandC, relative to the existing competitors at current market prices. If the goal is to estimate demand curves for all brands in the study, the usual procedure is to record the share for a brand at each price level while holding all other brands at the average (middle) price. It is often interesting to plot these demand curves and look at the patterns of price sensitivity between brands and the different slope of the curves from one segment of the curve to the next. It is also common to want to characterize the degree of price sensitivity using a single value, referred to as an elasticity. The Price Elasticity of Demand (E) is defined as: E = %rQ %rP where, "%r" means "percent change in," Q is defined as quantity demanded, and P refers to the price. If the brand or product follows the law of supply and demand (most products do), price increases lead to decreases in quantity demanded, and the elasticity is negative. The larger the absolute value of the elasticity, the more price sensitive the market is with respect to that brand or product. Using the "midpoints" formula, we can compute the average price elasticity of demand across the demand curve for BrandC:
E = q2 q1 (q1+q2)/2 23.9 50.5 (50.5+23.9)/2 -0.715

p2 p1 (p1+p2)/2 200 100 (100+200)/2 0.667 = -1.073

SMRT (Market Simulator) Help

777

Another way to compute the average price elasticity of demand (which can be more accurate if more than two price points along the curve have been estimated) is the "log-log" regression. One takes the natural log of prices and shares and regresses the log of share on the log of price (you can do this within a spreadsheet). The resulting beta is the average price elasticity of demand. As with all conjoint simulation results, the resulting elasticities from conjoint simulators are interpreted bearing in mind some assumptions. In particular, the degree of noise within the conjoint data is particularly relevant. For example, if the respondents to the conjoint survey answered in a more haphazard way compared to buyers in the real world, the price elasticities estimated from conjoint simulations may be uniformly understated (too insensitive). Also, you should recognize that changing the Exponent will shift the elasticities. Even if this is the case, the relative price sensitivities for brands are still useful.

778

SSI Web v7

8.6.4

Designing Products to Appeal to Unique Market Segments

Customizing products to appeal to target segments or even individuals is a common theme in marketing. Many companies dedicate significant resources to developing a portfolio of products that it hopes will appeal to unique segments. For line extensions, the challenge for any company is to design new product(s) that take share from its competitors without stealing an unacceptable amount of share from products within its existing line. One common approach to designing an effective line extension is to use the conjoint data to segment the market into latent (not observed) market segments (sometimes referred to as clusters) that have similar preferences. These segments are termed "latent," because they are not simply delineated based on an explicit variable such as gender, income or company size. Rather, the underlying segments are revealed through a statistical segmentation technique such as cluster analysis or latent class modeling. Segments are formed with the goal of maximizing the differences in preference between groups while minimizing the differences in preference within groups. Once these latent segments have been identified, one can profile them in terms of other variables in the survey (i.e. demographics, usage or media habits). If simulations are based on Latent Class (for CBC data) or if you used a cluster technique on individual-level part-worths, you can conduct simulations "by segment." If using a Latent Class utility run, this happens automatically. If based on a cluster analysis, the segment membership information can be made available to you as a "banner point." For example, let's assume that a cluster analysis revealed three relatively different segments for the hypothetical example we've been using throughout this section. Let's also assume that you merged a variable containing the segment membership information into the data set as a "banner" variable called "SEGMENT." If you select SEGMENT as the banner variable and click Compute!, the market simulator displays the part-worth utilities and importances for each segment. By examining the part-worths and importances for each group, you can gain insight into the product features that might appeal to each. You also should bear in mind the size of each segment, as this represents its demand potential. Consider the following part-worth utility preferences:
Part-worth Utility Preferences Segment1 n = 128 BrandA BrandB BrandC StyleA StyleB StyleC $100 $150 $200 39 5 -44 61 -23 -38 56 7 -63 Segment2 n = 283 -51 39 12 -52 45 7 55 2 -57 Segment3 n = 216 -44 -29 73 -34 -9 43 50 6 -56

SMRT (Market Simulator) Help

779

We can study the part-worths to learn about the differences among the segments. We can also use these preferences to simulate market choices for the following market scenario:
Product Specifications: Product Name Brand Style "BrandA" 1 1 "BrandB" 2 2 "BrandC" 3 3 Price 1 3 2

Shares of Preference for Products, by Market Segment: Segment1 Segment2 Segment3 Total n = 128 n = 283 n = 216 627 BrandA BrandB BrandC 84.8 7.4 7.8 21.5 40.0 38.5 22.2 14.2 63.6 34.7 24.5 40.8

(Note that these shares do not match the shares reported for earlier examples in this section. Since these results are for illustration only, no significance should be attached to this difference.) Let's assume your company produces BrandC with StyleC at $150. Your total share of preference is 40.8%. We see from the simulation by segment that yours is the most preferred product within Segment 3, and the second-most preferred product in Segment 2. BrandA clearly dominates Segment1 (the smallest segment). Let's assume that your company was interested in offering an additional product. We could examine the table of part-worth utilities presented earlier as a first step in formulating hypotheses about what additional product might be successful. Starting in order, you may first consider Segment 1, but this segment does not seem to offer many opportunities for your brand. BrandA offering StyleA at a low price has got this relatively small segment nearly wrapped up, and this segment doesn't seem very receptive to your brand: BrandC. You next consider Segment 2, which seems to represent a better opportunity for your brand. It is a relatively large segment that prefers BrandB, but also seems receptive to BrandC. Note also that Segment 2 strongly prefers StyleB, but your company currently offers only StyleC. By offering a StyleB product, you might be able to convert some current BrandB customers from within Segment 2 to your product line. You currently dominate Segment 3 and should probably not consider designing another product to appeal to this segment, since a good deal of the possible share to be gained from a new product would be taken from your existing product within that segment. Let's simulate what happens if in addition to your current product (BrandC, StyleC, $150), you offer another product (BrandC, StyleB, $200). Note that since this simulation involves products that have similar definitions (there are two BrandC products), you should probably use a technique that adjusts for product similarities, such as Randomized First Choice.
Shares of Preference for Products, by Market Segment: Segment1 Segment2 Segment3 Total n = 128 n = 283 n = 216 627 BrandA 82.2 17.2 18.6 31.0 BrandB 7.2 32.0 11.9 20.0 BrandC (old) 6.8 27.7 47.8 30.4 BrandC (new) 3.8 23.1 21.7 18.7

The new product has somewhat cannibalized the existing product, reducing its share from 40.8 (see the previous simulation) to 30.4, but has resulted in a relative overall gain of 1 - [(30.4 + 18.7)/40.8] 1= 20% in preference.

780

SSI Web v7 For line extension simulations you conduct, the answer will likely not be so clear and the process so direct as we've shown here. You'd certainly want to investigate other product configurations to make sure you weren't overlooking even better opportunities to enhance share. You would also want to consider the cost implications of different options for line extensions. Also, you would probably want to conduct sensitivity analysis for the new product with respect to price, to determine a strategic price point (given your costs and market share goals). Viewing the preferences and shares by segment is not required in designing an effective line extension. However, viewing the separate market segments can help you more quickly recognize patterns of preference, size the different segments of the market, and thus more easily arrive at a good solution. This exercise of viewing segment-based preferences and designing products to fill heterogeneous needs is a useful approach. However, it would seem more efficient to let an automated search algorithm find an optimal product or set of products rather than to proceed manually. Sawtooth Software's SMRT Advanced Simulation Module includes automated optimization search capability.

SMRT (Market Simulator) Help

781

8.7
8.7.1

Technical Details for Simulations

First Choice Method
This option is the simplest and is often referred to as the "Maximum Utility Rule." It assumes the respondent will choose that product with the highest overall utility. All other less preferred products receive zero share for this individual. The results for the First Choice model are invariant over many kinds of rescalings of the utilities. If two or more products are exactly the same in terms of utility (and more preferred than any other product in the simulation scenario), the respondent's choice is split evenly amongst them.

782

SSI Web v7

8.7.2

Share of Preference Options

These options do not assume that the respondent always chooses the product with highest utility. Instead, they estimate probability of choosing the simulated product, arriving at a "share of preference" for the product. This is done in two steps: 1. Subject the respondent's total utilities for the product to the exponential transformation (also known as the antilog): s = exp(utility). 2. Rescale the resulting numbers so they sum to 100. Suppose two products, A and B, have total utilities 1.0 and 2.0. Then their shares of preference would be computed as follows:
product utility 1.0 2.0 exp(utility) 2.72 7.39 share of preference 26.9 73.1

A B

(For the Share of Preference with Correction for Product Similarity, shares of preference are modified by further computations.) Unlike the First Choice option, the scaling of utilities can make a big difference with the Share of Preference Options. Consider what happens if we multiply the utilities by constants of 0.1 or 10.0: For multiplier of 0.1
A B product utility 1.0 2.0 exp(utility) 1.105 1.221 share of preference 47.5 52.5

For multiplier of 10.

A B product utility 10.0 20.0 share of exp(utility) preference 22026 100.005 4.85E08 99.995

If we multiply the utilities by a small enough constant, the shares of preference can be made nearly equal. If the utilities are made small enough, every product receives an identical share of preference, irrespective of the data. If we multiply the utilities by a large enough constant, the shares of preference can be made equivalent to the First Choice model. It is apparent that scaling of the utilities can make a big difference with the Share of Preference Models. Most of Sawtooth Software's utility estimation methods result in utilities appropriate for use with Share of Preference models. We strongly suggest you include holdout (fixed) choice tasks in your conjoint questionnaires to use for calibration, or check the simulation predictions against actual market share. The Market Simulator lets you scale the utilities within the Share of Preference option at the time the simulation is done. This is accomplished by a parameter called the "exponent" that you can set when preparing for simulations. The default value of the exponent is 1. It has the same effect as the multiplier illustrated immediately above. The exponent can be used to adjust the sensitivity of the simulation results so that it more accurately reflects holdout choices, or actual market behavior. A smaller exponent causes small shares to become larger, and large shares to become smaller it

SMRT (Market Simulator) Help

783

has a "flattening" effect. In the limit, with a very small exponent every product receives the same share of preference. A large exponent causes large shares to become larger, and small shares to become smaller it has a "sharpening" effect. In the limit, a very large exponent produces results like those of the First Choice option. If you have solid external information (such as existing market share data) and have reason to expect that conjoint shares should resemble market shares, you may want to tune the exponent within simulations. If you do not have solid external information, you probably should not change the exponent from the default value of 1.

Share of Preference Model with Correction for Product Similarity

Share of preference models that do not consider similarities among products have a serious problem: if an identical product is entered into the simulation twice, it can receive up to twice as much total share of preference as it would when entered only once. Although no researcher would make the mistake of entering the same product twice, the principle is still troublesome. Products differ in similarity to one another, and "plain" share of preference models tend to give too little share to relatively unique products. The third choice model in the Market Simulator includes a correction to prevent the preference shares of similar products from being overstated. The correction is based on each product's total similarity with other products. The basic idea behind this correction was suggested by Richard D. Smallwood of Applied Decision Analysis, although we are responsible for the details of its implementation. The procedure is: 1. For the n products in a simulation, an n x n similarity matrix is computed, with 1's indicating complete similarity, 0's indicating total lack of similarity, and fractional values for differing degrees of similarity: First a "dissimilarity" matrix is computed. Consider a scale for each attribute where its levels are coded 10, 20, 30, and so on. The dissimilarity of a pair of products on an attribute is taken as the absolute difference between their codes, but with a maximum of 10. (This allows "continuous" and "categorical" attributes to be treated in the same way.) The total dissimilarity between two products is the sum of their dissimilarities over all attributes. Two products differing by an entire level on each attribute are maximally dissimilar. Next, total dissimilarities are rescaled by a constant so the maximum possible is 3.0, rather than 10 times the number of attributes. Dissimilarities are then converted to similarities by a negative exponential transformation. At this point the minimum possible similarity is exp(-3) ~ .05, achieved if two products have completely different levels on every attribute, and the maximum possible similarity is exp(0) = 1, achieved only if they have identical levels on all attributes. Finally, the similarities are subjected to a further rescaling that sets the minimum to 0 and the maximum to 1. 2. Column totals of the similarity matrix are calculated to get a vector of "total similarities." The smallest possible value is 1.0, which is found if a product is maximally dissimilar to all others and similar only to itself. If two products are identical but maximally dissimilar to all others, those products each have values of 2.0.

784

SSI Web v7 3. For each respondent, shares of preference are divided by corresponding "total similarities," and then renormalized to have sum of unity. This has the effect of reducing shares for products that are relatively similar to others, and increasing shares for products that are relatively unique. In the limit, where two or more products are identical and totally unlike any others, they divide among themselves the share that each product would have if the others were not present.

SMRT (Market Simulator) Help

785

8.7.3

Purchase Likelihood Option

The purchase likelihood option estimates the stated purchase likelihood for products you specify in the simulator. Each product is considered independently. If you intend to use the Likelihood of Purchase option in the Market Simulator, your data must be appropriately scaled. The following estimation methods result in data appropriate for the purchase likelihood option: 1. ACA, if calibration concepts have been asked. 2. CVA, if single-concept presentation was used, and the logit rescaling option used with OLS regression. 3. CBC/HB, if calibration concepts have been asked and the CALIB program used to rescale the utilities. Any other procedure will result in simulations that are not an accurate prediction of stated purchase likelihood. Also keep in mind that the results from the Purchase Likelihood model are only as accurate as respondents' ability to predict their own purchase likelihoods for conjoint profiles. Experience has shown that respondents on average exaggerate their own purchase likelihood. You may use the Purchase Likelihood model even if you didn't scale the data using calibration concepts, but the results must only be interpreted as a relative desirability index. The purchase likelihoods that the model produces are not to be interpreted literally: They are meant to serve as a gauge or "barometer" for purchase intent. This model provides a means of simulating a product category with only a single product. The other three choice models are comparative models that require at least two products to be specified in a simulation. Likelihoods are estimated for product concepts by summing scaled utilities and estimating probabilities with the following transformation: p = eu _____ 1 + eu where, p = probability of purchase e = the constant e u = product utility

786

SSI Web v7

8.7.4

Randomized First Choice

The Randomized First Choice (RFC) method combines many of the desirable elements of the First Choice and Share of Preference models. As the name implies, the method is based on the First Choice rule, and can be made to be immune to IIA difficulties. As with the Share of Preference model, the overall scaling (flatness or steepness) of the shares of preference can be tuned. Most of the theory and mathematics behind the RFC model are nothing new. However, to the best of our knowledge, those principles have never been synthesized into a generalized conjoint/choice market simulation model. RFC, suggested by Orme (1998) and later refined by Huber, Orme and Miller (1999), was shown to outperform all other Sawtooth Software simulation models in predicting holdout choice shares for a data set they examined. The holdout choice sets for that study were designed specifically to include product concepts that differed greatly in terms of similarity within each set. Rather than use the utilities as point estimates of preference, RFC recognizes that there is some degree of error around these points. The RFC model adds unique random error (variability) to the utilities and computes shares of preference in the same manner as the First Choice method. Each respondent is sampled many times to stabilize the share estimates. The RFC model results in a correction for product similarity due to correlated sums of errors among products defined on many of the same attributes. To illustrate RFC and how correlated errors added to product utilities can adjust for product similarity, consider the following example: Assume two products: A and B. Further assume that A and B are unique. Consider the following product utilities for a given respondent:
A B Avg. Product Utilities 10 30

If we conduct a first choice simulation, product B captures 100% of the share:

A B Avg. Product Utilities 10 30 Share of Choice 0% 100%

However, let's assume that random forces can come to bear on the decision for this respondent. Perhaps he is in a hurry one day and doesn't take the time to make the decision that optimizes his utility. Or, perhaps product B is temporarily out-of-stock. Many random factors in the real world can keep our respondent from always choosing B. We can simulate those random forces by adding random values to A and B. If we choose large enough random numbers so that it becomes possible for A to be sometimes chosen over B, and simulate this respondent's choice a great many times (choosing new random numbers for each choice iteration), we might observe a distribution of choices as follows:
A B Avg. Product Utilities 10 30 Share of Choice 25.0% 75.0%

(Note: the simulation results in this section are for illustration, to provide an intuitive example of RFC modeling. We assume shares of preference are proportional to product utilities.)

SMRT (Market Simulator) Help

787

Next, assume that we add a new product to the mix (A'), identical in every way to A. We again add random variability to the product utilities so that it is possible for A and A' to be sometimes chosen over B, given repeated simulations of product choice for our given respondent. We might observe shares of preference for the three-product scenario as follows:
A A' B Avg. Product Utilities 10 10 30 Share of Choice 20.0% 20.0% (A + A' = 40.0%) 60.0%

Because unique (uncorrelated) random values are added to each product, A and A' have a much greater chance of being preferred to B than either one alone would have had. (When a low random error value is added to A, A' often compensates with a high random error value). As a simple analogy, you are more likely to win the lottery with two tickets than with one. Given what we know about consumer behavior, it doesn't make sense that A alone captures 25.0% of the market, but that adding an identical product to the competitive scenario should increase the net share for A and A' from 25.0% to 40.0% (the classic Red Bus/Blue Bus problem). It doesn't seem right that the identical products A and A' should compete as strongly with one another as with B. If, rather than adding uncorrelated random error to A and A' within each choice iteration, we add the same (correlated) error term to both A and A', but add a unique (uncorrelated) error term to B, the shares computed under the first choice rule would be as follows:
A A' B Avg. Product Utilities 10 10 30 Share of Choice 12.5% 12.5% (A + A' = 25.0%) 75.0%

(We have randomly broken the ties between A and A' when accumulating shares of choice). Since the same random value is added to both A and A' in each repeated simulation of purchase choice, A and A' have less opportunity of being chosen over B as compared to the previous case when each received a unique error component (i.e. one lottery ticket vs. two). The final utility (utility estimate plus error) for A and A' is always identical within each repeated first choice simulation, and the inclusion of an identical copy of A therefore has no impact on the simulation result. The correlated error terms added to the product utilities have resulted in a correction for product similarity. Let's assume that each of the products in this example was described by five attributes. Consider two new products (C and C') that are not identical, but are very similardefined in the same way on four out of five attributes. If we add random variability to the part-worths (at the attribute level), four-fifths of the accumulated error between C and C' is the same, and only one-fifth is unique. Those two products in an RFC simulation model would compete very strongly against one another relative to other less similar products included in the same simulation. When C received a particularly large positive error term added to its utility, chances are very good that C' would also have received a large positive error term (since four-fifths of the error is identical) and large overall utility.

788

SSI Web v7

RFC Model Defined

We can add random variability at both the attribute and product level to simulate any similarity correction between the IIA model and a model that splits shares for identical products: Ui = Xi ( + Eattribute) + Eproduct where: Ui = Xi = = Eattribute Eproduct Utility of product i for an individual or homogenous segment at a moment in time Row of design matrix associated with product i Vector of part-worths = Variability added to the part-worths (same for all products) = Variability (Gumbel) added to product i (unique for each product)

Repeated draws are made to achieve stability in share estimates, computed under the First Choice rule. In RFC, the more variability added to the part-worths, the flatter the simulations become. The less variability added to part-worths, the more steep the simulations become. Under every possible amount of attribute variability (and no product variability), shares are split exactly for identical products, resulting in no "inflation" of net share. However, there may be many market scenarios in which some share inflation is justified for similar products. A second unique variability term (distributed as Gumbel) added to each product utility sum can tune the amount of share inflation, and also has an impact on the flatness or steepness of the overall share results. It can be shown that adding only product variability (distributed as Gumbel) within the RFC model is identical to the familiar logit model (Share of Preference Model). Therefore, any degree of scaling or pattern of correction for product similarity ranging between the First Choice model and Share of Preference can be specified with an RFC model by tuning the relative contribution of the attribute and product variability. The exponent also can play a role in RFC, similar, but not identical to, product variability. Decreasing the exponent (multiplying the utility estimates by a value less than unity) decreases the variance of the utility estimates relative to the variance of the random variation added within RFC simulations, in turn making simulated shares flatter. There is a subtle difference between increasing product variability and lowering the exponent, though both result in a flattening of shares. If only attribute variation is being used in an RFC simulation, decreasing the exponent flattens the shares, but the overall model still does not reflect the IIA property. Adding product variability, however, flattens the shares and causes the RFC model to reflect at least some degree of IIA behavior. Though the exponent is not required to simulate different patterns of correction for product similarity and scaling, in the next section we show that it is useful to retain the exponent adjustment from an operational point of view. The RFC model is very computationally intensive. With the suggested minimum of 100,000 total sampling iterations for a conjoint data set, the results tend to be fairly precise. But, if you have dozens of products in the simulation scenario, some product shares can become quite small, and greater precision would be needed. You can increase the precision by increasing the number of sampling iterations. The RFC model is appropriate for all types of conjoint simulations, based on either aggregate- or individual-level utilities. It provides the greatest benefits in the case of aggregate (logit and Latent Class) models, which are more susceptible to IIA difficulties than individual-level models. If you plan to use the RFC model with individual-level utilities to compute reasonably stable estimates of share at the individual level, you should sample each respondent at least 5,000 times, and preferably more. This can take a good deal of computing time. The software asks how many total sampling iterations to use, where the number refers to the total iterations across all respondents. If you have 500 respondents and want each respondent sampled 5,000 times, you should specify a total of 2,500,000 iterations. The greatest complexity of the RFC model from an operational point of view is that the magnitude of the attribute variability multiplier must be adjusted whenever the number of products or number of

SMRT (Market Simulator) Help

789

attributes on which products differ changes across simulations to maintain comparable scaling of shares of preference. Our implementation of the RFC model includes an auto-calibrating attribute variability multiplier (developed through Monte-Carlo simulations) so that this issue is transparent to the user. Ideally, you will provide your own validation data, in the form of holdout concepts or actual market shares, to permit calibration of the attribute and product variability terms. This allows you to best fit the scaling of shares and degree of correction for product similarity appropriate for the specific market you are modeling. If you are using the auto-calibrating attribute variability multiplier, you can adjust the relative contribution of attribute and product variability by manipulating only the exponent and the product variability multiplier. For example, if you want to decrease the correction for product similarity by adding some product variability and adding less attribute-level variability to the utilities, you could increase the product variability multiplier (say, from 0.0 to 0.2). After doing so, the resulting shares of preference would be flatter than before the adjustment. (The more variation added to utilities, the flatter the resulting shares.) To "sharpen" the shares again, you would adjust the exponent upward. To adjust the amount of correction for similarity, you should have outside information about choices for holdout choice tasks, or actual market share, where the product concepts involved had significantly different degrees of product similarity within each set. For more information about the RFC model, please refer to Huber, Orme and Miller's paper entitled, "Dealing with Product Similarity in Choice Simulations," available for downloading from our home page: http://www.sawtoothsoftware.com.

790

SSI Web v7

8.8
8.8.1

CBC Analysis: Counts and Logit

Counting Analysis for CBC
Overview
CBC data can be analyzed in a number of ways. Counting analysis is probably the most simple and intuitive method. It calculates a proportion for each level, based on how many times a concept including that level is chosen, divided by the number of times a concept including that level occurred. This is done automatically for each main effect and for each joint effect (between either two or three attributes). This simple analysis method may be adequate for some studies, but we generally recommend more sophisticated approaches, especially if the sorts of questions that need to be answered can only be approached through the use of a market simulator.

Importing CBC Data

Prior to conducting this analysis, you must import the data from your CBC study. We'll assume that you have already downloaded the data from the Web server, and prepared the .CHO and .ATT files using the SSI Web system (by clicking Analysis | Prepare .CHO and .ATT Files for Analysis). To import the CBC data for use within the CBC Analysis Module and Market Simulator (SMRT): 1. Open the SMRT software by clicking Start | Programs | Sawtooth Software | Sawtooth Software SMRT. 2. Create a new study, by clicking File | New and providing a study name in the desired directory. 3. Import your CBC data by clicking File | Import, specifying Choice Data (*.cho) as the Import Type, and clicking the Import button.

Counting Choices
Counts provides quick and automatic estimation of the main effects and joint effects for collected CBC data. It calculates a proportion for each level, based on how many times a concept including that level is chosen, divided by the number of times a concept including that level occurred. When you run Counts, processing begins immediately and the results are displayed in the report window. Every time you click Compute!, a new report is appended below the previous, and the report is automatically scrolled to display the most recent run. You can clear the report window by clicking Clear. You can weight the data, select subsets of respondents or tasks to process, or include banner points. Following are data for a sample of 100 respondents, each of whom answered 8 randomized choice tasks, each including 4 concepts and a "None" option. The data are real, but the attribute levels have been disguised. Main effects and two-way interactions are included in the report.
Brand Total Respondents Brand A Brand B Brand C Brand D Total 100 0.387 0.207 0.173 0.165

Within Att. Chi-Square 148.02 D.F. 3 Significance p < .01

SMRT (Market Simulator) Help

Shape Shape 1 Shape 2 Shape 3

791

0.259 0.245 0.195

Within Att. Chi-Square 13.45 D.F. 2 Significance: p < .01 Size Large Size Medium Size Small Size

0.263 0.240 0.196

Within Att. Chi-Square 13.64 D.F. 2 Significance: p < .01 Price Price Price Price Price

1 2 3 4

0.132 0.175 0.254 0.372

Within Att. Chi-Square 151.32 D.F. 3 Significance: p < .01 None None chosen: Brand x Shape Brand A Brand A Brand A Brand B Brand B Brand B Brand C Brand C Brand C Brand D Brand D Brand D Shape Shape Shape Shape Shape Shape Shape Shape Shape Shape Shape Shape 1 2 3 1 2 3 1 2 3 1 2 3

0.068 0.415 0.412 0.337 0.232 0.223 0.168 0.201 0.181 0.138 0.188 0.171 0.135 0.82 6 not sig

Within Att. Chi-Square D.F. Significance:

(Due to space considerations, we have not shown the remainder of the report. Five additional tables would follow specifying the remaining two-way joint effects: Brand x Size; Brand x Price; Shape x Size; Shape x Price; Size x Price.) Each of the main effects is the proportion of times when a concept containing that attribute level occurs that the concept is selected by respondents. Brand A was the most popular, having been selected 38.7 percent of the times it occurred. Brand D was least popular, having been selected 16.5 percent of the time it occurred. Since there are four brands, and also four concepts per task, each brand appeared once in every task. The sum of proportions for the four brands (not shown) is 0.932. The balance, 0.068, is the proportion of tasks in which respondents selected "None." Price level 1 was the most expensive, and concepts at that price level were only selected about a third

792

SSI Web v7 as often as concepts at the lowest price, level 4. Since Price also has four levels, each level appeared exactly once in each task, and the sum of their proportions is identical to the sum for Brand. The Size and Shape attributes only had three levels, so their levels sometimes appeared twice in a task. That produces proportions with smaller sums. If a level appears twice in the same task, and if one of the concepts including it is selected, then the other concept is rejected. When an attribute has fewer levels than the number of concepts in a task, the sum of its proportions will be lowered. When making comparisons across attributes it is useful first to adjust each set of proportions to remove this artifact. One way to do so is to divide all the proportions for each attribute by their sum, giving them all unit sums. Counts reports a Chi Square statistic for each main effect and joint effect indicating whether the proportions in that table differ significantly from one another. We refer to that Chi Square test as the "Within Att. Chi-Square." In the case of a main effect count, the Chi Square indicates whether levels of that attribute differ significantly in their frequency of choice. Beware of interpreting the Chi Square from aggregate counts as a measure of "Importance" for an attribute or assuming that the main-effect Chi Square test that is not significant indicates that the attribute had little impact on choice. Disagreement between individuals on what level is preferred can mask the impact of an attribute when respondent choices are aggregated. For example, if only two brands are tested and half of the respondents strongly prefer Brand A over Brand B, whereas the other half feel exactly the opposite, the aggregate count proportions will be equal, and the Chi Square will also be zero. In that case, we would be in error to infer that brand had no impact on choice for individuals. The tests for joint effects measure differences among the proportions in the table beyond those due to the main effects. For example, suppose the proportions in the first row of a joint-effect table were all just half the size of corresponding proportions in the second row. Such differences would be due to the main effect for the row attribute, and would not show up as a large Chi Square for the joint effect. A large Chi Square value suggests a significant interaction effect between the two attributes. (No Chi Square is reported for 3-way joint effects). Each effect is classified "not significant," "significant with p< .05," "significant with p< .01." Actual values of Chi Square and the appropriate number of degrees of freedom are also reported, so you can consult a table of Chi Square to determine the precise level of significance for each effect. The Chi Square tests reported by Counts are computed differently than those reported by Logit, and will not agree precisely, though effects with highly significant Chi Square values for either module should also have highly significant values for the other. As with other conjoint methods, it is often useful to summarize choice data with numbers representing the relative importance of each attribute. With utility values we base importance measures on differences between maximum and minimum utilities within each attribute. However, with proportions, corresponding measures are based on ratios within attributes. To summarize relative importance of each attribute we might first compute the ratio of the maximum to the minimum proportion for each attribute, and then percentage the logs of those ratios to sum to 100. We generally do not recommend computing attribute importances using aggregate (summary) data. Attributes on which respondents disagree will appear to have less importance in the aggregate, even though respondents feel very strongly about their differences in opinions. We recommend computing attribute importances using Latent Class, HB or ACBC utilities.

SMRT (Market Simulator) Help

793

The joint effects tables provide the same kind of information as main effects, but for pairs of attributes rather than attributes considered one-at-a-time. For example, consider the following table for Brand and Price:
Brand Brand Brand Brand A B C D Price1 0.262 0.083 0.104 0.078 0.132 Price2 0.320 0.146 0.100 0.129 0.174 Price3 0.398 0.254 0.163 0.206 0.255 Price4 0.570 0.347 0.321 0.249 0.372 Avg 0.387 0.207 0.172 0.165

Average

We have labeled the rows and columns for clarity, and also added a row and column containing averages for each brand and price. Comparing those averages to the main effects for Brand and Price, we see that they are identical to within .001. The similarity between main effects and averages of joint effects depends on having a balanced design with equal numbers of observations in all cells. That will only be true with large sample sizes and when there are no prohibitions. Finally, counts data are ratio-scaled. A count proportion of 0.30 versus 0.15 means that respondents on average chose (preferred) the first level twice as much as the second. Since preference for an attribute level depends upon the desirability of the other alternatives within that same attribute, it is not appropriate to directly compare a count proportion from one attribute level to a level from a different attribute.

Some Notes on Counting Analysis

As mentioned at the beginning of this section, Counting analysis is a quick way to summarize the results of choice data. However, for getting the most from your CBC data, we recommend more sophisticated means of analysis, such as Logit, Latent Class, and HB. Counts analysis reflects some known biases that can prove problematic in some situations. Given a large enough sample size, the number of times each level was displayed should be nearly balanced, in a 1-way, 2-way and even 3-way sense (though CBC's design methods only pay attention to 1- and 2-way representation). But with smaller sample sizes, random imbalances in the design can distort counts proportions. For example, if a particular brand level happened to be shown at a low price more often than other brands, the count proportion for that brand could be distorted upward. Other methods of analysis (Logit, Latent Class, and HB) are not subject to this difficulty. The counting analysis produces different results from the logit approach. CBC's counting results are just the proportions of times when levels (or combinations of levels) are offered that they are in fact chosen. Since all attributes are constantly varying, the counting result for a specific attribute level does not depend on that level alone. Even an extremely undesirable level may be chosen because it is paired with desirable levels. The result is that the main-effects counts proportions are biased in the direction of being flatter than the results of logit simulations, in which all omitted attributes are assumed to be constant. Other effects may work in an opposite direction. Under the Complete Enumeration, Shortcut, and to a lesser degree the Balanced Overlap method, CBC tries not to show the same attribute level twice in the same task. This means that a strong level has weaker levels as competition, which tends to accentuate differences between levels, partly overcoming the effect described above. For example, consider the case of the brand x price joint effect counts table, often used to reflect demand curves. If attributes have different numbers of levels, the maximum possible choice proportion differs between them, which can make it even more difficult to compare results between attributes.

794

SSI Web v7

8.8.2

Logit Analysis for CBC

Overview
The previous section described how to analyze CBC results using the Counting approach. This section introduces a more sophisticated technique for analyzing CBC data called multinomial logit. Multinomial logit is a general model for fitting choice data, but in this section, we'll be referring to a specific pooled (aggregate) logit model. We'll also refer to three relatively new methods which have been demonstrated to work even better in practice than aggregate logit: Latent Class, and HB. These three methods are not available in the base SMRT system, but are offered in advanced add-on modules. The Market Simulator uses utility (part-worth) information to estimate respondent interest (preference) for different product concepts. Before you can use the Market Simulator, you must compute utilities using either logit, Latent Class, or HB estimation.

Importing CBC Data

Prior to conducting this analysis, you must import the data from your CBC study. We'll assume that you have already downloaded the data from the Web server, and prepared the .CHO and .ATT files using the SSI Web system (by clicking Analysis | Prepare .CHO and .ATT Files for Analysis). To import the CBC data for use within the CBC Analysis Module and Market Simulator (SMRT): 1. Open the SMRT software by clicking Start | Programs | Sawtooth Software | Sawtooth Software SMRT. 2. Create a new study, by clicking File | New and providing a study name in the desired directory. 3. Import your CBC data by clicking File | Import, specifying Choice Data (*.cho) as the Import Type, and clicking the Import button.

What Are Part Worth Utilities?

A part-worth utility is a measure of relative desirability or worth. When computing utilities using logit, every attribute level in a conjoint project is assigned a utility (also referred to as a part-worth). The higher the utility, the more desirable the attribute level. Levels that have high utilities have a large positive impact on influencing respondents to choose products. When using logit, Latent Class, or HB, the raw utilities are zero-centered within each attribute. For example:
Level $300 $200 $100 Utility -0.6 0.1 0.5

This example shows respondents preferring lower price levels to higher ones. For information on interpreting conjoint utilities, please read Appendix B of the SSI Web manual.

SMRT (Market Simulator) Help

795

Choosing Effects for Logit

By default, SMRT estimates utilities for all main-effects. Main-effects reflect the impact of each attribute on product choice measured independently of the other attributes. Main-effect models are the simplest models, resulting in a single utility value associated with each attribute level in the study. You can add additional terms to the model to account for two-way attribute interactions. For example, if the combination of "Red" with "Mazda Miata" results in greater preference than the main-effect utilities would suggest, an interaction term can capture and reflect that synergy. Interaction terms can also reflect differences in price sensitivity for different brands. We suggest only adding interaction terms that result in a significant improvement in the overall fit of the model. Including too many terms in a model can lead to overfitting, which has the potential of modeling a good deal of noise along with true effects. When using aggregate logit, it is important to investigate whether including interaction terms can significantly improve the fit of the model. With methods that recognize respondent differences (Latent Class and HB), many complex effects (including, but not limited to, interactions) can be reflected in market simulations using only main-effects. It is our experience that including interaction terms in logit models can often significantly improve predictability of the model, but that those same terms added to Latent Class are not as valuable and can even, in some cases, be detrimental. Thus, with little effort, one can achieve excellent results with Latent Class and HB using main-effects.

Running Logit Analysis

Logit analysis is an iterative procedure to find the maximum likelihood solution for fitting a multinomial logit model to the data. When only main-effects are estimated, a value is produced for each attribute level, which can be interpreted as an average utility value for the respondents analyzed. After logit converges on a solution, the output is displayed in the results window, and you are given the option to save the run to the utility file. You can save multiple runs to the utility file, and each run is available to you when running simulations. The computation starts with estimates of zero for all effects (utilities), and determines a gradient vector indicating how those estimates should be modified for greatest improvement. A step is made in the indicated direction, with a step size of 1.0. The user can modify the step size; a smaller step size will probably produce a slower computation, but perhaps more precise estimates. Further steps are taken until the solution stops improving. For each iteration the log-likelihood is reported, together with a value of "RLH." RLH is short for "root likelihood" and is an intuitive measure of how well the solution fits the data. The best possible value is 1.0, and the worst possible is the reciprocal of the number of choices available in the average task. For these data, where each task presented four concepts plus a "None" option, the minimum possible value of RLH is .2. Iterations continue until the maximum number of iterations is reached (default 20) or the log-likelihood increases by too little (less than 1 in the 5th decimal place). The user also has the option of saving variances and covariances of the estimates. The square roots of the variances are equal to the standard errors, and are always displayed. The default is not to display variances and covariances. Here is the output for a computation, using the same data as were used in the second example

796

SSI Web v7 described in the previous section on Counting analysis (about 100 respondents, each of whom answered 8 randomized choice tasks, each including 4 concepts and a "None" option).
CBC System Multinomial Logit Estimation Copyright 1993-2006 Sawtooth Software Main Effects Specifications for this run: Max iterations 20 Variances and covariances not saved Step size 1.00000 Max change in loglike 8e-007 Iter Iter Iter Iter Iter Iter 1 2 3 4 5 6 log-likelihood log-likelihood log-likelihood log-likelihood log-likelihood log-likelihood = = = = = = -1471.20588 -1462.75221 -1462.73822 -1462.73822 -1462.73822 -1462.73822 rlh rlh rlh rlh rlh rlh = = = = = = 0.24828 0.25028 0.25028 0.25028 0.25028 0.25028

Converged. Log-likelihood for this model = Log-likelihood for null model = Difference = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 236.82822 Std Err 0.05126 0.06021 0.06411 0.06509 0.04899 0.04962 0.05199 0.04895 0.04939 0.05201 0.07101 0.06409 0.05708 0.05185 0.12434 -1462.73822 -1699.56644 473.656

Chi Square = t Ratio 12.12462 -0.95331 -4.12943 -4.59957 2.82895 1.54217 -4.13734 3.10636 0.99716 -3.87086 -7.45947 -3.54773 3.03928 11.25616 -8.65299 1 1 1 1

Effect 0.62150 -0.05740 -0.26472 -0.29938 0.13859 0.07652 -0.21510 0.15207 0.04925 -0.20132 -0.52970 -0.22737 0.17347 0.58361 -1.07590

Attribute Level 1 Brand A 2 Brand B 3 Brand C 4 Brand D

2 1 Shape 1 2 2 Shape 2 2 3 Shape 3 3 1 Large Size 3 2 Medium Size 3 3 Small Size 4 4 4 4 1 2 3 4 Price Price Price Price 1 2 3 4

NONE

The output starts with a listing of values that govern the computation. The values shown here are default values, and the user will rarely need to modify them. The default value for maximum number of iterations is 20, and this computation required only 6. Logit analyses are often evaluated by Chi Square statistics. The procedure is to determine the log likelihood that would be obtained, given the sample size and the nature of the data, if the estimated effects were all zero. That log likelihood is compared to the log likelihood for the estimates obtained. Twice the difference between those two log likelihood values is distributed as Chi Square, with degrees of freedom equal to the number of parameters estimated. The number of parameters estimated here is 11, obtained by adding the total number of levels and subtracting the number of attributes. With 11 degrees of freedom, a Chi Square of about 25.0 would be significant at the .01 level. The obtained value of 473.6 is safely larger than this, so we would conclude that respondent choices are significantly affected by the attribute composition of the concepts.

SMRT (Market Simulator) Help

797

Within each attribute, the effects sum to zero. That is because we actually omit one level for each attribute in doing the estimation, and then supply a value afterward for the missing level that is equal to the negative of the sum of the others. Our use of "effects coding" makes this possible. In the logit analysis output, "effect" is synonymous with utility. To the right of each estimate is a standard error, and to the right of that is a t ratio. The t ratio is a measure of the significance of the difference between that level's effect and the average of zero for all levels within the attribute. When there are no interaction effects, as in this example, the relative attractiveness of a concept can be assessed by adding up the effects for its component attribute levels. For example, consider these two hypothetical concepts:
Concept 1 Effect Brand A Shape 3 Small Price 3 Total Concept 2 Effect Brand B -0.05740 Shape 1 0.13859 Large 0.15207 Price 1 0.52970 -0.29644

0.62150 -0.21510 -0.20132 0.17347 0.37855

These two hypothetical concepts are each "scored" by adding up the effects of their component attribute levels. Concept 1 should be strongly preferred to Concept 2 by these respondents. In fact, we can go one step further and estimate how strong that preference would be. If we exponentiate (take the antilog of) each of the total values we can then express them as percentages to predict the proportion of respondents who would choose each concept if they had to choose one or the other:
Concept 1 Concept 2 Total Total 0.37855 -0.29644 exp(total) 1.460 0.743 2.203 Percent 66.3% 33.7%

If forced to choose between Concept 1 and Concept 2, about two thirds of these respondents should choose Concept 1 and one third should choose Concept 2. To show how the "None" parameter is used, we add a third item with total equal to -1.07590, the estimate for choosing "None."
Concept 1 Concept 2 "None" Total Total 0.37855 -0.29644 -1.07590 exp(total) 1.460 0.743 0.341 2.544 Percent 57.4% 29.2% 13.4%

In a three-way contest between Concept 1, Concept 2, and the option of choosing neither, about 57% should choose Concept 1, 29% should choose Concept 2, and 13% should choose "None." To see how logit output and analysis by counting choices output are similar, we use the same procedure to estimate the distribution of choices for four concepts differing only in brand, plus a "None" option. The resulting numbers in the logit output should be similar to the main-effects for Brand as reported in the output for analysis by counting choices.
Effect Brand Brand Brand Brand None Total A 0.62150 B -0.05740 C -0.26472 D -0.29938 -1.07590 exp (effect) 1.862 0.944 0.767 0.741 0.341 4.655 Proportion 0.400 0.203 0.165 0.159 0.073 1.000 From COUNT 0.387 0.207 0.173 0.165 0.068 1.000 Difference 0.013 -0.004 -0.008 -0.006 0.005 0.000

798

SSI Web v7 Although the estimates derived from logit analysis are not exactly the same as those observed by counting choices, they are very similar. The differences are due to slight imbalance in the randomized design (and due to other factors mentioned at the end of the previous section). The estimates produced by the logit calculation are slightly more accurate. With a larger sample size we might expect differences between the two kinds of analysis to be even smaller. After logit has finished running, results are displayed to the results window, including summary information about the computation (the total number of tasks and time for the computation). You are given the option to save the run to the utilities file. You can also print, or cut-and-paste the logit report to a file. We emphasize that logit analysis is an advanced statistical technique that requires understanding to be used properly. We have made it easy to conduct logit analyses with SMRT. The burden is on you to use the results wisely. We urge you to consult appropriate reference materials. We also urge you to consider the limitations of logit analysis relative to the Latent Class or HB models.

Managing Multiple Logit Runs

In the process of analyzing your data, you will probably run logit a number of times. Some runs may test the inclusion of specific interaction effects; others may include just a subset of the sample or tasks. The Market Simulator includes a Run Manager that helps you manage this process. After each logit run has completed, the results are displayed in the results window, and you are prompted to "save the run." If you choose to save the run, the utilities and set-up parameters (respondents included, tasks included, effects, etc.) are stored in the studyname.UCS file. If you do not store the run, the utilities are deleted. You can store as many runs as you want in the studyname.UCS file. You can view the settings from previous runs. You can also delete runs you no longer need. The Market Simulator lets you choose a run to use for each simulation, and then reads the appropriate utility information from the studyname.UCS file.

Notes on Determining Significant Interaction Effects

By default, SMRT's logit program calculates all main-effect utilities. Sometimes main-effects models can be improved upon by adding interaction terms. We'll define an "improved" model as one that: 1. Significantly improves the model fit in terms of log-likelihood, 2. Improves the accuracy of the market simulator in terms of aggregate shares vs. fixed holdout choice tasks or market shares. We urge you to only include interaction terms in the model that significantly improve results. To determine whether to include interaction terms, we suggest you first study the two-way tables provided by Counts analysis. The Chi-square statistic is one indication of possible significant interaction, but we suggest you look further. We also suggest plotting the Counts proportions with a scatter plot display to verify that deviations from proportionality in the table seem plausible given what you know about buyer preferences. Most importantly, we suggest you conduct a "2 log-likelihood" test. We alluded to this test earlier in this section with respect to determining whether the computed logit effects significantly improve our ability to predict respondent choices relative to the null model (effects of zero). A 2 log-likelihood test can also be used to determine whether the addition of an interaction term to the logit model significantly improves the fit. We will illustrate with a simple example.

SMRT (Market Simulator) Help

799

Assume a simple CBC design with two attributes, each with four levels. Under main-effects, a total of eight part-worths result. Recall, however, that each four-level attribute is coded as just three parameters in the logit model under effects-coding. Assume we run this main-effects logit model and achieve a log-likelihood of -1500.00. The interaction between two four-level attributes, each effects coded as three parameters, results in an additional 3 x 3 = 9 parameters added to the model. Suppose we include this interaction term and the log-likelihood improves to -1490.00. Adding the interaction term has improved the log-likelihood by 10.00. Two times the log-likelihood is distributed as Chi-square. We refer to a Chi-square table (available in most statistics textbooks), and look up the p-value for a Chi-square of 20.00 with 9 degrees of freedom (the number of additional parameters added to the model). We find that the pvalue is roughly 0.02, suggesting a significant improvement in the model by adding the interaction terms with a confidence level of 98%.

Technical Notes on Logit

SMRT uses multinomial logit analysis (MNL) to estimate effects for attribute levels. Additional information is provided by Hosmer and Lemeshow in Applied Logistic Regression (Wiley, 1989). MNL is similar in many ways to multiple regression and discriminant analysis. Like those methods, it seeks "weights" for attribute levels (or for combinations of them, if interactions are included in addition to main effects). Those weights are analogous to "part-worth utilities" in conjoint analysis, and are computed so that when the weights corresponding to the attribute levels in each concept are added, the sums for each concept are related to respondents' choices among concepts. MNL assumes that the relative probabilities of respondents' choosing each concept within any task can be estimated from the weights in the following way: 1. Sum the weights for the attributes appearing in each concept to get a value analogous to that concept's "total utility." 2. Convert total utilities to positive values by exponentiating them. The resulting values may be considered analogous to relative probabilities, except that they do not lie within the unit interval. 3. Normalize the resulting values so that within each task they sum to unity, by dividing the values for concepts within each task by their sum. Suppose we have such probabilities for the concepts in one choice task. Then we can say that, according to our model, the likelihood of a respondent's choosing each concept is equal to the probability for that concept. If we assume choices for different tasks to be independent, then we can compute the likelihood (under our model) of seeing any particular pattern of choices for any number of respondents and tasks. That likelihood is just the product of a number of computed probabilities. If there are N respondents, each having responded to k choice tasks, then the likelihood of that pattern of responses is just the product of Nk such probabilities. MNL chooses weights that maximize the likelihood of the observed pattern of respondent choices, using probabilities derived from the weights as just described. Since likelihoods are obtained by multiplying many probabilities together and each is less than unity, likelihoods are usually very small positive numbers. It is more convenient to think about their logarithms, and users of logit estimation usually speak of "log-likelihoods" rather than likelihoods themselves. This is just a matter of convenience, because the set of weights that maximize the likelihood must also maximize the log likelihood. In SMRT's logit routine we also report another somewhat more intuitive measure, "root likelihood," which we abbreviate "RLH." This is just the geometric mean of the probabilities corresponding to the

800

SSI Web v7 choices made by respondents, obtained by taking the Nk'th root of the product of the Nk probabilities. The best possible value of RLH is unity, achieved only if the computed solution correctly accounts for all the choices made in all tasks by all respondents. The computation method used by SMRT is iterative. It starts with a "null model" consisting of weights all equal to zero, and computes the likelihood of the respondents' choices, given probabilities corresponding to that initial solution. During that computation information is also accumulated about a "gradient vector," indicating how the initial solution should be changed for greatest improvement. Those changes are made, and a second iteration evaluates an improved solution, while also accumulating information about how a third iteration might improve on the second. The iterative process continues until one or more of the following occurs: The number of iterations exceeds a limit. The default is a limit of 20. The change in log-likelihood from one iteration to the next is less than a limit. The default is 1 in the fifth decimal place.

We do not anticipate that users will need to change these defaults, but it can be done by clicking the Advanced Logit Settings button. The dialog includes four settings: 1. 2. 3. 4. Output option. If checked, more information about variances and covariances of estimates is provided in the output. Step size (default = 1.0). This number governs the sizes of changes made in the iterative computation. If set to a smaller number, such as .5 or .1, the computation will be slower but may be somewhat more precise. Iteration limit (default 20). Log-likelihood convergence criterion (default l in the fifth decimal place).

Effects Coding
Logit analysis, like least squares, suffers from the dependencies among attribute levels when all the levels of an attribute are entered into the computation. For this reason one level of each attribute is deleted automatically, although results for that level may be obtained by inference from the others, and those results are provided in the output. Logit deletes the last level. Remaining levels are represented using "effects coding." This is a way of coding for levels so that the average effect within each attribute will be zero. After the logit computation, we provide the estimate for each deleted level just by computing the negative of the sum of the included levels. Logit analysis permits estimates not just of main effects, but also of two-way interactions.

Tests of Significance
SMRT offers two kinds of significance tests: a t ratio and an overall Chi Square test. Probably the more useful is the Chi Square for the difference between two "nested" models, where the second includes all effects of the first plus one or more others. Twice the difference between the loglikelihoods of those two models is distributed as Chi Square, with degrees of freedom equal to the number of additional effects in the second model. This test is performed automatically in each logit analysis, comparing the results of that analysis to what would be obtained with no effects at all. The test can also be computed manually to compare any other nested runs, such as one run with all main effects and another with just a subset, or one with just main effects and another with main effects

SMRT (Market Simulator) Help

801

and one or more interactions. (The Chi Square of the difference is the difference of the Chi Squares.) The other test is a t ratio provided for each attribute level. This tests the difference between that level and the average of all levels for that attribute (which is zero). These t ratios can provide useful guidance, but for measuring whether an effect is significant the overall Chi Square test is preferable. The standard errors used to compute t tests are taken from the matrix of covariances among estimates that is obtained by inverting a sum of squares and cross-products matrix. If many parameters are being estimated or there are relatively few observations, this matrix may be illconditioned (having determinant less than 1.0E-10). in this case a "ridge adjustment" is made by adding unity to each diagonal position of the sum of squares and cross-products matrix, reinverting, and displaying an error message. We do this so the computation will provide useful information; however, the presence of a "ridge adjustment" message indicates the solution to be unsatisfactory, and you should probably remove some interaction terms and re-estimate.

802

SSI Web v7

8.9
8.9.1

Appendices for Simulation Topics

Importing Conjoint Data from Text-Only Files
The Market Simulator can import conjoint part-worth utility data from any of the files generated by Sawtooth Software's current conjoint analysis systems. If you are importing data that you generated yourself, you should create a text-only file matching the ".HBU" format. If your utility file includes linear terms, you must also provide a .VAL file. The formats for these two files are described below.

The ".HBU" Import Format

This format gets its name from the CBC/HB system, which creates text-only files containing conjoint part-worth estimation with the extension ".HBU". It is the most flexible of the Sawtooth Software partworth file formats, accommodating: Part-worth or linear estimation of attribute effects Main effect and optional first-order interactions Individual-level records, with a measure of fit (e.g. correlation) that can be used as a filter for exclusion The .HBU file should be created as a text-only, space or tab-delimited format. A Simple, Main Effects Example Here is an example file for a simple study with 3 attributes (main effect part-worth estimation only) and a total of 2 respondents:
3 3 1 0 0 1 1 1 2 2 2 2 3 3 3 3 0 11 1 1 4 4 0 0 1 0 0 1 1 Brand A 2 Brand B 3 Brand C 1 Style A 2 Style B 3 Style C 4 Style D 1 $8 2 $10 3 $15 4 $20 1001 778 0 11 0 -1.323 0.667 0.656 -.520 0.836 -1.394 1003 540 0 11 0 -1.042 0.235 0.807 -.239 -0.211 -1.219

0.310 0.310

-0.626 0.140

1.271 1.060

0.652 -.529 0.254 -.095

The data structure consists of a header section (the first 16 lines in this example), followed by a respondent header (one per respondent) and preference values for each respondent. The first line contains the number of attributes (3), whether a "None" utility is included (1 if yes, 0 if no), the total number of parameters estimated for each individual (11), and the numbers 1 and 1. (These last two values are just to conform to the file format for the Latent Class module, which uses the same header format.)

SMRT (Market Simulator) Help

803

The second line contains the number of estimated terms for the main effects for each attribute (3, 4, 4). Following is a line for each attribute, each with as many entries as there are attributes. This is an attribute-by-attribute matrix of ones and zeros (or minus ones) that indicates which effects were estimated. This example represents main effects only and is indicated by ones on the diagonal. If interactions were included (they weren't for this example), they would be indicated by a one in other positions. Linear variables (there weren't any for this example) are indicated by negative ones on the diagonal. Following are labels, one for each parameter estimated. There are two leading space-delimited integers on each line, indexing the attributes and levels. These attribute/level labels are in the same order as the parameter estimates that follow in the file, and serve to identify them. If interaction parameters were estimated, then this list will include a label for each term. The record for each respondent starts with a line that contains: Respondent number (respondents 1001 and 1003 are found in this example). Respondent numbers are limited to 9 total characters. A measure of fit such as R-squared or RLH can be included and used as a filter in simulations. The range of acceptable values is from 0 to 1000. If no measure of fit is available, you can set this to an integer within that range such as "0". A value of zero (this is a carry-over from other Sawtooth Software formats) The total number of parameter estimates per respondent (in this case, 11 part-worths) A value of -1 if a "none" utility exists, or a zero otherwise. This respondent header is followed by the parameter values for that respondent, in the same order as the labels in the header. Even though the example above uses utility values with just three decimal places of precision, up to 15 significant digits of precision may be used. The utility values can be formatted on a single or multiple lines (hard return characters within a respondent record are ignored). However, each respondent's record should begin a new line. A Complex .HBU File Example If your conjoint part-worths include linear terms (such as a single coefficient representing the main effect for Price), you must supply two files: .HBU and .VAL (format described below). The example below illustrates a linear term and the inclusion of a first-order interaction.
3 1 12 1 1 3 4 1 1 0 1 0 1 0 1 0 -1 1 1 Brand A 1 2 Brand B 1 3 Brand C 2 1 Style A 2 2 Style B 2 3 Style C 2 4 Style D 3 1 Price Brand A x Price Brand B x Price

804

SSI Web v7
Brand C NONE 1003 -0.099 -0.928 1004 7.197 3.776 1005497 3.314 0.522 x Price 368 0 1.082 1.004 708 0 4.854 3.872 0 12 1.854 2.709 12 -1 -0.983 -0.070 12 -1 -12.051 -1 -5.169 7.022 3.656 2.106 2.581 1.736 -3.585 -4.098 -3.652 1.549 -5.504 -1.740 -1.463 -0.735 -0.726 0.391 -1.496 -0.114 0.537 -2.280 -0.408

Note in this example that both linear effects and interactions have been specified. Because only one coefficient is estimated to represent the main effect for Price, note the "1" in the second line, third position. We've copied lines three through five below and added bolding and underlining for further illustration:
1 0 1 0 1 1 0 0 -1

The elements along the diagonal (in bold) correspond to the main effects. All main effects are included, but attribute 3 is represented by a single coefficient (linear term). The linear terms are indicated by "-1" in the design matrix. Off-diagonal elements reflect interaction terms. There is one interaction (underlined) specified: attribute 1 x attribute 3 (note that it is indicated in both the lower and upper halves of the matrix). The order of utility values must be as follows: Main effects Interactions, in the following order (assuming complete specification of interactions): Attrib1 x Attrib2 . . . Attrib1 x Attribn, Attrib2 x Attrib3 . . .Attrib2 x Attribn, etc. None (only one value) Important Note: The Market Simulator assumes that the independent variables used for linear variables were zero-centered. If you used Sawtooth Software's CBC/HB or Latent Class systems, this happened automatically during utility estimation. If you are providing utility files from your own estimation procedure, you should make sure your linear codes are zerocentered.

The .VAL File

If your .HBU file includes linear terms, you must provide an accompanying .VAL file. Let's assume that you estimated a linear term for Price, and that the dollar values were $8, $10, $15 and $20. The Market Simulator assumes that you zero-centered the codes for linear variables for estimation. Therefore, you may have coded these prices as -5.25, -3.25, 1.75, 6.75 in your independent variable matrix. In that case, the accompanying .VAL file should be as follows:
Brand 1 2 3 Style 1 2 3 4 Price -5.25 -3.25

1.75

6.75

SMRT (Market Simulator) Help

805

There are two lines for each attribute: the first is a short attribute labels and the next contains blankdelimited values to be assigned to that attribute's levels for use in defining products in the simulator. There must be one value for each level. In this example the first two attributes have values of 1, 2 and 3, which are used as nominal level codes for these attributes. The third attribute has values of -5.25, 3.25, 1.75 and 6.75. When you import the .HBU and .VAL file into the Market Simulator, these numeric codes will be assigned to the different levels of price. However, you can always add back the constant to change these values to 8, 10, 15 and 20 (the original prices in our example) for ease of use in the simulator by clicking Assign Level Values. Adding back the constant (that was subtracted for zero-centering) is appropriate as during simulations the Market Simulator always zero-centers level values for linear terms before applying them to the coefficients in the utility file.

806

SSI Web v7

8.9.2

How the Simulator Interpolates between Levels

When dealing with main effects, the Market Simulator does linear interpolation between adjacent levels. That is a straightforward procedure that requires no explanation. However, when dealing with two-way interactions, the situation is considerably more complex. This appendix explains and justifies the procedure used in two-way interpolation. With one-dimensional interpolation, as for main effects, we first find a line containing the two extreme points, and then choose a result from that line. With two-dimensional interpolation the situation is different. We have a total of four points, composing the four corners of a square. For convenience we might label those points as follows:
Attribute 2 Level 1 Attribute 1 Level 2 C D Level 1 A Level 2 B

In this scheme, point A represents the combination of the first level for both attributes, D represents the combination of level 2 for both attributes, and B and C each represent combinations of level one on one attribute and level 2 on the other. By analogy with the one-dimensional case, we might try to fit a plane to all four points, and then do our interpolation within that plane. Although a plane can always be found to fit three points, it will not generally be true that a plane will fit four points very well. Our circumstance is particularly troublesome because the surface required to fit all four points would be quite warped. If both attributes happen to have only two levels, then the row and column sums of the 2 x 2 array are zero, implying that A = -B = D = -C. If both dimensions have more than two levels the relationship is not so precise, but the surface containing the four points still tends to be saddleshaped. Accordingly, we use a nonlinear interpolation procedure, which we describe with a numerical example. Suppose a product has specifications of 1.2 for attribute 1, and 1.6 for attribute 2. Also suppose the logit estimates for the four points are:
Attribute 2 Level 1 Attribute 1 Level 2 C=-0.5 D= 2.0 Level 1 A= 1.0 Level 2 B=-1.5

Finally, let p1 = the proportion of the distance between the two levels of Attribute 1 for which we would interpolate (0.2 in this example) and p2 = the proportion of the distance between the two levels of Attribute 2 for which we would interpolate (0.6 in this example). We interpolate by forming the convex sum of four corner points:
Result = + + + P(1-p1) (1-p1) p1 p1 * * * * (1-p2) p2 (1-p2) p2 * * * * A B C D

This procedure describes a surface that includes all four corner points as well as their centroid. Notice that if p1 = p2 = 0, then the result is equal to A. If p1 = p2 = 1, the result is equal to D. If p1 = 0 and p2 = 1, the result is equal to C, and if p1 = 1 and p2 = 0, the result is equal to D. If both attributes have integer specification, this method of interpolation therefore gives the same answer as if no interpolation

SMRT (Market Simulator) Help were used.

807

This procedure has the desirable property that there are no discontinuities in the regions of the four corner points, as there would be if we tried to fit the points with a plane. If one attribute has an integer specification but the other requires interpolation, then interpolation is automatically linear for the one attribute requiring it. For example, if p1 = 0, then the result is (1-p2)*A + p2*B, a linear interpolation between A and B. Similarly, if p1 = 1, then the result is a linear interpolation between C and D. Also, if p1 = p2 = 0.5, the result is (A+B+C+D)/4. Therefore the result at the center of the square is the average of the four corners. If both attributes require interpolation, then the surface is quite regular, with only as much curvature as required to fit the four corners. For our numerical example, the result is -0.02. The center of the square has a value of 0.25, but the location corresponding to p1 = 0.2 and p2 = 0.6 is upward and to the right of the center, toward the corner that has a value of -1.5. Since our procedure is nonlinear, it would be hazardous to extrapolate beyond the known points. Therefore, the simulator does not permit the user to extrapolate beyond measured levels of the attributes for those involved in interaction effects.

808

SSI Web v7

8.9.3

How the Simulator Uses Latent Class Data

Sawtooth Software provides Latent Class analysis software for use with our CBC System for ChoiceBased Conjoint. With Latent Class, rather than computing a set of part-worths (utilities) for each respondent, the algorithm finds groups of respondents with similar preferences and estimates average part-worths within these segments. As when using cluster analysis, the Latent Class analyst specifies how many groups to use in the segmentation. In contrast to cluster analysis, respondents are not assigned to different segments in a discrete (all-or-nothing) manner under Latent Class analysis, but have probabilities of membership in each segment that sum to unity. The sum of the probabilities of membership across respondents for each group defines the total weight (class size) of that segment. One can conduct overall market simulations with Latent Class results by computing shares of preference within each segment and taking the weighted average of these shares across segments. This is the approach used in Sawtooth Software's Latent Class Simulator in version 1 of the software (LSIM.EXE). Another way to use Latent Class data is to convert the segment-based results into individual-level estimates. While these estimates are not as accurate at characterizing respondent preferences as Hierarchical Bayes analysis, they are an appropriate extension of the Latent Class model. The Market Simulator converts the group-based Latent Class part-worths into individual-level partworths in the following way: For each respondent, a weighted combination of the group part-worth vectors is computed, where the weights are each respondent's probabilities of membership in each group. Converting the Latent Class utilities to individual-level part-worths provides added flexibility for market simulations. It lets the analyst apply segmentation variables as filters, banner points or weights without requiring that a new Latent Class solution be computed each time. However, creating individual-level utilities from a segment-based solution slightly alters the results when comparing the output of Sawtooth Software's Latent Class module to the same data used within the Market Simulator. While the overall shares of preference for the market are nearly identical, the within-class results reported in the Market Simulator output are slightly less differentiated between segments (pulled toward the overall market mean). That is because for the purpose of banner points and filters, the Market Simulator assigns respondents fully into the latent class for which they have the greatest probability of membership. For example, consider a respondent whose preferences are characterized as 80% like group 1 and 20% like group 2. His contribution to the mean values reported in the group 1 column (banner point) includes some group 2 tendencies. The differences between the within-class means reported by Latent Class and the Market Simulator are not usually very great since respondents' probabilities of membership in classes usually tend toward zero or one. The smoothing that occurs when reporting the by-segment results in the Market Simulator will probably not substantially change your interpretation of the results. If the differences concern you, you can always refer to the original Latent Class output.

Index

809

Index
-AA priori order 331, 596, 677 Abs (Absolute Value) 156 ACA 550, 606, 752, 758, 769, 775, 785 ACA Algorithms 614, 617, 619 ACA Calibration Concepts 590 ACA Control Parameters 600 ACA Customizing Importance Question 612 ACA Data Storage 603 ACA Dropping Importance Question 610 ACA Functions 160 ACA Interview Sections 590 ACA Overview 552 ACA Staying Out of Trouble 608 ACA Tutorial 553 ACA/HB 621 ACAAttribute 160 ACABest 160 ACACalMax 160 ACACalMin 160 ACAImportance 160 ACAImportance Question 590 ACALeastImpAttLabel 160 ACALeastImpAttValue 160 ACAMostImpAttLabel 160 ACAMostImpAttValue 160 ACAPair Question 590 ACAPriorsBestLevelLabel 160 ACAPriorsBestLevelValue 160 ACAPriorsWorstLevelLabel 160 ACAPriorsWorstLevelValue 160 ACARating Question 590 ACAStrictImportance 160 ACAUtility 160 ACAWorst 160 ACBC Attributes and Level Setup 405 ACBC Constructed Attribute Lists 435 ACBC Design Algorithm 429, 434 ACBC Designing a Study 402 ACBC Estimation 449 ACBC Functions 164 ACBC Hands-On Tutorial 410 ACBC Motivation 391 ACBC Overview 389 ACBC Price Variable 407

ACBC Sections and Flow 393 ACBC Specifications 388 ACBC Testing and Validation 472 ACBCBYOLabel 164 ACBCBYOValue 164 ACBCCurrentCalibration 164 ACBCCurrentChoiceTask 164 ACBCCurrentScreener 164 ACBCIsMustHave 164 ACBCIsUnacceptable 164 ACBCNumCalibrations 164 ACBCNumChoiceTasks 164 ACBCNumScreenedInConcepts 164 ACBCNumScreeners 164 ACBCWinnerLabel 164 ACBCWinnerValue 164 Accumulate CAPI Files 248 ACD File Format 622 ADD 148 Adding Questions 35 AddSorted 148 Admin Module 251 Admin module customization 103, 266 Aggregation in CBC 283 AIC 148 AIE 148 AIG 148 AIL 148 Allocation Questions 60 Alternative-Specific Designs 361 Analyzing Conjoint Data 729 ANC 148 ANE 148 Answered 151, 166 AreAnyQuotasClosed 155 AreQuotaCellsClosed 155 Assign Level Values 765 Assumptions of Market Simulators 750 Attribute Interactions 337 Attributes 338 Attributes and Levels 331, 596, 677 Auto Calibrating Attribute Variability Multiplier

785

-BBack Button Disable 99 Backing Up 250 Backwards Skips 250

810

SSI Web v7 Composing Ranking Questions 63 Composing Select Questions 41 Composing Start Question 39 Composing Text Questions 54 Conditional Display in CBC 360 Conditional Display in CVA 699 Conditional Graphics in ACBC 441 Conditional Pricing in CBC 357 Conditional Pricing in CVA 700 Confidence Interval 773 Constant Sum Questions 60 Constraints in CVA/HB 726 Constructed Lists 122, 148 Cookies 209 Copying Questions 35 Copying Questions across Studies 32 Correlation Cutoff 769 Cost Data 754 Counting Analysis for ACBC 446 Counting analysis for CBC 790 Counting Analysis in MaxDiff 522 Counting Data Fields 84 Counts 729 Creating Passwords 200 Cross-Elasticities 754 Cross-Elasticity Effects 752 CSS 87 CSS hooks 87 Custom CBC Questions 371 Custom Pricing 357, 700 Custom Segmentation Variables 739 Customized MaxDiff Questionnaires 531 Customizing Admin Module 266 CVA 752, 758, 769, 775, 785 CVA Design Strategy 686 CVA Designer 714 CVA Functions 162 CVA Tutorial and Example 637 CVA Utility Estimation 715 CVA/HB (Hierarchical Bayes) for CVA 721 CVAVersion 162

Bad Respondents 530 Banner 765, 778 Base Case 755 BlockPosition 151 Boolean functions 166 BooleanToNumber(Boolean) Branching 117 Browser 99, 151 BYOCondText 164

151

-CCALIB Program 758, 785 Calibration Concepts 758, 785 Cannabilism 752 CAPI Deleting Sensitive Data 249 CAPI Interviewing 245 CBC 752, 754, 757, 764, 765, 775, 778 CBC Functions 159 CBC Introduction 283 CBC Setup 343 CBC Tutorial 286 CBC/HB 537, 758, 769, 785 CBC: Overview 281 CBCAttributeLabel 159 CBCDesignConceptValue 159 CBCDesignLevelText 159 CBCDisplayTotal 159 CBCNone 159 CBCVersion 159 Ceiling 156 Changes After Fielding Begins 243 Check-Box Questions 41 CheckSelect 151, 168 Chip Allocation Questions 60 Chi-Square test for Counts 790 CHO File 378 Cho file export ACBC 470 CHO File Format 383 CHS File Format 385 CiW Tutorial 4 Cluster Analysis 778 Composing Constant Sum Questions 60 Composing Grid Questions 57 Composing Numeric Questions 48 Composing Open-end Questions 51 Composing Questionnaires 32 Composing Questions 35

-DData Doctor 743 Data Fields 84 Data Management (in Field) Debug 151

251

Index Decimal Point Character 105 D-Efficiency in CVA 686 Deleting Data From CAPI 249 Demand Curves 754, 775 Design Efficiency 347, 542 Designing a CBC Study 325 Designing MaxDiff Studies 516 Displaying a Progress Bar 98 Displaying Previous Responses 145 DisplayListLabels 148 DisplayTotal 151 Double-Byte Characters 196 Downloading Data 255 Downloading Password Reports 261 Dual-Response None 339 Dummy Coding 719 Fit Statistic in MaxDiff 530 Fixed Choice Tasks 341 Floor 156 Font Colors 96 Foreign Language Studies 196 FormatListLabels 148 FORMATNUMBER 156, 168 Free Format CBC 371 Free Format Questions 66, 69, 70, 71, 72, 73, 74 Full Profile Conjoint 676

811

-GGetting Data 255 Getting Started 1 Getting Started with ACA/Web 549 Getting Started with CVA 633 Global Font Colors 96 Graphical buttons 94 Graphics 113, 114 GraphicsPath 151 Grid Questions 57

-EEfficiency 347 Elasticities 775 Email 238, 239 Email Invitations 239 EncodeForURL 151 Ending a Survey 55 Ending Surveys 86 ErrCursum 158 ErrField 158 ErrMax 158 ErrMin 158 Error Functions 158 Error Messages (for the Survey Respondent) ErrQName 158 ErrText 158 ErrTotal 158 Exp (Exponentiate) 156 Exponent 756, 758, 761, 769, 775, 782 Export Select File Type 268 Export Selecting Data 269 Exporting CBC Designs 354 Exporting CVA Designs 692 Exporting Open-End Data 273 External Effects 750, 761

-HHB Estimation 621 HB estimation for ACBC 454 HB Estimation in MaxDiff 524 HBU Format 802 Head Tag 97 Heterogeneity 752 Hidden Variables 75 Hierarchical Bayes 378, 621, 752 Hierarchical Bayes Analysis 537 Holdout Choice Tasks 341, 732 Holdout Tasks in CVA 696 Holdouts 756, 758 Holdouts in CVA 692 How Data Are Stored 250 HTML Cheat-Sheet 109 HTML Elements 107

100

-IICE 752, 769, 785 IIA 757, 758, 764 Images 113, 114 Importance Question 590 Importance Strict 168 Importances 729, 769

-FFatigue 683 Files created by SSI Web 278 First Choice Model 751, 756, 758, 781

812

SSI Web v7 ListHasParentMember 148 ListLabel 148 ListLabelsArray 148 ListMax 148 ListMin 148 ListValue 148 ListValuesArray 148 Ln (Natural Log) 156 Log10 (Log Base-10) 156 Logging On 253 Logit Model 764

Importing CBC Designs 354 Importing Conjoint Data 802 Importing CVA Designs 692 Importing Passwords from Text File 205 Importing Utility Runs within SMRT 736 Incompletes Report 262 Inline Scripting 143 INSERT 148 Installing Survey on Web 226, 229 Interaction Effects 752 Interaction Effects in ACBC 457 Interaction Effects in CVA 703 Interactions 337 International Surveys 196 Internationalization 196 Interpolation 763, 806 Interpreting Conjoint Results 729 Invitations 238, 239 IPAddress 151 IsAlphaNumeric 157 Isnumber 166 IsNumber 157 IsQuotaCellOpen 155 Item List in MaxDiff 513

-MMarginals 258 Market Share 752, 756, 761 Market Shares 729 Market Simulator 632 Market Simulator Dialog 765 Math Functions 156 MaxDiff Designs 542 MaxDiff Functions 163 MaxDiff Introduction 477, 478, 481 MaxDiff Items 513 MaxDiff Motivation 478 MaxDiff Paper-and-Pencil Studies 535 MaxDiff Questions 509 MaxDiff Scores on-the-fly 534 MaxDiff Tutorial 482 MaxDiffDesignLabel 163 MaxDiffDesignValue 163 MaxDiffRankAttLabel 163 MaxDiffRankAttValue 163 MaxDiffScore 163 Merged Fields 200, 204 Merging Segmentation Data 739 Method of Paired Comparisons 478 Mid 157 Midpoints Formula 775 Monotone regression for ACBC 461 Monotone Regression in CVA 713 Moving Data into SMRT 275, 380, 629 Must-Haves (ACBC) 446

-JJavaScript 76, 151, 166 Javascript Turning Off 105 JavaScript Verification 76

-LLabel 151 Latent Class 378, 539, 752, 778, 808 Length 157 Level Prohibitions 334, 599 Level Values 763, 765 License Sizes 84 Limbo Data 250 Line Extensions 778 Linear Dependency 719 Linear Terms 763 LinkButton 151 Linking Surveys 241 Linking to other surveys 55 List Building 122, 143 List Building Functions 148 ListHasParentItem 166

-NNaming Questions 35 New Product Introduction Next and Previous buttons 773 94

Index None Concept 339 None threshold in ACBC 467 None Weight 764, 769 NumberToBoolean 151 NumberToString 151 NumChecked 151 Numeric Questions 48 Price in ACBC 407, 450 Pricing Studies in CBC 357 Pricing Studies in CVA 700 Print Survey 199 Progress Bar 98 ProgressBar 151 ProgressBarOff 151 Prohibitions 331, 334, 596, 599, 677 Prohibitions for CVA 680 Purchase Likelihood Model 758, 785 Purchase Likelihoods 729 Purging Files 249

813

-OOLS in CVA 710 One-Click Access to Survey 239 Online Simulator 274 Open-End Questions 51 Optimization 778 Optimize Study Files 743 Other Specify Responses 41 Output Precision 765

-QQuestion Names 35 Quota Control 212, 216, 219, 220, 222 Quota Control Functions 155 QuotaCellCompletes 155 QuotaCellLimit 155 QuotaCellName 155 QuotaCellRemaining 155 QuotaCellValue 155 Quotas Changing During Fielding 223

-PPage Breaks 38 Page Headers and Footers 97 PageNumber 151 PageTime 151, 168 Paired Comparisons MPC 478 Pairs 617 Pairs (how many) 600 Pairwise Comparison in CVA 633, 681 Paper-and-Pencil in CBC 376 Paper-and-Pencil Studies in CVA 697 Part worths 604, 729 Partial-Profile Designs 365 Part-worth Utilities 751 Pass-In Fields 200, 208 Password Importing from Text File 205 Password Report 261 Passwords 200, 202, 211, 252 PDF File 199 Perl 174 Piping 122, 145 Popup 151, 168 POWER 156 Practice Data 235 Precision 758, 773, 786 Predefined Lists 122 Predictive Quota Control 220 Pre-Qualification 222 Price Adjustments in ACBC 439

-RRadio Button Questions 41 RadioSelect 151, 168 RandNum 151 Random Numbers 173 Randomize 148 Randomized First Choice Model 758, 786 Randomizing Blocks of Questions 115 Randomizing Pages 115 Randomizing Question Order 115 Ranking Questions 63 Red-Bus/Blue-Bus Problem 757, 758 Remove 148 RemovePrevious 151 Resetting Survey 265 RespNum 151 Respondent Verification 158 Restart 39, 55, 200, 239, 241, 255, 263 Restrictions 331, 334, 596, 599, 677 Reverse 148 RIC 148 ROUND 156

814

SSI Web v7 System Functions 151 SystemTime 151

-S-

Sample Size 758 -TSample Size for MaxDiff 524 Scale Factor 756 Tables program 746 Scenarios 755 Tabulating Data Online 258 Score Estimation in MaxDiff 524 Tabulating segmentation data 746 Screener Questions 212 TabWidth 151 Segmentation 539, 754, 778 Technical Support 1 Segmentation data into SMRT 275, 380, 629 Terminating Question 55 Select Questions 41 Test Survey 263 Sensitivity Analysis 754, 769 Testing CVA Designs 690 Server directory paths 103 Testing Designs in CBC 347 Server Installation 226, 229 Testing Questionnaire 224 Server-Side Problems 236 Test-Retest Validity 732 SetListLength 148 Text/HTML Filler Questions 54 Setting Page Breaks 38 TextExists 157 Setting Passwords 252 Throw Out Obvious Tasks 685 Share of Preference Model 756, 758, 782 ToLower 157 Share of Preference with Correction for Product SimilarityTotalPages 151 758 ToUpper 157 Shares of Preference 729 Tuning the Simulator 750, 756, 782, 786 Shelf-Facing Display 368 Shown 151, 166 -USimple Browser 99 Unacceptables (ACBC) 446 Simulation Scenario 765, 767 Unverified Perl 174 Simulations 632 Unverified Perl in ACBC 444 Single Concept in CVA 633, 681 Updating in ACA 619 Skip Logic 117, 143 Uploading Survey 226, 229 Skipping Backward 250 URL link 55 SMRT 275, 380, 629, 735 UserAgent 151 Software Requirements 28 User-defined passwords 209 SortByLabel 148 UserName 151 SortByValue 148 Utilities 604, 729 Sparse Designs in CVA 705 Utilities: Feedback 627 SPSS File Export 271 Utility Constraints in ACBC 458 Sqrt (Square Root) 156 Utility Estimation in CVA 715 SSI Script 143 Utility Runs 765 Standard Error 758, 773 UTL File Format 622 Start Question 39, 201 Strict Importance 168 -VString Functions 157 Validation 341 StringToNumber 151 Validity 732, 752, 758 Study Management 29 Value 151 Styles 87 Verification 100 Styles Modifying 87 Verification (Custom JavaScript) 76 SysRand 151 Versions in CVA 686

Index Viewing Data on Server Volumetric CBC 375 256

815

-WWeighting 742 Weights 769 Whats New? 30 Write Questionnaire Dialog

32

-ZZero-Centered Diffs 769