Conjoint Analysis Best Practices

Contributor:  Kris Stark and Hank Sanftleben
Posted:  11/23/2009  4:01:00 PM EST  |  1

Rate this Article: (3.2 Stars | 5 Votes)   Thanks for your rating!

As with most scientific procedures, there are many ways to conduct an experiment with some methods being easier than others. When using conjoint analysis, there are tools and techniques that will help make the process simpler and more efficient. Due to the reliance of conjoint analysis on the interpretations and opinions of the respondents participating in the analysis, it is extremely important that the process, attributes and levels are clearly defined. This article highlights lessons learned and best practices of traditional conjoint analysis.

Designing a Conjoint Analysis

When designing a conjoint analysis, it is important to limit the number of choice cards to ensure that the experiment is manageable. A typical respondent should have 10 to 20 choice cards. While a respondent needs several questions to get acclimated to the process, a respondent can get fatigued after 30 choices. Generally, an analysis should have at least 1.3 to 1.5 times the minimum number of degrees of freedom.

For consistent results and to minimize error, the designer should clarify the attributes, attribute levels and choice combinations of the experiment. Some researchers create an informational sheet which defines each attribute and level. The advantage of using an informational sheet as shown in Figure 1 is that it ensures consistency over the course of the analysis. Without the use of consistent definitions, researchers could subconsciously influence the respondents with knowledge gained through experimentation. This could create a long term bias in the experiment. (Click on diagram to enlarge.)

 
Figure 1: Example of an information sheet used in a conjoint analysis experiment for a change management study. Reference: The Mythological Transactional Business Process Design of Experiments.

Planning the Experiment

Before starting the analysis, the researcher should plan the experiment. A useful tool for planning an experiment is a Design of Experiments (DOE) planning sheet. Completion of a conjoint Design of Experiments planning sheet will ensure that all steps are accounted for, that a measurement system has been defined, and that data will be collected for all inputs and outputs identified as key to the experiment. The planning sheet may contain the following information:

• Background information
• Objective
• Assumptions
• Population under study and potential sub-grouping (covariates)
• Response variables and corresponding measurement techniques
• Factors and the corresponding levels
• Noise or background variables and the corresponding control method
• Design matrix, planned method of analysis
• Estimated Cost
• Schedule and other resources
• Data collection forms

Additional Conjoint Analysis Best Practices to Follow

Other conjoint analyses best practices include the following:

• Plan ahead: lay out the plan before actually executing the analysis
• Define study objectives: write them down to ensure a common understanding
• Remove prohibited combinations (e.g. Honda Prius; Prius is manufactured by Toyota)
• Perform a dry run (random data)
• Trial run the experiment (e.g. take it yourself and with others) before collecting real data
• Identify respondents
• Determine respondent subgroups/covariates
• Number cards (will make entering data easier)
• Limit the total number of cards to less than 30
• Define attributes and levels before collecting data
  • All respondents get the same information
  • Responses are not biased by methodology changes
  • Think about the impact of attribute order on cards
• Sample size can be increased as needed to improve signal to noise

Continuously analyze data, as suitable results may appear early in the experiment, eliminating the need for subsequent trials. 

If the analysis cannot be restricted to less than 30 cards, multiple respondents can be used by dividing the cards between respondents. It is important to minimize variation between respondents if this is done. Figure 2 illustrates this concept. (Click on diagram to enlarge.)

 
Figure 2:  Segmentation of a large conjoint analysis across multiple respondents

Achieve Efficient and Accurate Analysis By Following These Conjoint Analysis Best Practices

Conjoint analysis is a very powerful tool. It can quantify information that is typically considered qualitative. Since the data is derived from people, it is important that special precautions be taken to ensure consistency. Information sheets will improve communication to the respondents. Design of Experiments planning sheets will ensure a common understanding regarding the experiment’s objectives, assumptions, attributes, factors and measurement systems. Following the best practices described in this article will help ensure a more efficient and accurate analysis.