Conjoint Analysis Example: Employment Selection Criteria

Conjoint analysis is a valuable tool for quantifying a person’s opinion or thoughts. The technique is not commonly used and can be confusing for a researcher who has no previous experience with conjoint analysis. This article demonstrates how to set up the Design of Experiments (DOE), define attributes and levels and optimize the design to obtain a manageable number of decision cards.

Basic Steps of a Conjoint Analysis

All conjoint analyses begin with the same basic steps as outlined below:

  1. Define the research objectives
  2. Determine the attributes or features
  3. Determine the number of levels and the size of the study
    1. Increasing factor levels
    2. Increasing factors
    3. Some number examples
  4. Determine logical subgroups
    1. Individual
    2. Group
    3. Region
    4. Job function
  5. Create combinations of attributes
    1. Include all of the permutations that are desired
    2. Remove prohibited combinations
    3. Create cards/profiles Collect subgroup data for future use
  6. Collect subgroup data for future use
    1. Covariates
    2. Between/within subgroups
  7. Plan the data analysis
    1. Appropriately categorize the data
    2. Small data sets—look at each individual
    3. Compare individual to the entire group
    4. Compare logical subgroups to each other
  8. Analysis dry run
  9. Determine sample size and respondent population
    1. Determine logical subgroups
  10. Determine ranking or rating systems/scale
  11. Apply statistical analysis

Conjoint Analysis Example

In this example, we will design a conjoint analysis to understand how potential job attributes impact job desirability. The model will only include main effects and is limited to 15 choice cards with six attributes and levels. The attributes and levels are as follows:

Scope of Operations Job Location Travel Days/ Month Job Focus Number of Assistants Salary
US Only Kokomo, IN 1 day Design 0 15 % < average $85K
World Wide San Francisco, CA 10 days Manufacturing 4 Average $100K
15% > average $115K

Next, the researcher should create a full factorial design. In this example, the number of attributes is six; there are five 2-level attributes and one 3-level attribute. For conjoint analysis, the number of replicates is equal to the number of respondents. For this example, the full factorial has 96 runs. Figure 1 shows a subset of the full factorial. (Click on diagram to enlarge.)

Figure 1: Full factorial DOE

Ninety-six cards are far too many for any conjoint analysis Design of Experiments. The design must be optimized using a statistical software optimizer. Twenty to 30 cards is the recommended maximum number of cards for a single person to evaluate. While the minimum number of cards required is the number of terms selected plus one, it would be advantageous to have 30–50 percent more than the minimum number. Typically, all interactions are removed to reduce the number of cards; however, certain interactions can be selected by removing or adding the desired interactions. Figure 2 shows examples of two cards and Figure 3 shows the data collection worksheet where respondent and covariate information is captured. (Click on diagram to enlarge.)

Figure 2: Cards derived from design of experiment

Figure 3: Data collection worksheet

To make sure the design works, the researcher should enter fictional or random data into the analysis. Next the researcher should analyze the data (do a dry run) to make sure that the conjoint design can be analyzed before a significant amount of time and effort is expended to collect actual data. The data to the left was used to dry run the analysis (Figure 4); the results of the Design of Experiments are shown to the right in Figure 5. (Click on diagram to enlarge.)

Figures 4 and 5: Fabricated data used to dry run the experiment and experimental dry run results

The researcher must now check for prohibited combinations. These are choices that do not exist. For the attributes and levels shown above, could any of these be prohibitive? In this example, a prohibited combination may be a high salary, well above average, in a low cost region such as Indiana. The results for the actual experiment (real data) are shown below in Figures 6 and 7. (Click on diagram to enlarge.)

Figure 6: Actual DOE results

Figure 7: Main effects plots

Best Practices Regarding Conjoint Analysis

The steps below provide best practices to follow when conducting a conjoint analysis:

  1. Create definitions for each attribute and level
  2. Create detailed profile cards
  3. Hand out profile cards to respondents
  4. Have the respondents group the cards into three piles representing preferences and order the cards by preference within each pile
  5. Respondents start with the top-ranked card and rate its attractiveness on a scale from 0 to 100 (0 least desirable, 100 most desirable)
  6. The researcher records and analyzes the data

Why Use Conjoint Choice Over Pareto/Tally Rankings?

If you considered Pareto/Tally Rankings over the use of conjoint analysis, the steps below show the advantages that conjoint analysis provides over the former:

  1. Choice reflects what people do when they make a selection
  2. Choice defines the competitive context
  3. A choice model is immediately useable
  4. People will almost always give their opinions about difference choices

Conjoint Analyis Provides A Powerful Tool For Quantifying Qualitative Data

Conjoint analysis is an excellent tool to quantify data otherwise thought to be only qualitative. As can be seen in this example, respondents’ feelings regarding job selection can be quantified based on their ranking of given combinations of key attributes provided at specific levels. The attributes are assigned P values, thus indicating their significance. In addition, a model can be created to predict respondents’ preferences. This valuable tool can be used to design systems and services to best capture the needs and desires of the customer.