Prof.
“Seenu” Srinivasan Optimal Strategix Group, July 2, 2020
Adaptive Self-Explication of
Multi-Attribute Preferences (ASEMAP)
6
Part-worth
Part-worth
5
4
2
0 1 0
1
6 9 12 15 0 1 2 3
Resolution (megapixels) Warranty (years)
Within – attribute information
Across – attribute information
1
Self-Explicated Approach
Example:Resolution Part-worths
10 6
Desirabilities
Part-worth
8 4.8
X 60% =
0 2 1.2
0
6 9 12 15 6 9 12 15
Resolution (megapixels) Resolution (megapixels)
Desirability Ratings X Importance = Part-worths
2
ASEMAP Step 1 – Relative Desirabilties
Ratings of levels of each attribute, one attribute at a time
How desirable to you are the options below for Size
relative to each other? Assume that all other
attributes are held constant.
E.g., Fannypack o o o o o o o o o o o
0 10
Lo Relative Desirability Hi
For ordered attributes (preference ordering of levels known)
10 and 0 are pre-filled for most and least preferred levels.
Two level ordered attributes are skipped from this step.
3
Measurement of Attribute Importance
Rating scales (Self-Explicated (SE) Approach)
Respondents tend to say that every attribute is
important thereby reducing its usefulness
4
Sawtooth – Adaptive Conjoint Analysis (ACA)
o Start with the Self-Explicated Approach. Augment it with 15-
21 (adaptive) paired comparisons similar to below:
5
Constant-Sum Method
(Improved method for measuring importances)
Constant-sum method
Allocate (say) 100 points across attributes
Levels
Attribute Importance
Least Most
Attribute a Level l Level m
Attribute b Level l Level m
… … …
_______________
Total 100
Information overload when number of attributes is large.
6
Attribute Importance – Ranking
ASEMAP Step 2 – Ranking attributes by Importance
7
ASEMAP- Step 3: Constant Sum Paired Comparisons
Which of the improvements below is more valuable to you? How much more?
Assume that price and all other attributes remain the same.
Resolution (6MP 15MP) 55
50
60
Warranty (None 3 Yrs.) 45
50
40
ASEMAP Third Step:
Adaptive Measurement of Attribute
Importance – Initial Step
(Example – 12 Attributes)
Three paired comparisons of attributes
1 vs. 12 (Most important vs. least important attribute)
1 vs. 6 (Most important vs. middle importance attribute)
6 vs. 12 (Middle important vs. least important attribute)
Estimate the importances v1, v6, and v12 by log-linear
multiple regression. (Details on the next page.)
9
Log-Linear Regression Estimation
of Attribute Importance
Example: V1/ V6 = 2, V6/ V12 = 3, V1/ V12 = 5 (some inconsistency)
(Log V1 ) – (Log V6 ) = Log 2 = 0.30 (Log is taken to the base 10)
We can take without loss of generality, V1 = 100 so that Log V1 = 2
Thus 2 – (Log V6 ) = 0.30, so that – (Log V6 ) = 0.30 - 2 = -1.7
Similarly (Log V6 ) – (Log V12 ) = Log 3 = 0.48, and
(Log V1 ) – (Log V12 ) = Log 5 = 0.7, so that – (Log V12 ) = 0.7 – 2 = -1.3
10
Log-Linear Regression
The three equations in the previous page can be written in matrix form as:
Log V6 Log V12 Dep. Var.
-1 0 - 1.7
1 -1 0. 48
0 -1 -1.3
This can be thought of as a multiple regression with three observations, two
independent variables, and no intercept. The two regression coefficients are
Log V6 and Log V12 . Adjusted R-squared gives a measure of the consistency
of the data. Taking antilogs, i.e., 10 raised to the power of the regression
coefficients, we get V6 and V12 . We had set V1 = 100.
Given V1 and V6, we can interpolate the values for V2 ,V3 , V4 , and V5.
Likewise, we can interpolate the values for V7,V8,…, V11 from V6 and V12 .
Normalize all V1 ,V2 , …, V12 to add to 100.
11
Adaptive Measurement of
Attribute Importance (ASEMAP)
Suppose we have measured the relative importance of the most (V1),
middle (V6), and least important (V12)attributes.
Consider the following two scenarios:
Scenario A Scenario B
Attribute Attribute
Importance Importance
1 6 12 Rank 1 6 12 Rank
I II I II
In scenario A determine importances of an attribute in interval I
In scenario B determine importances of an attribute in interval II
12
Criterion for Selecting the Interval to Explore
Each interval has a top attribute, a bottom attribute, and
one or more intermediate attributes.
Importance Worst error
scenario
Linear
Interpolation
Top Intermediate Bottom
Attributes in the interval
Difference in importance # of
Maximum Possible error =
between the top and bottom X intermediate /2
attributes attributes
13
Step 3 (Contd.) Adaptive Paired Comparisons
Suppose the interval between attributes 1 and 6 is the chosen. We choose
an attribute in the middle of the interval to minimize the maximum
interpolation error.
Suppose attribute 3 is chosen. The respondent provides data for two
additional paired comparisons (1, 3) and (3, 6).
We combine the original three paired comparisons with the two additional
paired comparisons, for a total of five paired comparisons.
This leads to a multiple regression with five observations and three
independent variables, resulting in regression coefficients for V3, V6 and V12.
Knowing V1=100 and V3, V6 and V12, we interpolate for the remaining
importances. Normalize all 12 importances to add to 100. 14
Iterative Method (ASEMAP)
Suppose we have the following attribute improvement rankings :
Attribute rankings Pairs 1-3 Pairs 1-5 Pairs 1-7
Resolution 21.36 25.96 23.32
Optical zoom
Price 12.70 11.72
Shot delay
Video Clip
Battery life 8.48 7.98 7.57
Warranty
Brand
Camera size 5.08
LCD size
Memory
Light sensitivity 0.70 0.76 0.62
15
Step 3 (Contd.) Adaptive Paired Comparisons
The number of paired comparisons is chosen so as to keep the interpolation
error to a small amount.
Using computer simulation (and consistent with empirical evidence) we
recommend:
# of attributes # paired comparisons
<= 10 9 ( for 3,4,5 attributes: 3, 5, 7 pairs)
11 to 15 11
16 to 50 13
Under these guidelines, the interpolation error is less than 5%,
i.e., average interpolation error / average importance <= 0.05.
16
ASEMAP: Step 4 (Linking Utility
to Likelihood of Purchase)
Market researcher chooses the number of most important attributes
(e.g., 6) out of all attributes (e.g., 12)
Respondent is shown five profiles on only his/her most important
attributes and asked for his/her likelihood of purchase if that profile
were the best available product/service in the market (assuming that all
remaining attributes are at their best levels). The five profiles are:
(1) all important attributes are at best levels
(2) half the important attributes are at best, the other half at medium
(3) all important attributes are at medium levels
(4) half the important attributes are at medium, the other half at worst
(5) all important attributes are at worst levels
17
Linking Likelihood to Total Utility
We can determine the total utility U for each of
the five profiles from the utility function
determined from stages 1 through 3.
Likelihood L is linked to total utility U by the
s-shaped curve:
Ln[L/(1-L)] = b0 + (b1*U); five observations
Delete the respondent if
Logit adjusted R-squared < 0.25
18
Empirical Comparison
Digital cameras with 12 attributes
Respondents randomly divided into two groups:
ASEMAP with up to 21 paired comparison questions (mean - 18
questions)
ACA with 10 two-factor paired comparisons and 11 three-factor paired
comparisons
Procedure:
Validation task -- ASEMAP/ACA -- Post survey evaluation
Validation task includes 2 (out of 4) choice sets with 4 alternatives each
ASEMAP (n = 52); ACA (n = 49)
Survey duration not significantly different between ASEMAP and
ACA methods (each averaging 15 minutes)
19
Research Design
ASEMAP 2 Validation Attribute level Rank order Paired Post Survey
Choice Sets Desirabilities of Attribute Comparisons of Evaluation
Importance Attribute
Importances
ACA 2 Validation Attribute level Attribute Paired Post Survey
Choice Sets Desirabilities Importance Comparisons of Evaluation
Ratings Partial Product
Profiles on
2 & 3 Attributes
SE ACA Data
20
Prediction Accuracy
Percent of (first) choices correctly predicted
ASEMAP 60.6 % (Adaptive Self-Explication)
ACA 39.8 % (Adaptive Conjoint Analysis)
The improvements are statistically significant (p < .01)
50% improvement in predictive accuracy of choices
21
Does ASEMAP Provide a Significant
Improvement over (Standard)
Self-Explicated Procedure?
ASEMAP 60.6% (Adaptive Self-Explication)
SE 44.9% (Self-Explication)
The improvement is large and highly statistically significant (p < .01)
22
Does Hierarchical Bayes Provide
Significant Improvement?
Individual Hierarchical
Estimates Bayes
ASEMAP 60.6% 64.4%
ACA 39.8% 40.8%
HB marginally improves the prediction ability of all methods
ASEMAP/HB significantly outperforms the ACA/HB
23
Choice Share Prediction
*
Mean Predicted Choice Share – Actual Choice Share
Individual Hierarchical
Estimates Bayes
ASEMAP 0.067 0.068
ACA 0.113 0.122
SE 0.082 N.A.
•Mean absolute deviation averaged over four brands in each of four choice sets.
•Smaller numbers are better.
24
Effect of # of Paired Comparison Questions
(Adaptive Self-Explication)
ASEMAP
# of Pairs Hit Rate
0 43.7% Uniform Importances
1 51.9%
3 57.7%
5 60.6%
7 60.6%
9 61.5%
11 63.5%
13 64.4%
15 62.5%
17 60.6%
19 62.5%
21 60.6%
25
Attribute Importance:
Coefficient of Variation
(= Std.dev./Mean)
Individual Hierarchical
Estimates Bayes
ASEMAP 0.849 0.728
ACA 0.487 0.430
SE 0.403 N.A.
ASEMAP shows more variation in importance
HB “shrinks” the importance variation
26
A Replication of the Previous Study
Comparing ASEMAP-C to ACA
Replication to the Laptop category
14 Attributes, n= 60 per method
Choice Set Hit Rates:
ASEMAP = 54.2%
ACA = 46.2%
ASEMAP statistically significantly better (p <.05)
27
Comparing ASEMAP-C to ACBC
% hits (first choices correctly predicted)
Headphones Laptops Average
ASEMAP 56.2% 64.1% 60.1%
ACBC 52.2% 65.0% 58.6%
12 attributes for each product category; 150 respondents/cell
5-6% reduction in time taken by the ASEMAP survey
Differences are not statistically significant
28
(ASEMAP-I vs. Constant Sum)
Marketing Science Institute Prioritization of Research Topics
15 topics, n = 160 managers/method
Average percent of pairs correctly predicted:
ASEMAP = 81.6%
CSUM (Constant Sum) = 59.6 %
ASEMAP statistically significantly better (p <.05)
29
Comparing ASEMAP to ACBC
% hits (first choices correctly predicted)
Headphones Laptops Average
ASEMAP 56.2% 64.1% 60.1%
ACBC 52.2% 65.0% 58.6%
12 attributes for each product category; 150 respondents/cell
5-6% reduction in time taken by the ASEMAP survey
Differences are not statistically significant
30
Other Advantages of ASEMAP
ACBC is limited to 12 attributes; If you do a study with 20 attributes, it will zero
out the importances of each respondent’s less important 8 attributes.
Sawtooth recommends Adaptive Conjoint Analysis (ACA) for more than12
attributes. ASEMAP predicts better than ACA by 14.2 percentage points on
average (across two product categories, as reported earlier.)
The importances and part-worths are immediately available as soon as each
respondent takes the ASEMAP survey; in ACBC you need to wait to get all the
data, do a hierarchical Bayes analysis (additional researcher time). This is an
important advantage for connected surveys.
ASEMAP provides an unbiased estimate of each respondent’s preferences;
ACBC provides a biased estimate because of the need to conduct Hierarchical
Bayes estimation, thereby biasing benefit segmentation results.
Comparing ASEMAP-I to MaxDIFF: #1
Presidential priorities (prior to 2008 elections)
17 topics, n = 102 /method
Average Absolute Error (based on a constant sum of 4 topics)
Smaller numbers are better
ASEMAP = 8.73
MAXDIFF (Maximum Differences Scaling) = 11.31
ASEMAP’s improvement over MAXDIFF = 22.8%
ASEMAP statistically significantly better (p <.05)
32
Comparing ASEMAP-I to MAXDIFF: #2
Mean Absolute Error (Smaller the Better)
Mean Absolute Error of Prediction (MAE)
ASEMAP-I 11.5 (32.5% Improvement)
MAXDIFF 17.0
MAE difference is highly statistically significant (p < .001)
Context: Importance of Issues for voters (2020 Democratic Presidential
Primary in the U.S.) Approx.155 respondents/method; ASEMAP-I takes
a little longer time (2 min. 51 sec. cf to 2 min. 14 sec. for MaxDiff.)
33
