Last week I invested US $12,000 in new software. That’s a ridiculous amount, and frankly worth about three times more than my car. Strangely, the decision to invest in this software was very simple. It was fuelled by an ideal: am I interested in doing what I do, to the highest standard that I can? The answer is an idealistic yes. Idealistic for sure. My financial adviser and good partner for 34 years just shook her head.
The software, by the way, included Sawtooth’s well-known Max-Diff module as well as their pricey but promising MBC module which takes choice modelling to a whole new level. In terms of learning new technology, MBC promises to stretch me to the limit. It is neither intuitive, nor pretty. But more about that in a later blog.
As soon as I had Max-Diff out of the box I used it on a client survey as part of a conjoint exercise. I’ve long been a fan of conjoint because that emulates realistic decision situations that people face in real life. We learn not just what they choose, but the architecture of their decision-making as well.
In this case I could compare the two approaches. In effect, the conjoint choice modelling and the Max-Diff exercise ran in parallel, testing more or less the same variables, but using different approaches.
With conjoint the respondent chooses (on-line) from a small array of cards, each with a different combination of attributes and features. They select the most optimal.
With Max-Diff the respondent chooses their favourite combination, as well is the least favourite.
Were the results similar? Well yes, they converge on the same truths, generally, but the results also revealed telling differences. One of the least important attributes, according to conjoint, proved to be one of the most important attributes according to Max-Diff. How could this be?
The lesson went back to some wonderful insights I learned from Alistair Gordon when we were working on the subject of heuristics – those rules of thumb that people use to evaluate complicated choices.
Most of us, when asked “how do people make choices?” figure that mentally we prepare a list, based on the variables, and we set about finding the best: in fact conjoint is predicated on exactly this process.
But Alistair introduced me to a fabulous concept: the veto rule. Put simply, if I was choosing between one brand of breakfast cereal and another, I may have a number of variables that contribute to optimality (Flavour, naturalness, organic-ness,) and no doubt my brain has worked up a complex algorithm that balances these things against the presence of raisins, puffed wheat, stone ground oats and dried apricots. Good luck trying to model that!
But I also have a few simple veto rules. If a competing breakfast cereal contains more than x% of sugar, then bingo – I drop it from the list of competitors.
This explained why some variables scored as important with Max-Diff, but scarcely registered with conjoint. Among the variables were a few conditions that might be described as veto conditions. Those who use Max-Diff alone seldom discuss these different effects.
So which approach – conjoint or Max-Diff – should one use? As ever, I think one should try both. My favourite research metaphor is about the blind men and the elephant, each discovering a different aspect of the animal, and each giving a different version of events. They are all correct, even if they have different answers. Together, they converge on the same answer: the whole elephant.
I do like the way research tools can give us these honest, statistically reliable, yet conflicting answers. They give us pause for thought, and they highlight the fact that numbers are merely numbers: quite useless without confident interpretation.