by Philip Graves
A book review by Duncan Brett
Publisher: Nicholas Brealey Publishing; 2nd Revised ed (including a new preface) edition (January 24, 2013)
Length: 241 pages
About the Author
Philip Graves is an ex-researcher who turned to psychoanalysis to try understand why people say one thing and then do something entirely different.
If you would like to know a little bit more about him, you can visit his website http://www.philipgraves.net/
Graves take on traditional market research is that it is a load of rubbish. That’s it. No holds barred. You should rather covertly analyze behaviour in the actual environment.
Well, to expand a little, market research is based on a false premise, that people can accurately tell you why they think the way they do. Asking them is not only a waste of time, but can actually corrupt the thinking process. You are better off spending your money on covertly observing people and conducting live experiments. Companies are mistaken in relying on research to reduce risk or predict the future. Don’t get him started on his most hated institution, the focus group.
The book is well researched, lists many valid flaws and backs up his argument with examples and case studies. If you think market research is a waste of time, this book will appeal. If you work in a traditional research agency, this will make for depressing reading.
However, reality is probably a little more nuanced, research is more than just focus groups and trackers, and one would expect experienced researcher to be all to aware of the flaws. . There are a multitude of research types, and objectives and some work well, others not at all.
I recommend researchers read it, sometimes you need to be open to criticism, even if it is hard.
For those not familiar with behavioural economics, the first chapters will act as a primer to better understanding of how the consumer mind works, although there is little new to students familiar with the topic.
The suggested solutions are worth considering, but fraught with difficulty themselves.
Consumer.ology focuses entirely on consumer behaviour in a retail context, although his premise could be applied to other areas.
The book is structured into three main parts. First, a tour of how the consumer mind works. Secondly, a critique of current market research techniques, and finally a suggested remedy.
Graves makes liberal use of examples to support his argument. This includes a wide range of psychological studies, and research disasters such as the Millennium dome in London and New Coke, as well as near misses such as Red Bull (which did not follow the research advice not to launch).
The consumer mind: Graves takes us on a short tour of our current understanding as to how the human mind works. He draws on the fields of psychology and behavioural economics here. He argues that human action is subjected to enormous influence by the unconscious mind, to which our conscious mind has no direct access, and is completely unaware. We make terrible witnesses to our own actions. We are hopeless at describing what we do and don’t want.
We are not aware of the subliminal influences and delude ourselves with credible post rationalizations to explain our actions. These myths are completely false even though we believe them. The more established our behaviour the more likely it is that the unconscious is taking charge.
Market research makes a critical error in believing that you can elicit true responses from people by asking them questions.
This area is well known to students of behavioural economics and those wishing to learn more would do well to read books such as Daniel Kahneman’s Thinking, Fast and Slow; Leonard Mlodinow’s Subliminal; Dan Ariely’s Predictably Irrational; and Barry Schwarz’s Paradox of Choice.
The most important biases or influences include:
- Loss aversion, where humans will try to avoid loss more than seek gain.
- Easy wins, where humans will always look to the easiest option mentally
- Social proof, where we rely on others for affirmation, confirming us as sheep rather than pioneers
- Priming, where our initial experience sets us up for our later response. It is an inevitable consequence of the research process and hugely problematic
- Confirmation bias, where we only see what we want to in order to justify our original decision
- Context is fundamental, environmental influences such as lighting, music, layout and human influences such as who we are with, or the salesperson, act as hugely influential subliminal nudges.
Market research creates an artificial situation which strips out all these critical influences. As a result the answers are given entirely out of context, which explains why people say one thing and then proceed to do something completely different.
A critique of current research methods covers the flaws inherent in asking questions, irrelevant answers, and the terrible institution known as a focus group.
Even worse, research does more than just ignore critical components why people behave in a certain way, it actually changes the way they think, by creating a focus that does not exist, and then attaches a process that shapes the outcome. As a result, research is not just wrong; it is misleading.
Graves raises many valid points, and powerfully illustrates it with liberal use of examples, including the millennium dome in London, the launch of New Coke, and the failure of research to identify opportunities such as Red Bull.
Perhaps the most problematic area is of the book is that he seems not to acknowledge that the points he raises are well known to experienced market researchers, and one could argue that many of the failures can be attributed to completely unrealistic expectations about what exactly market research can deliver. He fails to touch on the issue that market research needs to have very specific objectives, which are grounded in reality.
The industry is well aware the market research cannot accurately predict the future. No one can. Especially not on a tight deadline with a low budget. Just because some charlatans claim they can, and some fall for it, doesn’t make the whole industry rubbish.
He blasts market research as a pseudo science, acting to seduce organizations with statistics with the promise of apparent consistency.
Graves does appear to regard the industry as a giant question and answer session, but market researchers would do well to acknowledge his points, and recognize that bad and inappropriate research can contaminate the image of the whole industry.
The problem with questions.
Graves believes we should avoid asking questions, it not only ignores critical components, but actually changes the way people think. Problems include:
- Questions inadvertently tell people what to think about
- Questions change what people think, by priming for the next question
- They inadvertently lead the witness
- They act as an accidental sell
- They may inadvertently persuade people to like something by the very act of asking them to conceptualise the message
- They artificially deconstruct the customer experience
- Questions may artificially re-inforce existing opinions
- There is little correlation between attitude and behaviour
- Questions can invite the wrong frame of mind – most research ignores the variation in frame of mind
- How easy or difficult the question is to answer affects things
- People are inclined to tell the researcher what they think the researcher wants to hear
If you have to ask questions, don’t take the answers at face value. Graves advises you take note of the following:
- Ask questions about behaviour and not post rationalized thoughts and feelings.
- Carefully consider the sequence of the questions.
- Makes sure the respondent is unaware of the research subject
- Explore unconscious elements
- Time and place matters. Ask as soon as possible, in the environment the consumer acts in, preferably after observing what they did so you can validate it.
- Scrutinise the words used in the answer
- Only pay attention to instantaneous answers
- Make sure you ask the right mindset. This is an interesting point and it is worth thinking about. It builds on the work of Eric Berne with respect to transactional analysis and the different states of mind (Parent, Adult, Child) and how it is important to put the respondent in the correct state of mind.
- Look for contradictions between behaviour and attitude
The terrible institution called focus groups
Graves regards the focus group method as deeply flawed. He cites numerous reasons why focus groups don’t work.
- People can’t help copying others – in low importance topics that focus groups cover, people just go with the conversational flow
- People change their mind to fit in with the group – people devalue their own opinion to fit in with the group. We can’t help but care what the group thinks.
- People agree with the prevailing majority – as a result groups produce an unrealistically unified view, often driven by one or two people.
- The act of discussing changes attitudes – focus groups may create the attitude they report on rather than reflecting prevailing views.
- An overly persuasive voice – repeating one view several times is almost as influential as several people making the same point independently.
- Group thinking can be dangerous – decisions are made with insufficient critical analysis and too much deference to the group view.
- The terrible, terrible viewing facility – which are created for convenience sake and are as far as one can imagine from a natural environment. Graves also cites medical studies to show that the mere fact that you know you are being observed can invalidate the results of a clinical trial.
- On top of it all recruitment is often poor, and unrepresentative
If the above has not convinced you then Graves returns to reiterate that market research is an exercise in wishful thinking. It will only guarantee an answer, not an accurate one.
Market research routinely claims to predict the future, which it can’t. People can’t tell you why they do what they do, let alone what they will do in the future.
And, here comes the solution….
When testing a new idea, there is nothing better than a live test, and to observe people in their natural environment. Graves paraphrases Edgar Allen Poe, “ believe nothing you hear, and only one half of what you see”…and almost everything the sales data says they have done.
He argues that at its most basic level, behavioural data is truth, it won’t tell you why but it will accurately tell you what is happening. To understand the why requires inference, but at least you are focusing on the right area.
To study consumer behaviour by observation you need to look out for the following:
- The observation must be covert – people must not know they are being watched
- To avoid confirmation bias (seeing only relevance in observations that fit our pre-conceptions) the observation phase must be separated from the inference phase
- Physical behaviour – the way people walk and touch things is revealing
- The way we make choices – generally we are more likely to buy when subsets are small
- Follow the eyes – where we look first is the prime for everything that follows
- Observe emotions – observed emotions are useful, but only exist in the live moment. Graves does not deal with the difficulties of accurately assessing emotion purely by observation particularly on a large scale.
As an example Graves, argues it would be better to observe a statistically valid sample of service interactions in the flesh, than to conduct customer satisfaction survey that is a post-hoc, post rationalized process about a fleeting experience that occurred days ago, primarily in the unconscious.
Companies will succeed by how astutely they understand responses to what they are doing, and how quickly they can evaluate and react.
To assess how much faith a company can have in its consumer insight, Graves has developed an evaluation tool under the acronym of AFECT.
It is explained as follows:
Analysis of behavioural data – is it information about what consumers do? Sales data and behavioural observation inspire the most confidence. Research that is derived on a behavioural focus is more reliable than conscious attitudes or feelings.
Frame of mind -Covert observation in the relevant environment is in the right frame of mind. Overt research conducted remotely from the environment will be at odds with reality. Where the research has been encouraged by an artificial mindset (such as a focus group), you may has well have asked the wrong subject.
Environment – If you have not got behavioural data, then at least do it in the right environment.
Covert Study -Where the purpose of the research is apparent, it increases the chance of influencing the results.
Timeframe– Instantaneous fast responses are more dependable.