Thoughts

Before a forthcoming pitch, a prospective client for The Opinion Company quipped ….

 “You might want to avoid telling us how many CATI stations and clipboards you’ve got!!”

Well, the particular client knows we’ve never had CATI stations and rarely go down the clipboard route but he wanted to make the point about the approach to gathering customer / market information and how agencies often focus on their in-house resources rather than their ability to derive helpful and actionable insights. (Read ‘death by stale PowerPoint’). 

Is this now a widely shared view of how the research industry (from a client perspective at least) is feeling? 

A quote from a recent read entitled ‘Consumerology - The market research myth, the Truth about Consumers and the Psychology of shopping’ by Philip Graves, perhaps echoes the sentiment….

“I would argue, the convenience of a technique that can be very good at providing research data – very good in the sense of the volume produced and how interesting it seems – has little to offer by way of dependable accuracy”

 Philip goes on to say …..

"Creative questioning styles can provide more interesting responses, but they are not necessarily more reliable. If any of the research techniques used has induced a frame of mind that is not present during the actual consumer experience, it is unlikely to have obtained an accurate picture of what people think."

This perhaps suggests that we need to get closer to the transaction and/or the real time brand experience to really gather meaningful insights – but then what of the post purchase / service feelings? Do store exit interviews not come under criticism for being a little short and rushed due to time pressures and a less than ideal environment generally? Traditional personal data collection modes can be said to give the answer the interviewer wants to hear, introducing additional unwanted bias. Established qualitative groups are at the mercy of (un)scrupulous recruitment to ensure the right individuals are in the room in the first instance and again, we must be ever conscious of direct moderator influence. Certainly for speed and access, online panels can be very effective but can also come under fire for sampling problems, representation and not getting to the why and how questions.   

So where do online communities fit in?  Do they resolve some of the more pertinent concerns of traditional market research?  For example, interviewer influence, environmental issues, timeliness, respondent selection etc. Can they really be used in a more flexible way, over longer periods of brand exposure time and capture valuable insights from naturally occurring conversations versus the interrogatory style of some traditional market research practices? Do quick start up, albeit technically competent, social media consultants offer an easy route into an arguably tired market research industry with their counts of brand mentions and sentiment analytics or are they too missing a trick by precluding and not integrating some of the more established data capture methodologies? 

I’m not sure we are all ready for a broad brush out with the old and in with the new but surely there is room for the twain to meet? Is it time for the industry to embrace and bring together the best of the past in a more holistic fashion and take advantage of the benefits of the new choices open to insight professionals today? 

The human brain is arguably the most complex object in the known universe. We have evolved brains that are superbly adapted to deal with complex environmental situations. But that doesn't mean that our brains don’t have limitations. For a start we are not particularly quick; try doing complex sums faster than a calculator; and our ability to remember large amounts of factual information is often hopeless; do you remember the last number you dialled?

Moreover, we are subject to certain cognitive biases; those really peculiar inconsistencies in thinking that cause us to make irrational decisions and take questionable actions. Here are several of the most common cognitive biases that anyone who measures customer feedback should know and consider when evaluating what customers have said or done.

Before doing this let’s first distinguish between cognitive biases and logic errors. A logic error is a fault in logical reasoning, for example, by basing an argument on factors that are not really related as one assumes. A cognitive bias on the other hand, is a genuine deficiency or limitation in thinking, for example, a flaw in judgment based on mistaken personal frames of reference.

Psychologists have long wondered why cognitive biases exist. Some believe these help us process information more efficiently, especially in dangerous situations. The argument is that you don’t need to consider all factors when escaping from danger; only the most familiar ones or those which the brain decides will lead to the quickest escape.

Still, cognitive biases in other situations lead us to make big mistakes in non life-threatening situations. We may all be prone to such errors in judgment, but at least if we are aware of them we can take them into account when understanding others. Here are some important ones to keep in mind.

Confirmation Bias

People love to agree with people who agree with them. It’s why they tend to associate with others who hold similar views and tastes. We tend to dislike individuals, groups, and situations that make us feel uncomfortable or insecure about our views – a feature labelled as cognitive dissonance. This mode of behaviour results in confirmation bias – the often unconscious act of referring only to those perspectives that fuel our pre-existing views, while at the same time ignoring or dismissing opinions — no matter how valid — that threaten our own view.

In-group Bias

Somewhat similar to the confirmation bias is the In-group bias – an expression of our inborn tribalistic tendencies. It means that we prefer to form tighter bonds with people in our own group and makes us suspicious, fearful and even hostile to others. Ultimately, the In-group bias causes us to overestimate the abilities and value of our immediate circle at the expense of people we don’t really know, who may actually be better able to help.

Gambler’s Fallacy

It’s called a fallacy, but it’s really an error in our thinking. People tend to put a lot of weight on past events, believing that they will somehow influence future outcomes. The classic example is coin-tossing. After flipping five heads in a row, our tendency is to predict an increase in the odds that the next toss will be tails. But in reality, the odds are still 50/50. As statisticians know, the outcomes in different tosses are statistically independent and the probability of any outcome is still 50%.

Positive Expectation Bias

This is closely related to Gambler’s fallacy and seems to fuel many gambling addictions. It’s the sense that our luck has to eventually change and that good fortune is on the way. It also contributes to the “hot hand” misconception, namely, that the next hand of dealt cards will be the winning one. Similarly, it’s the same feeling we get when we start a new relationship that leads us to believe it will be better than the last one.

Post-Purchase Rationalization

Do you remember the time you bought something totally unnecessary or expensive and then you rationalised the purchase so much that you convinced yourself it was a great buy all along. That’s post-purchase rationalization in action — a mechanism that makes people feel better after they make poor decisions; especially at the checkout. It is also known as Buyer’s Stockholm Syndrome.

Probability Neglect

Very few people worry about getting into a car but many of us experience great fear about boarding an airplane. Flying, quite obviously, is an unnatural and seemingly dangerous activity. Yet virtually all of us know that the probability of dying in a car accident is much higher than getting killed in a plane crash — but our brains won’t release us from this unnatural logic. Probability neglect leads us to overstate the risks of relatively harmless activities, while forcing us to overrate more dangerous ones.

Observational Selection Bias

This is the common event of suddenly noticing things we didn’t notice before — making us wrongly believe that the frequency has increased. The classic example of this is after we buy a new car. Suddenly we start to see the same car virtually everywhere. A similar effect happens to women who suddenly notice a lot of other women wearing the same dress as them.  It’s not that these things are happening more frequently, it’s that we have registered the item in our mind, and in turn, are noticing it more often. The problem is that people don’t recognize this as a bias and actually believe these items or events are happening with greater frequency. It also leads to the feeling that the appearance of certain things or events couldn’t possibly be a coincidence, even though it is.

Status-Quo Bias

Most people tend to be wary of change, which often leads us to make choices that guarantee that things remain the same, or change as little as possible. This has important consequences in everything from politics to shopping. We like to stick to our routines, political parties, things we buy and our favourite meals at restaurants. Part of the negative impact of this bias is the false assumption that another choice will be inferior or make things worse. This is one of the main reasons why change in society is so slow to happen.

Negativity Bias

People pay more attention to bad news or events — and it’s not just because we are naturally pessimistic. Sociologists speculate that it’s because we have selective attention and that, given the choice, we perceive negative news as being more important, interesting or newsworthy. We also tend to give more credibility to bad news, perhaps because we’re suspicious of claims to the contrary. Unfortunately we run the risk of dwelling on negativity at the expense of genuinely good news. Consequently even though crime, violence, disease, war and other injustices are generally declining in the world today; most people still argue that things are getting worse.

Bandwagon Effect

People love to go with the crowd. When the masses start to pick up on something our individual minds seem to shut down and enter into a kind of “groupthink” mentality. But it doesn’t have to be a large crowd or a whole nation; it can be very small groups, like a family or even a small group of office workers. The bandwagon effect is what causes certain (sometimes strangely odd) behaviours or norms to propagate among groups of individuals — regardless of their sense or implication. Much of this bias stems from our integral desire to fit in, conform and obtain the approval of others.

Projection Bias

As individuals it is very difficult for us to look outside of our own consciousness, preferences and experience. We tend to assume that most people think just like us — though there may be no reason for it. This bias often leads to a related effect known as the false consensus bias where we tend to believe that people not only think like us, but that they also agree with us. It’s a bias where we overestimate how typical and normal we are, and assume that a consensus exists when there may be none. Moreover, it can also create the effect where the members of a radical or fringe group assume that more people on the outside agree with them than is the case.

The Current Moment Bias

People have a really hard time imagining themselves in the future and altering their behaviours and expectations accordingly. Most of us would rather experience pleasure now, whilst deferring the unpleasant for later. This bias is of particular concern to economists in the current financial climate, typified by our unwillingness to spend money now and saving for later. But if e don’t spend how can we stimulate the economy? Another example is food. In a study looking at food choices for the coming week, 74% of participants chose fruit. But when the food choice was for the current day, 70% chose chocolate.

Anchoring Effect

This is a tendency to compare and contrast only a limited set of items. It’s called the anchoring effect because we tend to fixate on a value or number that in turn gets compared to everything else. The classic example is a sale item in store; we tend to see (and value) the difference in price, but not the overall price itself. This is why some restaurant menus feature very expensive entrees, while also including more (apparently) reasonably priced ones. It’s also why, when given a choice, we tend to pick the middle option — not too expensive, and not too cheap.

Decision Event Bias

People find it really difficult to imagine when they may do something that is not routine or urgent or when they haven’t already committed to doing it. Thus asking them what they might do and when they might do it, nearly always lead to the wrong answer. An example is asking someone if they are likely to buy a new car and how likely they are to buy it in the next three, six or twelve months. Unless the person is already considering this action and is actively engaged in the decision-making process they will usually underestimate the length of time, pointing to an earlier demand where there is none.

Social Desirability Bias

This is the tendency of people to respond to questions in manner they think will make them more acceptable or pleasing to others. It can take the form of over-reporting "good behaviours" or under-reporting "bad" or undesirable behaviours; especially in the context of what they believe are socially acceptable norms. An example of this is sexual or stimulant-taking activity. Whereas men tend to inflate the numbers, women tend to underestimate theirs. In either case, the results from both groups are likely to be highly distorted by this bias.

Getting your marketing strategy right has never been more important than right now. The economic climate means that creating the right market insight is vital to targeting the right markets and customers. Yet for many organisations such insight remains elusive. In this article we try to highlight the most common insight mistakes and what you can do to avoid them.

1.Basing everything on Demographics

Demographics have been proxy for virtually everything - starting with radio and television metrics - because there was no ability to look at data by individual or household. It was a massive extrapolation system. After 70 years of this it's time to retire it. When so much personal information is available there is no need to use anything this inefficient anymore.

2.Assuming that correlation means causality

Correlation is often thought of as evidence of cause and effect. But this is rarely the case - an action or occurrence can cause another (such as smoking causes cancer), or it can correlate with another (such as smoking is correlated with alcoholism). If one action causes another, then they are most certainly correlated. But just because two things occur together does not mean that one caused the other, even if it appears like they do.

3.Thinking that people have the ability to explain the reasons for their decisions

A core belief of market research is that people are capable of explaining their reasons for past decisions/choices and predicting what they will do in the future. However the bare facts from the field of Decision Science shows us that the processes underlying decision-making are implicit, non-conscious, and there is no introspective access to them. 

4.Using outdated consumer models

Basing metrics and business decisions on consumer behaviour and communication models which were formulated in the 60s and 70s and which are now patently wrong. So much has been learned since then about the science behind why we buy what we buy and how advertising really works - yet this is not yet mainstream knowledge or practice.

5.Confusing  customer satisfaction with loyalty

Customers can be completely satisfied with your product yet have no loyalty whatsoever to your brand. I was extremely satisfied with my Mercedes for many years, but didn't seriously consider them for my next car because I had no loyalty to the brand and felt Toyota was making a product better suited for my needs. Yet I would have been top box on virtually any satisfaction survey Mercedes could have given me.

6.Not using insight you already have

The classic mistake is not doing enough with what you already know, and assuming that each issue needs a brand new solution. Sometimes that is the case, but making that assumption every time is a total waste of time, money, experience, judgement and sets up unreasonable expectations of break-through thinking.

7.Accuse respondents of lying

The reality is that 99% of the time respondents are doing their best to help. The reason they give unexpected answers is that you are asking the wrong questions or waiting for them to do things that cannot be done in their situation. For example, they may be asked to describe the steps in a making a decision but there is a built in fault in this line of questioning given that the part of the brain governing recall and actual behaviour are different. 

8.Expecting consumers to be graphic designers, name experts and copy editors.

When testing names or logos, it's not unusual for clients to say something like, "Why don't we ask them what they think a good name would be?" These are plumbers and teachers and dentists and shop assistants. Naming companies get millions of pounds to come up with names but do you really think John the Tesco employee is going to come up with the new name for your product on a whim? Really? 

9.Size of the Prize

This is the one where companies over-estimate the consumer demand for their product. They start by asking consumers whether they will buy the product and then extrapolate the answers to work out demand. The approach has two major flaws. First, many factors come into the purchase decision, so asking the question in isolation ignores issues such as actual need and price. And second, consumers don’t want to cause offence so often over-claim their interest or desire for a product.

10.Unwillingness of management to leave their own biases 

Probably the most subtle of insight mistakes made by businesses is the inability or unwillingness of management to leave their own biases out of the interpretation of the insight...perhaps because they don't want to fully commit to making the changes necessary to leverage that insight, or because they don’t want to expose their lack of knowledge or experience of how to do so, or simply because it is easier to do nothing than to take action.

Everyone talks about insight being desirable and valuable, but all too often people have difficulty defining what insight actually is. Indeed, the word has been used or abused so much by the research industry, depending on your point of view, that it has become a cliché in itself. So does anyone actually know what insight is and when it has been collected and delivered?

In a bid to crack this question we have talked to a number of clients to see what they perceive insight to be. Based on these discussions we think we have identified the key elements of what constitutes good insight.

Before we start, however, it is important to accept that insight is often not a direct output from market research. Instead insight follows on from thinking about what the research generates. Consequently the way research is designed and conducted can make it more or less likely to contribute to insight generation. The way a study is structured, what is done during the research and what happens to the output can all influence the value and insight that comes out of the research. Therefore whenever insight happens to be the objective of the activity, it is vital that the role of research in its creation is considered accordingly.

Having got this preamble out of the way, here is what we believe insight should be in the true meaning of the idea:

1.Insight is ultimately about guiding and fulfilling the business strategy. It’s about the end-game and all the eventualities that may play out in between.  It means having the right knowledge to know which path to take and what to do in case the direction changes. 

2.Insight must be actionable. It must lead to an action that can be taken by the business. It’s lovely to know things, but this knowledge is wasted if the business can’t do anything about it. Any study must therefore be designed with actionable outputs in mind. Asking the right questions at the start helps.

3.Insight must be engaging. Can the insight tell a story? Can it make an audience sit up and listen? Can it find a way of sparking off something in the audience that is inspiring? The way that insight is reported must therefore focus on involving the audience, not just presenting numbers. 

4.Insight needs to be bold. It needs to challenge and question the direction and the way of getting there. Successful businesses look for strategic directional input, not just short-term tactical recommendations.

5.Insight needs to be nimble, flexible and proactive. It needs to solve problems, anticipate challenges and suggest ways forward that haven’t necessarily been considered. It needs to reflect an understanding of the rapidly changing contexts and landscapes that all businesses face.

6.Insight needs to be credible. If it’s going to be believed and acted upon it has to be robust and not be easily knocked down. That means that the insight must have a clear trail of how it was derived and developed.

7.Finally, insight needs to be memorable, so that it’s spread quickly around the business. Therefore it needs to be simple and easily understood by anyone who comes across it. 

We hope that the above requirements of effective insight are useful. Certainly, whenever we work with any of our clients, we use them as the underlying principles for our own performance.