Backwards in a forwards World

Backwards in a forwards World

Would you drive your car with a blacked-out windscreen and the only way of steering ahead would be by looking in your mirror? Not likely! But this is just the case with tracking studies. 

Whilst trackers are the staples of many research departments, we recently spoke to some leading CMO's what they really thought of them. Their replies were far from flattering. Here is what they said: 

Backward looking – trackers are great at telling you what happened in the past, but they have little or no predictive power; no way of telling you where you are heading. It’s rich information but of limited future planning value. 

Lack of causality – businesses love targets and KPIs; and trackers deliver these aplenty. But, it is often impossible to show a causal relationship between a given KPI and actions taken by the business. We’ve seen trackers where moves in the headline KPI can't be explained by any measure on the tracker. Consequently a businesses with low scores has absolutely no idea what they should do to improve them.  

Inconsistent metrics – many trackers have grown piecemeal over time, adding new products, services and measures as they go along. Apart from making these surveys really tedious for respondents to complete, it also means that individual elements are measured using different questions and using different scales - defying all attempts to allow for comparisons between the company’s products, brands or services. 

Too much data and too little insight – We all recognise hundreds of charts and tables, piling up quarter after quarter, simply because there is so much information that any insights are too hard to find to be worth the time spent searching through the vast numbers. 

Too accurate – In the quest for ever greater statistical accuracy trackers abound with data to umpteen levels of significance, but no one really questions the purpose of this accuracy nor gives thought to the enormous sample sizes needed to collect this amount of data. Often a much smaller sample will be enough to serve as the basis for a decision; few business decisions are made to this level of exactness. 

Poor flexibility – most trackers are highly complex with multiple routes through the questionnaire depending on customer type and touch-point. It means that even small changes become difficult and often lead to errors. Changes are therefore expensive and very time consuming; reasons why many trackers are not adapted to the changing needs of the business. 

Expensive – the nature of most trackers means that they are hugely expensive. Long questionnaires, large sample sizes and poorly chosen data collection methods mean that costs rapidly mount up. Great for the research agency but poor use of capital which could be used more effectively elsewhere.  

So why do companies continue to use tracking studies?  

Most often it’s down to inertia. They do tracking because they've always done it. Sometimes it is necessary to show the CEO some form of quantitative evidence that marketing is working; another number for the dashboard. Creative agencies like it because they can ‘prove’ the effectiveness of their campaign by pointing to a spike in data. And managers like it, because flat lines mean they probably don’t have to do anything and carry on as normal. 

We believe it’s time to change.

Brands need more effective ways of collecting customer information and smarter ways of understanding what is being said, why it’s being said and what it means for the business. What’s needed is a radically different approach, based on a much more holistic view of the customer and their brand experience, using methods that collect feedback from places visited by, and convenient for, the customer and not intrusive questionnaires.

That’s why online customer communities are ideal for this. Listening to detailed conversations will give your business an instant pulse of how well you are doing. If you need to ask detailed or complex questions you can, because community members are people who are interested in taking part.

Now when you have your key insights, you can quantitatively validate them using more traditional surveys to ensure they are representative of the wider market.

Do you agree? 



Rate this item
(0 votes)