Tag Archives: text analysis

Here’s why I think Market Research companies are flying backwards.

Image
Yes, technically we’re flying, but only just. Kitty Hawk thinking doesn’t help high-flying clients.

The very last meeting I attended when I was with my previous employer did not go particularly well. It was the weekly meeting of senior execs and on the agenda were things like staff training, client account management, and the usual HR concerns. I already knew that I was going to be leaving the organisation, so I wanted to end by contributing something which I felt was very important. “I want to run some staff training on SPSS.”

I didn’t expect the answer from my senior colleague. She asked simply: “why?”

I was quite taken aback, and I must admit I fumbled for an answer, mumbling things like; “extracting more value from the data we collect.” I can’t remember my exact words but they were along those lines. Anyway the workshop suggestion got kiboshed, and I left the organisation a few weeks later.

Three weeks ago one of the bigger clients of that same organisation announced that they were getting SPSS in-house, and they introduced me to a new member of the team, a particularly bright analyst who had previously worked with the banks.

I realised I had just witnessed a well expressed closure to my fumbling argument from 18 months earlier. The reason we need to smarten up our analytical skills as market researchers is because if we don’t,  we will simply get overshadowed by our clients.

In fact I see this all over the place. In-house analytical units within banks are using SPSS and SAS quite routinely, and most of the major banks have adopted text analysis software to help them swiftly code and analyse their vast streams of incoming verbal feedback. The text analysis centre-of-gravity in New Zealand is located client side, and not on the side of the market research industry. The same could be said of Big Data analytics. MR companies are scarcely in the picture.

Meanwhile, what is happening within market research companies? Well this last week I’ve been in conversation with two analysts looking for better more challenging roles than they have been given within their market research firms. One of them, an employee with one of New Zealand’s leading MR firms, (and one I largely admire – they are by every measure a world-class operator,) had for several years been asked to produce crosstabs and other descriptive outputs, and had on no occasion, ever, had his mental capabilities even remotely stretched. I’m talking about a graduate in statistics who has effectively been cudgeled to death by the rote boredom of low calorie market research thinking. I asked what he was equipped with, software-wise and he told me: “Microsoft Excel.”

This is simply not good enough both professionally or stragically. While globally the volume of marketing data analytics is growing by something like 45% per annum, the market research industry is relatively flat-lining or showing single digit growth at best. In other words most of the growth is happening over at the client’s place. And they aren’t even specialists in data.

If the market research industry wishes to gain relevancy, then it has got to offer something that clients can’t provide for themselves. It used to be the market researchers provided unrivalled capabilities in the design and execution and analysis of market research projects. The key word here is “unrivalled” but I’m afraid the leading research firms are being simply outstripped by their own clients.

The mystery to me is why the larger firms appear blind to this phenomenon. Perhaps in building their systems around undernourished, under-equipped DP departments, they have a wonderfully profitable business model. Pay monkey wages, and equip them with Excel. And for that matter, keep them at arms length from the client so they never get to see how their work even made a difference. The old production line model. Tighten the screws and send it along the line.

Or perhaps the big firms are simply comfortable in doing things the way that I’ve always done, or perhaps senior managers, having grown up in Kitty Hawk thinking lack the imagination or the will to fly into the stratosphere.

Either way, if I was a young analyst and looking at my career prospects my attention would be over on the client side, or on dedicated data analytics operators such as Infotools. That’s actually a painful thing for me to say, speaking as a life member of my market research professional body. But if the status quo prevails, then we are going to see not just the relative decline, but the absolute decline of our industry.

What can market research firms do to rectify this problem?  Here are 4 suggestions:

  1. Invest in decent analytical software. Just do it. A few thousand dollars – for a much better return than sending your exec to that overseas conference.
  2. Reignite the spirit of innovation becomes from your groovy team of analysts. Rather than focus merely on descriptive data, let them loose on the meta data – the stuff that explains the architecture of the public mood.
  3. Develop a value add proposition to take to clients. Look, all of us can describe the results of a survey, but we market researchers know how to make that data sing and drive decision-making.
  4. Employ specialists in big data, so that as market research companies we can integrate the thinking that comes from market surveys, and qualitative work, with the massive data sitting largely untouched in the world of the client.

In my view market research industry has been going off-course for the last 20 years. We are stuck at Kitty Hawk. We stopped shooting for the moon.

DSC00104

 

 

Data from the call center…how to improve it

Recently I’ve been working with rich verbatim data from a customer call center – and boy, this is where you hear it direct: complaints, hassles, confusions and also stories of unreasonable clients who feel they ought to be exempt from all fees and charges. Some people seem to think that institutions ought to run for free.

Image
String data – may be hard to work from if the questions are wrong to begin with.

Having worked with the client to develop a pretty accurate code-frame, we still found that our analysis wasn’t getting deep enough. For example take all those customers who felt some mistake had been made.  We had a well-constructed codeframe that picked these people up into one sizeable bucket – but somehow, through some combination of our coding architecture and the sheer diversity of the English language, we hadn’t adequately picked up the nature of those mistakes and errors.

So back we went and hand-coded our shortlist of Mistakes & Errors, and we found four basic causes for these. We also asked ourselves if we might somehow have changed our coding architecture to get a sharper result next time. The answer is: no – the real problem was the way the verbatims had been recorded. A lot of the story was between the lines and inferred. There was no way that automated text analysis could pick a lot of this up.

The bigger answer is this. There are good ways to ask open-enders as well as less-good ways and one of our suggestions was to tweak the way the call center asks customers to tell their stories by asking clearly what the cause of the error had been (in their view.)  We also suggested a check-box to precode some of the answers where possible. For example: Was a mistake or error made? Check. Then what general type of error was this? Type 1, 2, 3 or 4. 

Text analysis is seldom really simple, but often analysts wrestle with the raw data – we struggle to make it sing. Sometimes we need to go back to the collection point. We don’t have to be victims in the process.

 

Open Enders – one way to get more value from these

Image 

With the broad arrival of text analysis as an increasingly mainstream analytical activity, a lot of heat is coming back on the question of how to ask more useful open-ended questions. Is there a “best way?”

Yes, there is. If traditional coding was a process of blunt word counts, then newer forms of text analysis are more about the relationships between ideas. Is this mentioned frequently in terms of that? 

For that reason you get much better mileage by asking for two or three thigs instead of just one. Instead of asking; “What is the main reason for choosing brand x?” you are better to ask: “What are your reasons for buying brand x? Please give us two or three reasons.”

This approach isn’t just about driving longer answers (it may help perhaps) but is more about being able to see how ideas associate: you can start to understand the structure or architecture of the public’s mindset.  A single answer “Name the main reason…” gives you lego blocks of data. A “give us two or three” type question enables you to assemble those blocks.

Why Wordle drives me nuts

In the past three years use of Wordle has spread like wildfire, profession by profession. Right now I have evidence that it is taking the HR profession by storm and late last year was at a conference of school guidance counselors where Wordle charts on PPT create a ripple of oohs and aahs from across the audience. How did the presenter create those?

In some respects I love Wordle, and I love the fact that IBM gave its developer free license to just go ahead and develop the thing, and the way they really put it in the public domain.

But in market research, a word cloud is a poor substitute for real analysis. At best it does a word count so you can see what words got used most. I guess that’s fine if the question is something like: “What is your favourite brand of coffee?” From the massive choice available, the open-ender might produce a cloud where some brands emerge as more dominant than others. But why not just do a bar chart based on mentions?

No, what drives me nuts about Wordle is the way too many researchers somehow feel that by generating a word cloud – which is akin to throwing a pack of cards on the floor – they have somehow “analysed” the content.

Analysis?  To illustrate how good this is as an analytic tool, I’ve created a Wordle to help us analyse some numbers. Hey, it works for words, so this should be brilliant.

WORDLE NUMBERS 2
Hey. This is a good analysis!

What does this tell us?  What numbers aren’t mentioned?  Are there themes at work here? Who can tell? All the process has done is throw them on the floor in a game my sister used to call “52-Pick-up.”  Well, this is all it does with words.

Analysis ought to involve more thought than this, and in a discussion I started in the LinkedIn group Innovation Insight, one member, Shannon Gray from Nashville, discussed how she develops word clouds to go just a few steps further by first of all using Excel to help look for common links between words “unfair – price” for example – and then running word clouds. She recommends Tagxedo (which is very similar to Wordle) because it offers more control.

I do think text analysis goes way, way beyond Wordle’s scope, and the pity is that too many people have been caught up in the novelty of word clouds (broadcasters were big on it last year, but it has fallen from favour with their graphics teams) without thinking: wait – is what we’re doing really adding value to the data?  If this is all we’re doing to verbatims, then it might be better to simply hand over the sheaf of responses to the client and say, “Hey, here’s what your customers are saying.”

If you wish to check out Tagxedo – CLICK HERE

Text Analytics – a well regarded software.

Clarabridge –  is a well-regarded text analysis software that is cloud based. The company offers the product both in Professional as well as Corporate packages – and for the former charge per line of data. The system is geared around several inputs including social media and for that reason might be a good alternative to SPSS Text Analysis that I currently champion – one benefit for users is that the pay per usage business model may have less immediate impact on the bottom line.