Recently I’ve been working with rich verbatim data from a customer call center – and boy, this is where you hear it direct: complaints, hassles, confusions and also stories of unreasonable clients who feel they ought to be exempt from all fees and charges. Some people seem to think that institutions ought to run for free.
Having worked with the client to develop a pretty accurate code-frame, we still found that our analysis wasn’t getting deep enough. For example take all those customers who felt some mistake had been made. We had a well-constructed codeframe that picked these people up into one sizeable bucket – but somehow, through some combination of our coding architecture and the sheer diversity of the English language, we hadn’t adequately picked up the nature of those mistakes and errors.
So back we went and hand-coded our shortlist of Mistakes & Errors, and we found four basic causes for these. We also asked ourselves if we might somehow have changed our coding architecture to get a sharper result next time. The answer is: no – the real problem was the way the verbatims had been recorded. A lot of the story was between the lines and inferred. There was no way that automated text analysis could pick a lot of this up.
The bigger answer is this. There are good ways to ask open-enders as well as less-good ways and one of our suggestions was to tweak the way the call center asks customers to tell their stories by asking clearly what the cause of the error had been (in their view.) We also suggested a check-box to precode some of the answers where possible. For example: Was a mistake or error made? Check. Then what general type of error was this? Type 1, 2, 3 or 4.
Text analysis is seldom really simple, but often analysts wrestle with the raw data – we struggle to make it sing. Sometimes we need to go back to the collection point. We don’t have to be victims in the process.