Monthly Archives: July 2013

How to find the needle in a haystack of 30 billion lines of straw.

When I was in NYC two years ago the Occupy Wall St movement was in full swing.  I wandered to nearby Wall St (the protest was in a park several hundred metres away) and the famous street was being protected by dozens of police mounted on horseback. There were cordons everywhere and brokers rushed from building to building during their lunch hour in what felt palpably like a state of siege. A pretzel cart operated in the heart of the empty street – selling drinks and snacks – and I asked the vendor, an Indian man, how business was going for him. Trading on Wall St, he assured me, was slow.

Still if trading has come down from the record highs set in late 2008, the NYSE still turns over something like 1.5 billion transactions per day, or around 30 billion every month. And somewhere, lurking in those 30 billion lines of data is evidence of insider trading. The question is, how do you find it?  The needle is small – the haystack unimaginably big.

The answer is that the SEC is often informed by the investigative team of FiNRA – the Financial Industry Regulatory Authority spearheaded by a man who used to be in the DA’s office: a prosecutor you wouldn’t want on your tail. His name is Cameron Funkhouser, and he’s a big thickset man described by one colleague as an “I can smell a rat, kinda guy.”

But get this – the investigative team he’s put together to delineate all those billions of lines of data, looking for patterns of fraud and insider trading (do two apparent strangers make the same trades on the same days, repeatedly – could they be connected?) this team he’s put together is not chiefly characterised by data geeks and statistical nerds. Heading the unit, underneath Funkhouser’s command is a team spearheaded by:

  • Whistle-blower unit – Joe Ozag. (Ex terrorism detective.)
  • Anthony Callo – once prosecuted homicides at the DA’s office.
  • Laura Gansler – screenwriter and the “brains” of the group.

This team is characterised by diversity, out-of-the-box thinking, and an understanding not so much about statistics as about people: our drives, our motivations and our craftiness: in short; our narratives. Laura Gansler might be the least statistical member of the group – she is an accredited screenwriter and author – but she’s well versed in developing credible storylines and getting inside the head of crooked characters.

And this is the point about dealing with big data. For sure, your programmers and number grunts can dig around and reveal black and white statistical evidence of those occasional insider trading schemes – but the real value in this analysis comes from those who can furnish a credible story and, using logic, suggest where best to look for the telltale finger prints, bloodstains and data trails that mark the crime.

FINRA has proved remarkably effective at helping nail the bad guys, and Funkhouser speaks highly of his team.

The lesson for researchers and business people is that the best way to deal with massive data – 30 billion lines of the stuff every month – is to remember two things.

1)  Every bit of data reflects human activity. It isn’t about numbers – it is about people.

2) Get inside the mind of the bad guys – and you know where to look.

Incidentally the FINRA unit reflects, surprisingly closely, the fictitious Department S – the 1970s show that gave us Stewart Sullivan, conventional crime-fighting agent, Annabel Hurst, computer expert, and the redoubtable Jason King, cravat wearing international novelist of mystery.  Department S was born of a time when traditional crime (bank jobs and murders) were giving way to global operations. (The French Connection was another movie that wrestled with the same escalation of crime.) So it is interesting that the escalation of data into Big Data should meet with a similar response: put talent and creativity onto the case otherwise there will never be enough resource.