Skip to main content

History Channel

Analytics, we are repeatedly told, will enable us to have these wonderful insights that will change our business and transform society, probably abolish world hunger at the same time. And the only reason they're not doing so is because we have not been able to implement it well.

All true, but to my mind a bit of bollocks. Its a bit like Nike advertising the same shoe that Michael Jordan has, and the only reason we're not in the NBA Basketball Hall of Fame yet is that we just haven't put in the hours of practice required.

Here's the problem with insight in analytics. One, insightful analytics is very very hard. It requires a lot of investment; to go back to the Jordan analogy it takes thousands of hours of practice to get anywhere near the NBA. One basically sacrifices everything else in life - something the average person is just not going to be able to do. A company whose job is analytics will put in the investment (an investment research firm, for instance) but a business that is busy doing some other core activity (as most companies are) is unlikely to.

Second, it takes more than investment and effort. 10,000 hours of effort will produce a very good basketballer, but it cannot guarantee a Jordan. Insights are rare, and in addition to investment and effort takes talented analysts and luck. Yes, luck - something that product companies don't like telling you because it isn't on the pricelist. Newton's brain wasn't enough - he also had to be under the apple tree at the right time. This puts a bit of a dent in that ROI thing, and probably explains why the extensive investments in analytics dont really lead to daily game-changing insights.

So am I advocating tossing out the analytics baby and getting on with other things? Not at all. I'm just advocating getting out of the insight steamboat, where the promise is of untold riches if only one buys a ticket to the magic kingdom. Forget using analytics for insight, but don't by any stretch forget analytics, because...

Analytics are about hindsight.

The real value of analytics to a company is the ability to see what has gone before, and learn from that; not to change the game but play the same game better. Hindsight is not 20-20 unless you study it (which is why history is so often repeated). If one has to learn the lessons of history, one needs data and analysis that describes well what exactly was happening - and what is likely to indicate its recurrence. Why my sales fell last quarter is a question of insight - but where they fell, by how much they fell, what else was happening the day they fell - these are all questions of hindsight. And these questions are much easier to answer predictably and reliably than the why.

Once you look at analytics through the hindsight goggles a lot more starts to make sense. Gathering more data points gives you better detail on what happened. Gathering more sources gives you breath of visibility on all the different things that may have happened. Correlation helps you identify what happened before, after or along with what other things. And every time you have a theory on something, analytics tells you (often quite accurately) if the data so far supports your theory or not.

This is quite analogous to the scientific process. One does not carry out experiments hoping to stumble on an insight; instead one tries to analyse experimental data to see if a potential insight is indeed backed by fact or merely a misguided fancy. Computers are not well suited to mull out insights from half-formed ideas - that is the domain of human brains. Computer software, on the other hand, is supremely good at crunching data to validate or junk an intuition.

Humans have since ancient times hoped for a way to reveal what is going to happen, and have repeatedly come away disappointed. Figuring out what has happened, and repeating the successes while side-stepping the failures may be more pedestrian but it is ultimately more rewarding. And that's not an insight; after years of looking for patters in the sand that's hindsight.

Comments

Popular posts from this blog

Rethinking Disaster Recovery

Disaster Recovery has been on the minds of companies ever since the early days of commercially available computing. Today's world of DR revolves around four acronyms - BIA (business impact analysis), RPO (recovery point objective), RTO (recovery time objective) and BCP (business continuity plan). The acronyms appear in a disaster recovery plan in roughly in that order, the thinking being that you first analyse the impact to business of systems being down, then figure out how far back in the past are you willing to turn the dial back to recover from (last day, last hour, last millisecond). Next focus on how long you can afford to be down. Finally - buy a boatload of hardware, software and services to convert all this into action. Setting up a DR is a hugely expensive affair that takes a significant amount planning and effort, not to mention all those drills and tests every now and then. CTOs have followed this prescription since the late seventies (apparently the first hot site wa

Outsourcing III–The "Who" Question

A little while ago, I was asked to give a presentation to CEOs on outsourcing. The audience wanted to know about adopting outsourcing for their companies; making use of its promise while avoiding its pitfalls. It seemed to me (unimaginatively, I must admit) that the whole thing boiled down to four fundamental questions - the why , the what , the who and the how . I decided to expand the presentation into a series of blog posts, one per question. The Who Question Once you've clarified why you're looking for an outsource partner and also which pieces to outsource, you're faced with the next big question – who? What should you look for in your potential outsourcing partner? The choice, I put to you, comes down to four linked characteristics. Ability The first characteristic, of course, is ability. A vendor cannot be under consideration at all if the basic ability to handle whatever you plan to outsource is not present. This is not always an easy thing to judge, especi

Looking Ahead

I just sat through a presentation about the four big trends in IT that even the polite struggle to describe as dull, but it did get me thinking. Incumbent technology vendors (pretty much like incumbents everywhere) have too much invested in the present and will hence sell incremental as disruptive. IBM once believed sincerely in the future of mainframes, big studios still insist on the glorious road ahead for DVDs now that the laser is blue, predicting the future is always fraught with problems. Armchair sniping is all well and good, but shouldn't I be in a position to predict better? Time to get my somewhat ample behind off the couch then, and stick my neck into the future. Here, I go - my four predictions for IT trends, at least as far as careful, conservative corporates are concerned. Cloud will vaporise IT Today's businesses are all concerned about the technology underlying the cloud, and how to adopt it. However, cloud isn't really a technology - its a method of deli