Key performance indicators (KPIs) are indeed “key”, but they’re also keyholes, offering only a limited view into your business data. To get the most insight out of your data, you need to open the door and get a better look. KPIs are summaries of aggregated data, with sexy visualization. By design, they distill the complex stories being told by your data into a smaller, blander form in order to accommodate the inherently low-bandwidth process of continuous manual monitoring.
Distill, distort, disaster
Here’s the problem, though: distillation can obscure or distort important details, like the pixelated artifacts around high-contrast areas in a low-quality JPEG image. Small details often have a big impact, as Domino’s Pizza discovered when a new online pizza delivery system in France failed to handle apostrophes correctly, erroneously telling users that their addresses didn’t exist, stopping those orders before they could be completed. This bug contributed to Domino’s losing $635 million in market value. That’s way more pain than a glass of fine Bordeaux can ease.
High rates of cart abandonment at the address entry step should have alerted them that something was amiss, if they were even collecting data that specific. Much more likely, that signal was hidden in some high-level KPI. Since the new pizza delivery system faced other problems (like too few choices for single toppings) at the same time, this particular contributing factor may have been hard to tease out if Domino’s wasn’t also monitoring cart abandonment at the single topping selection step.
Any enterprise monitoring data that specific would end up keeping tabs on an astronomical number of metrics. Granularity brings scale which in turn brings…something other than KPIs. The view through the keyhole is just too restrictive to see a whole landscape of data through it.
AI analytics leads to better monitoring of metrics
Real-time anomaly detection vendor Anodot brings up a few interesting points about metrics monitoring: due to the fact that the number of monitored metrics skyrockets as metrics get more specific, KPIs (and the dashboard tools which display them) can’t handle the level of detail you actually need to fix problems as they happen.
KPIs can’t handle the combinatorial explosion, but analytics built on AI can.
When AI is unleashed on your business metrics, the result is much more intelligence than “business intelligence” tools provide. There are industries that desperately need intelligence. One of them is programmatic advertising, a key component of adtech. The ability of AI analytics to extract insights from massive amounts of granular data could help companies like Microsoft avoid inadvertently funding Islamic extremists by buying ads on Jihadi propaganda websites.
This is a real problem in adtech, most notably because programmatic advertising was supposed to usher in a new era of online advertising in which ad buyers would pay for impressions displayed to specific demographics, allowing all of us to benefit from algorithm-driven targeted advertising which yields greater results with less wasted money. So, either Microsoft has decided that the type of person who would willingly visit an extremist website to fill up on violent anti-Western ideology is their new growth market for Azure cloud services or something is seriously broken here.
When a supposedly data-driven process delivers the exact opposite of the intended result, it’s an indication that wrong data is doing the driving. In all likelihood, an algorithm spotted a tempting value for a KPI like CPM and decided to buy. KPIs can provided general indication, not useful correlation.
If you’re only monitoring KPIs it’s time to pull over
A KPI will tell you that the engine is overheating, it won’t tell you if it’s because all your radiator fluid leaked out, nor will it tell you where the hole is, nor how to plug it. And besides, the truth is that your engine overheated because your fully loaded car was climbing a steep hill in 110°F desert while your AC was on full blast. Of course, the dashboard won’t tell you that, because it can’t.
Those other pieces of information (the outside temp, load on the engine, etc.) add significant context to the engine overtemp light on the dashboard, but only after they are all intelligently considered together. An AI-powered approach to business metrics monitoring can find those important correlations at the scale of millions of metrics and do so in real-time, which is always the right time for adapting to rapidly developing business incidents.