Data & Analytics have become main-stream. Executives and their boards are increasingly starting to question whether their organizations are truly realizing the full value of the insights. A study suggests that 58% of organizations have difficulties evaluating the quality of the data and its reliability, raising a big question to the stakeholders as to “can you trust your data?” On one hand these is this set of people who are worried about the authenticity of their organizational data, or the data they intend to use.
On the other hand you may encounter a set of people coming up with lame excuses, and claiming that they are happy with their data-sets and find their data to be trustworthy. They are not in need of any kind of data cleansing or data processing or assistance of data management experts. They are not wrong completely at what they feel and so what they say. The recent reports by Gizmodo, The Independent, New York Post and various others, about “Balls have zero to me to me” where Facebook’s AI chatbots Bob & Alice created their own language. Such incidents are enough to send chills down your spines.
Investigations to aforementioned incidents are on, and most likely it would be the bad data or absence of data cleansing process; the root cause. Don’t get us wrong. We advocate data driven decisions. However, on a thoughtful note, all this and much more can happen only if your data is in place. When we are talking about the trustworthiness of your data, it’s the appropriateness and accuracy that we are referring to.
Evolvement from common sense to data sense
We as a society have moved away from decisions made based on limited information or gut feel, to a data and information driven society; where applicability of common sense is minimal or nil. However; the challenge is that though the society has evolved, people have not. Business and enterprises are still being led by baby boomers that are better suited to hunt mammoths and not take financial decisions based on accurate data and insights derived from them.
Now that, everyone has realized that human judgement in a business context is poor, organizations are increasingly basing decisions on data driven facts. But is their data trustworthy? Let’s see why they should not trust numbers blindly?
1. Question the data tracking set-up
Believe it or not, but a lot of things can go wrong. Even Google Analytics is prone to mistakes, which is backed up with this discussion on GA data. Anything and everything starting from data collection to data integration, data interpretation to data reporting; should be questioned rigorously. For example, events not named in an explanatory fashion, inclusion of start date and many more; can lead decision analysts to commit errors while calculating results.
2. Question the interpretation of numbers
Yes, the numbers can be misinterpreted if the context is not understood completely. Sales manages, would die thinking why the conversions rate was not going up, even after making improvements to purchase funnel. Unless it was questioned to discover that sales team has started an acquisition campaign, which did result in higher volume of visitors who were ‘less qualified' than earlier; and hence less conversions.
In an opposite situation, if the conversion rate had skyrocketed, no one would have questioned the positive numbers and the sales manager would have taken pride in the hike of the conversion rate.
3. Question the successful metric
It’s time to move away from one size fits all belief. One key success metric for all, will certainly not work. For example, in publication industry - the user behavior varies depending on the device used, and so does the metrics too. They do not have the easy task of measuring against a purchase funnel, as in the aforementioned case, and it can be challenging to find the right KPIs to be taken care of. The most important aspect here is content consumption, which tends to show low performance on mobile, wrongly portraying that there is a problem.
Also at times, the metric is either neglected or not adapted completely. A same scroll depth target of 75% for desktop and mobile users both; is as good as neglecting the metric. On the mobile all the left hand and right hand elements of the desktop page were stacked one on another under the main content. It made the user to go only 50% upon reading the full article.
4. Question the good numbers first
If you witness a huge drop of conversion from page 4 to 5 in the purchase flow, it is obvious that you will check the user experience. But what about if the conversion goes up between those two pages, due to the reason that your user missed out on a crucial piece of information and all they saw was the “next” button. Numbers have the tendency to make you feel all is well, when actually it is not. Irrespective of the kind of analytics one uses, Predictive – Descriptive or Prescriptive; it cannot replace the value of regularly watching customers use your services.
5. Question the KPIs
One of the big culprits – thankfully slowly becoming irrelevant – is page views. If set as a target, someone will surely find ways to grow this KPI without any improvement in customer behavior. And this is probably how these endless galleries of images were born, where each picture counts as a page view, or articles broken down in multiple pages, with no benefit to the users.
Don’t take numbers for granted
Check and verify numbers, good and bad both, as if you are a quality assurance manager and testing the code. The most glorified numbers are the ones which can damage your business the most. Be critical and challenge the numbers. Also ensure to adapt the metrics completely, as you enhance your own product. And lastly, do not forget to combine it with qualitative insights.
Some opinions expressed in this article may be those of a guest author and not necessarily Analytikus. Staff authors are listed in http://www.datasciencecentral.com/profiles/blogs/data-and-analytics-don-t-trust-numbers-blindly
