How To Measure the ROI of Data Analytics
Header image by @neonbrand via Unsplash
As digital analysts (or people who manage digital analysts) we have a pretty cushy job. We get to play with data all day, make pretty charts and visualizations and tell people how to fix stuff. Ok, so that's not entirely true. But the truth is that many analytics teams spend a good amount of time shining a light on the performance of other teams’ work, such as marketing or product. But less often is that same light shone on the analytics function itself.
So what is the ROI of data analytics? Avinash Kaushik wrote about this a while back when he proposed an equation for measuring the return on analytics, what he called (ROA). The equation broadly involves attributing incremental sales to analytics time and resources and then subtracting your analytics-related costs. Here’s his equation.
Source: Avinash Kaushik - Return on Analytics
If you’re familiar with calculating marketing ROI using equations like ROMI or ROIMI, then Avinash’s equation will be familiar.
But as straightforward as the equation is, getting the numbers needed to calculate it can be tricky. For example, segmenting sales attributable to analytics activities can be really tough, especially if you’re a medium to large firm with other marketing activities running.
I think the idea of ROA is great, and I believe that attributing incremental sales to an analytics practice can be useful. That being said, I also think there are other ways to think about the return on data analytics.
When it comes to companies that properly invest in data programs, analytics teams don't always feel pressure to justify their existence. But I sometimes wonder if we should question investments in data more often. Any business function, be it sales, marketing or product development, would typically have a set of core KPIs they need to achieve. And the truth is that the outcomes of any department, including research, data analytics and data science, are only as good as the people and strategy driving them. As such, it can be a useful practice to treat data teams the same as any other, and to set clear KPIs for growth and to scrutinize outcomes.
Kaushik's article on ROA is a great starting point to think about business value, but allow me to propose another way to measure the value of data analytics.
A while back, I came across a fascinating survey of data scientists run by Rexer Analytics. The image below is a slide from one of their annual Data Miner Survey report, and this got me thinking.
Source: Rexer Analytics 2013 Data Miner Survey
The idea of measuring the time it takes to analyze data from the point at which it’s collected and, subsequently, the time needed to take action changes is fascinating to me. And I think this is a great way to think about the value of investing in data analytics.
The chart below, taken from the Data Miner Survey, shows that a majority of companies (33%) reported being able to analyze their data within days of it being generated.
Average time to data analysis
Consider for a moment where you think you might fall on the chart above. If you find yourself located more to the right side, where it takes you (or your agency) weeks or more to analyze data and generate insight, you may have some work to do. Ask yourself, what do I need to do to reduce the average time to generate actionable insights, faster.
But I’d like to build on the Rexer Analytics survey, and propose a slightly different metric. Rather than time to analysis, I think we should focus on Time to Insight. I define insight as an interpretation of data that creates value and enables a business decision (i.e. action).
And the difference between analysis and insight is an important distinction, because I don't think analysis always delivers insight.
So, after collecting, processing and analyzing your data, you should consider measuring the time it takes you to arrive at an interpretation of your data. This could be done in many ways, from using a time-tracking tool, such as Clockify, to track time spend on tasks related to data analysis. Or you could simply track this in an Excel document.
But in addition to tracking time-to-insight, I think it’s also worthwhile to track the time it takes you to action the insight (i.e. time to action). This is a tricky one, as the type of action you want to take can have a considerable impact on the time required to deploy changes.
For example, changing the colour of a button on your website or updating copy is relatively easy and fast. But if your insight led you to change bigger things, like the entire layout of your website or app, that’s going to take much longer.
But you can start by first identifying the types of change you would commonly take as a result of analyzing your data, such as optimizing copy, creative assets, promotions or price points, etc, then categorize the time-tracking data you have by easy, medium or hard changes / optimizations. That way, your average time-to-action metric will be normalized based on the difficulty of deploying certain changes.
The next chart below, also from the Data Miner Survey, shows the average time to deploy changes based on data analysis. What was notable for me in this chart is that many more companies are further to the right. So it takes much longer to deploy change than it does to arrive at insight, which is to be expected. But the fact that 57% of the respondents said their company, on average, takes weeks or more to deploy change and 6% stated change doesn’t get deployed at all was surprising to me.
Average time to deploy changes
But I also believe there’s a difference between action and deployment. You can take action from your data but not always deploy the change.
For example, imagine you found some killer insight from your data analysis that you believe could improve conversion rates on your website. But deploying this change will require an investment from your company. So you start by building a business case, and you take it to your c-suite. They buy into the idea, and you begin the initial work needed to build out the changes on your website. But just as you’re about to get started, your boss informs you that their budget has been cut and you need to scrap the project (I’ve witnessed this exact thing happen a few times throughout my career). In this example, action was taken, first when you built the business case to take to your stakeholders and then later when the initial planning work was being done. But the project was scrapped, and the change was never actually deployed.
So it can be helpful to make a distinction between action and deployment - where action broadly refers to any steps you take toward deployment after gaining the insight, while deployment refers specifically to the rollout of your changes / optimizations.
Again, ask yourself where, on this chart, you or your department or company might be. Then consider what it would take to move to the left of the chart and improve your time to action.
So overall, we have three great potential KPIs to help you measure the impact your analytics practice is generating for the company:
Time-to-Insight (T2I) - The average time it takes you to produce actionable insight from the point at which your data is generated.
Time-to-Action (T2A) - The average time it takes you to take action from the point at which your data is generated (or you could base this on the time the insight was generated). And remember, action doesn’t also mean change is deployed, as it can include steps you take toward deployment.
Time -to-Deployment (T2D) - The average time it takes you to successfully deploy a change from the point at which your data is generated (or you could base it on the time at which the insight was generated or when the action was initiated).
Your end goal with data analytics should always be grounded on trying to accomplish more with less. Simply put, you want to move your business toward the left of both charts above. The faster you can collect, analyze and generate insight, the faster you can start taking action.
Time-to-Insight, Action and Deployment, as well as Avinash’s ROA are just a few ways to quantify the impact of analytics. But the best metrics and overall approach for your company will depend on you, your stakeholder's needs and your organizational culture.
Investing in internal analytics impact measures might be a tough pill to swallow for some people. It will almost certainly put you and your team under a new spotlight, and it could create added pressure. So why do it? Simple, investing in this kind of measurement will foster a strong data culture as well as future-proof your organization’s investment in analytics by ensuring the right KPIs are in place for tracking performance.
And working toward improving metrics like time-to-insight, action, and deployment can help make your business work smarter, not harder.
