Analytics Strategy 101: How to Benchmark Success

Reading Time: 9 Minutes


Introduction

Having benchmarks in place for all your core KPIs is key to understanding what success means for your marketing and communications activities. Without benchmarks your data is meaningless. 

Just imagine you logged into your web analytics reporting suite and found that your website received 5,000 visits last month. Your first question should be 'so what'? Was this above or below a historical benchmark for your site? Or let's say you just wrapped up an email marketing campaign and found that you're average open rate was 18%. Again, so what? Does this 18% represent a success or failure in terms of past performance?

Benchmarks give your data both context and meaning and help you see the bigger picture when it comes to defining what constitutes success. They also allow you to spot quick wins or losses in your standard reporting without requiring deep dive analysis. As a general rule, you should always have a relevant benchmark handy for every core KPI you track and report on. 

When putting benchmarks in place you will need to consider both the type of benchmark you need as well as the time intervals that govern how you view and compare performance.

The 3 Types of Benchmarks

1 - Historical Benchmarks

These are benchmarks based on your own historical performance for a given channel or data set. For example, if you’re looking at the performance of your email marketing activity and found that your latest campaign open rate was 18%, a historical benchmark could be the average open rate for all previous campaigns you’ve run which would tell you whether your current open rate is above, below or on par with past performance.

Historical benchmarks are usually the most accessible and the most meaningful benchmarks you can use, and I generally recommend that you start with these before looking at the other types of benchmarks.

2 - Competitor Benchmarks

These are benchmarks based on competitor performance. Depending on the channel competitor benchmarks generally require you to focus on metrics that are publicly available or that you can purchase (e.g. 2nd or 3rd party data) as you usually won’t have access to owned competitor data sets. For example, if you want to benchmark your website performance against a competitor, you could use metrics like visits or bounce rate as these can be pulled from competitor sites using sources like Alexa or SimilarWeb. However, it’s worth noting that these sources provide inferred values for these metrics, therefore you should exercise some level of caution when using benchmarks from such sources as accuracy can be a trade-off.

On the other hand, you generally won't be able to benchmark against metrics based on 1st party data (e.g. site conversions or conversion rate) as you wouldn't have access to such data for your competitors. 

Generally speaking, if you are benchmarking against competitors you should consider the following questions:

  • Which metrics are you comparing and how accessible are they?
  • Which data sources/vendors will you use to retrieve the competitor data and how frequently?
  • How accurate is the data (e.g. is it a concrete value or is it inferred)

3 - Industry Benchmarks

These are benchmarks based on an industry or category standard of performance. Of all 3 types of benchmarks industry benchmarks are the most difficult to obtain and there is no single approach or methodology for defining or retrieving them. Generally speaking, there are 2 approaches to developing an industry benchmark:

Using a 3rd Party Source

An industry benchmark can simply be a value you retrieve from an authoritative 3rd party source. For example, MailChimp (an email campaign and marketing automation provider) publishes email marketing benchmarks on their blog which they update every year. This includes email open, click and bounce rates broken down by industry and it's a great resource if you conduct email marketing. The data is based on MailChimp’s own customer campaign activity, and they’re not the only source where you can obtain these types of industry benchmarks for email marketing. In fact, many of the leading technology service providers in this domain publish benchmarks you can use.

There are, however, a few things you should be wary of when using benchmarks from 3rd parties. First, always consider the credibility of the source. MailChimp is an authority in this space because they are respected email marketing software provider who sit on a large volume of customer data which they can leverage to generate the benchmarks. However, if the benchmark isn’t coming from a source that owns or has direct access to a rich dataset (e.g. a marketing blog) you should be cautious about using their data. Another thing to consider is how fresh the benchmark is. If the benchmark was generated more than a year ago then the data may no longer be relevant.

Creating your own Industry Benchmark

An alternative to using 3rd party sources for industry benchmarks is to create your own. One way to do this is to combine both your historical and competitor data to create a highly relevant and customized view of industry performance. For example, let’s say you want an industry benchmark for bounce rate and you have the following data:

Industry Bounce Rate - Bounce Rate

Industry Bounce Rate - Bounce Rate

Given the information above, you could create an industry benchmark by taking the average of your historical performance plus your competitors. So based on this data our industry benchmark would be 47% ([45+48+39+57] / 4).

There are several benefits to using this approach to industry benchmarking. First, the benchmark is highly localized and more relevant to your business setting as it's based on both your data as well as your actual competitors. Second, you don’t have to wait for a 3rd party to update the figures, as the benchmark will be based on live data that is updated by you.

Benchmark Time Intervals

A time interval is simply a measure of time that governs how you view your data. For example, if you generate a monthly website report then it would make sense to include a benchmark based on a monthly time interval (e.g. month over month change).

For inspiration on how to think about benchmarking and time intervals I usually turn to the stock market. Below is a screen grab of the stock tickers featured on Bloomberg. For each stock listed you can see a number of different things, including the stock (or index) name, the current value, a sparkline, and finally, both the absolute and percentage change in the stock value. It's important to note that the change in stock value shown in the graphic below is based on a day over day change, or a daily interval.  

Bloomberg Stock Chart

As you can see above, stock market trading and analysis involves a fair amount of short term benchmarking (e.g. daily intervals), but what if you wanted to see how a company has performed over a longer period of time. This is where things get really interesting. Below is an stock chart from Google which shows us the data for Apple's stock value.

Apple Stock Chart

Take note of the time intervals available above the chart line (i.e. 1 day, 5 day, 1 month, etc). This is something you will see in almost every stock chart you find online, which is the ability to quickly and easily pivot between different time intervals, from the short term (e.g. daily) to the long term  (e.g. 5 years).

You should think about benchmarking your own performance data in the same way. That is, consider including a range of time intervals so you can easily compare current performance against short, medium and long term performance.

For example, consider the graphic below (another stock chart) which shows both 1 year and 5 year stock performance for Yahoo. This chart tells us that Yahoo has seen short term losses but long term net gains in its stock price. One might look at the 1 year trend and think that the company's stock price is tanking. But opening up the time interval to a longer duration tells a very different story. Sure, the 1 year trend in stock value for Yahoo isn't looking good, but based on long term performance the story around Yahoo's market capitalization is far from devastating. What's interesting here is that interpretation of these charts will vary depending on the person and their needs. Day traders, for example, would be more likely to see a major issue in Yahoo's short term stock value trend and might consider selling their stock, while the more disciplined traders (i.e. the Graham and Dodders) might consider holding the stock in hopes that the company will turn things around in the long term. The same principles apply to analyzing and interpreting outcomes for marketing and communications activity. You need to consider how current performance compares to both short and long term trends, and your interpretation of the data will depend on your business objectives. Like I said before, context is everything.

Yahoo Stock 1 Year Performance

Below is an example from a website performance dashboard I worked on for a client which shows how you can integrate benchmarks across multiple time intervals into a standard report. This is based on a report that goes out monthly, and it shows the current performance value for bounce rate as well as a sparkine for that metric over the last 6 months (in the green section). The benchmarks (located in the dark grey area below) include a short (month over month), medium (3 month average) and long term (6 month average) benchmark. Looking at the graphic below, you can see that there have been short term gains (i.e. improvements) in this site's bounce rate, but long term losses. Interestingly enough, if you were to limit yourself to just short term benchmarks you're going to miss the bigger picture. 

Bounce Rate Benchmarks

One final point on benchmarks and time intervals is that it's up to you to define what short vs long term intervals actually are. Depending on your cadence of reporting short term could be based on daily, weekly or monthly intervals. Whereas long term could be based on 6 month, 1 year or 5 year intervals. You need to consider what duration of time will be the meaningful and relevant so you can effectively put performance outcomes into perspective.

Key Takeaways

  1. You need benchmarks for all core KPIs, otherwise your data is meaningless.
  2. There are 3 types of benchmarks; Historical, Competitor and Industry.
  3. You should always start with developing historical benchmarks first before competitor or industry.
  4. You have to consider what time intervals you need your benchmarks in to ensure you can always put current performance into perspective.

Giving a presentation? Check out the Slideshare version of this post