Forecasting Search Outcomes

Digital Marketing - Study Notes:

What is comparative data analysis?

Comparative data analysis is the item-by-item comparison of two or more comparable sets of data. It looks at the variance between the data sets over the same or a different period, and it shows the percentage or actual change.

Why is comparative data important?

It’s an important technique for validating hypotheses or tests, and for reporting on performance. When you compare data, you can draw insights, such as how mobile traffic compares to desktop traffic over a set period. Then you can try to work out the basis for the difference in performance. It might be the customer type, the product, or another reason.

By comparing data sets, you can follow a path of enquiry to find out more about the patterns you're seeing in the data. Marketers use time comparisons quite often; for example, comparing this month to last month or this year to last year.

To compare data over time:

1. Choose your date range.
2. Choose your data sets for each date range.
3. To see the actual difference, simply subtract the older data set from the new data set.
4. To see the percentage difference, use this formula:
   • (new metric - old metric)/old metric x 100 = percentage difference

Using comparative data to forecast

Analysts can use comparative data to make simple forecasts over time. To do this, follow these steps:

1. Choose recent comparable data sets (for example, last month versus the month before last), and work out the percentage difference in the metrics between your months. You can use this difference to validate your forecast at the end.
2. To account for seasonal trends, compare last month to the same period last year (for example, April this year versus April last year). The percentage difference in the metrics is your yearly multiplier.
3. Take a full month's data from last year (for example, May of last year), and multiply it by the yearly multiplier you worked out for April in step 2.
4. This gives you a forecast of what future months look like based on yearly growth.
5. It's important to validate your multiplier and outcome against your current percentage difference, which you worked out in the first step.

Using historical web metrics to forecast

Analysts can also make forecasts using historical web metrics. For e-commerce, this involves forecasting outcomes using one or two static metrics, and one variable metric.

Decide on your business goal

When you want to forecast spend or other results for your channels or campaigns, you first need to decide on your business goal. This might be a certain number of e-commerce transactions. When you know this information, you can use distribution metrics to figure out where these transactions might come from.

Work out where your transactions come from

To do this, look at the channel data for a typical calendar month in your website analytics tool. Then work out in percentage terms where your transactions come from. For example, maybe 50% of transactions come from paid search, 30% from organic search, 15% from social media, and 5% from direct.

Distribute your goal among your channels

Next, take your business goal and distribute it among your channels. As a result, you realize that 50% of your transaction objective will come from paid search, 30% from organic, and so on.

Work out the user conversion rate

If you’re using Google Analytics or a website analytics tool, it's important to be mindful of the conversion rate shown for each channel. This could be based on sessions rather than users. ‘Users’ is a more reliable metric because it counts unique visitors. Therefore, you should work out the user conversion rate yourself rather than taking the data directly from the tool interface. To work out your user conversion rate, divide the number of conversions in each channel by the number of users for that channel.

Work out the number of users you need on each channel

Now you need to work out how many users you need to get on your site from each channel. To work this out, divide your business goal for that channel by your conversion rate for that goal.

For example, if paid search is expected to drive 10 e-commerce transactions and the conversion rate for paid search is 5%, you need to divide your goal, 10, by the conversion rate, 5%, to get the traffic required. Ten divided by 5% equals 200. So, you need 200 users at a 5% conversion rate to get 10 transactions from paid search.

Do this for all your channels, and use your goal distribution to forecast where your conversions are likely to come from. Simply divide the expected number of conversions by the user conversion rate for that channel to work out the traffic needed.

Calculate your media spend

When working out the spend required for paid channels, you should use the cost per user metric. To work this out, divide your spend for a channel by your total number of users for that channel. Now that you have a user target for the channel, you can multiply this by the typical cost per user to get a total media spend.

Be consistent with your metrics

Be consistent with the metrics you’re using. You shouldn’t chop and change between metrics, or the forecast won't be accurate. Settle on a metric as part of your single source of truth and work out the required metrics for that. In this case, it was the user conversion rate and cost per user. Keeping metrics consistent will ensure a high level of accuracy in your forecasting.

Be transparent with your assumptions

When explaining your methodology to other stakeholders, it's important to mention you’re assuming that some metrics will be static, and the forecast is based on this assumption.

For example, you might assume that the e-commerce conversion rate and the average price of an item remain the same. From there, you can forecast outcomes based on an increase or decrease in the number of users on the site.

Pro tip

Each channel, such as PPC, social, display and so on, has different conversion rates. It's best practice to forecast with individual channel conversion rates, not average conversion rates. This should prevent you from falling into the Average Trap!