The Future of Digital Attribution Models: Moving Beyond Last-Click

As tried and trusted digital channels become mature and struggle to deliver the yields that they used to, marketers at digital-born startups are looking at a broader range of channels to drive growth, including offline channels. Couple this with the current shift to a cookieless world and the challenge of understanding Return on Advertising Spent (ROAS) is one that can no longer be tackled with simple last-click attribution models.

So, how do you understand what channels work best for your business and where to invest your Marketing budget? Is it just a shot in the dark like the old days?

In short, the answer is 'no' but there isn't just one handy silver bullet you can rely on. Some would say there never was! Understanding return on investment and the effectiveness of your advertising spend is complex and should be looked at from multiple angles.

To get to the bottom of the thorny subject of attribution in 2024, I spoke to Franky Athill who set up Attribution Lab to solve a problem that he came across time and time again as a startup CMO himself.

Enjoy!

------

Hi Franky. Welcome to the Rocket Ship. Why don't you start by telling us about your career highlights?

I got into start-up marketing joining Patch Plants as the first marketing hire. And after four and a half years I went to Mindful Chef and Finisterre on interim contracts as Chief Growth Officer and Chief Marketing Officer. And since then, I’ve been consulting for brands Wonderbly, Monica Vinader and Trip as an attribution consultant or Fractional CMO.

And tell us more about Attribution Lab, the consultancy that you’ve founded. What inspired you to set it up and what is the problem that you're trying to solve for digital start-ups?

"If we found the right people with the right media plan, we could supercharge the growth of this exciting new brand, but if we got it wrong, we could sink our one shot at success very, very quickly. "

Yes, actually on that question, it would be good to not say ‘digital start-ups’ because really, we come in when people are going offline. That's where we're most valuable with it: when they're stopping being exclusively digital.

As the marketing person in five or six start-ups and scale-ups, I was the person with sweaty palms and nervous feet, going into meetings with the board and the CEO having to explain why we were spending a million pounds on TV and not a million pounds on Meta or a million pounds on Google, or a million pounds on Tiktok. And this was the most stressful part of my job. And so I was searching for many years for better ways to come to that decision, because it was such a make-or-break decision for the business. It was the second biggest cost centre and we knew that if we found the right people with the right media plan, we could supercharge the growth of this exciting new brand. But if we got it wrong, we could sink our one shot at success very, very quickly.

And so in that search, I built various attribution models in house with data teams, I worked with external media agencies on models, I worked with econometric specialists. And I found it very hard to find an independent expert who didn't sell media themselves and therefore could be really trusted to help with this media mix decision. And hard to find not just data people but actually marketing people who could give an actionable report that we could take straight to the CEO to make these big channel mix decisions.

I also talked to over 100 other CMOs of start-ups and scale-ups and found that everybody shared the same frustration. In a survey I did of consumer CMOs, 36 answered the survey, and not a single one gave an 8, 9 or 10 to the question: ‘How would you rate your attribution set-up out of 10?’ And if this was an NPS score, it would be minus 68. So I felt I was not alone. There was a huge amount of demand for more help from independent experts when it comes to attribution. So I decided to set myself up as one.

Fantastic. And what have you found to be the real challenges and pitfalls around attribution and how it's evolved over time?

"Click-based models are going to get less accurate over time. And so, investing in them now isn't a wise decision.

Well, most people are still using click-based models for attribution. This still dominates. In all the conversations and the survey that I did of these CMOs, 40% were still using just last-click models for all of their channel mix decisions. Even if they have online and offline channels and online and offline sales. Another 30% were using last-click in combination with some survey data. So, really, last-click is still dominating. These click based models are still dominating in our attribution decisions, although they've become a lot, lot worse in the last couple of years and they've never been useful at all for brands that have offline spend or offline sales when it comes to the CMO’s decision of channel mix. They're very useful for the channel owners’ decisions of optimising their channel over time, but not for this big decision: the primary decision we start with, which is how much to spend on each channel.

We've all seen the headlines, it's only going to get worse, our ability to track people on the internet. And therefore, our click-based models are going to get less accurate over time. And so, investing in them now isn't a wise decision. This is also illustrated by the fact that the tech tools and agencies that were focused on multi-touch click-based attribution, are now moving to modelling approaches.

The other thing about click-based models is that they will always bias towards the bottom of the funnel. They'll always ‘see’ more conversions that have happened closer to the purchase, and as a result, they will only ever see the tip of the iceberg when it comes to the full effect of your marketing spend. And what this does is pigeonhole brands who are trying to expand from being start-ups to being scale-ups. It pigeonholes their spending to just bottom of funnel channels, which the click-based models get an okay read on.

And the opportunity for those brands is to start including in their attribution stack, survey data, modelling data and experiments and they'll start to see more of the big chunk of the iceberg below the water. The full effect of their marketing. That's the opportunity.

I think the best way to look at it is in this table (below) which shows that there are four pillars of marketing measurement. Most people are pigeon-holed into click-based attribution options, and what we need to do if we want to have more confidence in our channel mix decisions, is to piece together all four columns of marketing measurement and add them into our stack as soon as we can.

What are the other approaches that people should be looking at?

"(If you have modelling) it can add much more accuracy into your overall picture, removing the need to track people across the Internet and looking at the correlation."

When brands start out, it's great to start with a click-based model, because you don't have any historical data to look at. It's free and available and you can start from day one. And it'll be okay if you're just running one or two digital channels and you're selling everything online. As you expand, your next step is to collect survey data. Running surveys post purchase to ask customers: ‘How did you hear about us? If you have a sales funnel, ask it in the sales funnel. That's going to give you your first view of all touch points, not just clickable ones and probably get you much closer to the truth than the click-based model. It’s a very big step up. It's also quick, free and easily available to any company.

Once you've got your survey set up and you start to use that, the next step is to start introducing modelling and experiments. Modelling, which can be called econometrics, or marketing mix modelling, looks at correlations between changes you've made, as well as external changes in your market, with your historic growth. And so, it needs at least two years of good data, and you probably need to be spending about £2 million a year for two years for it to be a worthwhile project. But if you have that, it can add much more accuracy into your overall picture, removing the need to track people across the Internet and looking at the correlation.

Experiments are excellent, but generally they can take a lot of time and money to run. And so most start-up and scale-up brands can only do a couple of experiments a year. But picking those experiments wisely is very important: focusing on your biggest channels, trying to get a read on them which you can then plug into your modelling.

So these are the steps that the brand should take as they mature from being start-ups to scale ups. And in quarterly reports, it should be looking at what all four of these clues suggest about the ideal channel mix and make a judgement call. For each channel, you may find that you trust one of these four ‘witnesses’ more than the other. For SEO, for example, you might only trust the click-based model because you haven't run the experiments, your marketing mix model doesn't get a good read because there's not much variation. And nobody in the survey knows the difference between an organic click and paid click. But for other channels, let's say Meta or Google where you've got a lot of data; and you've got a read from your marketing mix model; you've got a read from the survey and click data, you might trust your marketing mix model the most out of those ‘witnesses’. So, you've got to play judge and jury. You've got to try and collect as many as possible ‘witness statements’ and then make a judgement call for what you think the channel £CAC has been in the last couple of months.

Makes total sense. And how have scale-ups found moving across that journey to a more complex, blended form of measurement?

"I think that the headlines over the last year have helped (...) people realise the dangerous trap that they would fall into if they follow a click-based model with their spend decisions."

I think that most start-ups are now using survey data because they see that their click-based models make no sense at all. I think that the headlines over the last year have helped, and will continue to help people realise the dangerous trap that they would fall into if they follow a click-based model with their spend decisions. Marketing mix modelling is really taking off. Facebook and Google themselves recommend marketing mix modelling and they've both released open-source code packages to help brands start doing exactly that. But they're still complex. Building a marketing mix model still needs data scientists, still need experience and you still need to make judgement calls, sensible judgement calls, when you build your model. It's still an expert practice.

But there is more online training, open-source packages and small agencies like us who are willing to help relatively inexpensively, if you want to get into it for the first time. Data Science as a practice has come a long way in five years. And it's meant that we can bring down the cost of building a useful marketing mix model from £100,000 to now £20,000. And that's really opened it up to scale-up brands spending £2 million+ a year, to do quarterly, and massively improve their confidence in their channel mix.

And the business case for it is becoming clearer to people. They're realising that how they make their channel mix decision is largely guesswork at the moment. And therefore, it's easy to believe that improving their overall ROAS by 5%,10%, 20% is very believable by investing in their attribution stack with a marketing mix model. And as a result, they only have to believe - if it costs £20,000 – that it only has to improve their efficiency by 1% for it to become a very obvious business case. It's not going to be perfect, but it's going to be a lot better than where they are at now.

Great. Thanks, Franky. What I'd love to hear is your top three tips for any start-ups and scale-ups when it comes to attribution and measurement.

 "If you're a marketing person, you're in the game of placing bets. (...) and in the way you're currently placing those bets, the first decision you make is how much should we spend in each channel."

My number one tip is to acknowledge your starting point honestly and have frank, honest conversations with your CEO and board if they don't understand how this game works. Acknowledging that, if you're a marketing person, you're in the game of placing bets. You are being asked by the business to grow fast and so you've decided as a business to invest a lot in paid advertising. In investing a lot in paid advertising, you have to place bets. And in the way you're currently placing those bets, the first decision you make is how much should we spend in each channel.

That first decision, if you're using a click-based model, is close to guesswork. It could even be worse than guesswork because the click-based model by design is going to be biassed towards the bottom of the funnel and the channels that skew that way. And therefore, acknowledging this starting point, acknowledging that you have to place bets, is the first piece of advice. Rather than the trap that you often fall into which is to have a comfort blanket of the perceived accuracy of a click-based model and presenting it as fact. Because then they get trapped against the ropes in future conversations having presented wide estimates based on click-based models as fact.

So acknowledge you're placing bets. Acknowledge it's all estimates and your job is to find a better estimate than the one you currently have. And therefore, the more ‘witnesses’ you can get in the room, the more likely your estimate will be closer to the truth of your channel ROAS. That'd be the first thing and that's really a conversation thing. It’s an education piece internally. And it's making sure you use the right language around talking about marketing measurement, and you put your hands up and have the confidence to say you don't know the answer. You're just trying to hone in on it.

Tip two would be: if you're not a pure digital business with a very short purchase journey, not spending much and only on one or two channels, then a click-based model isn't good enough on its own and you must start using other views. So, get your survey data live and make sure your survey is set up well. With randomization and a ‘booby trap’ answer that you don't really run so you can tell how many people are guessing. And get enough data through it monthly to do a monthly report.

And then tip number three: if you're spending over £2 million a year, it's worth investing in a marketing mix model. I would buy rather than build. If you build, it'll take many months for probably your only data person in your scale-up and it will end up costing more and take a lot longer than if you find an agency to build it for you. But find one that doesn't sell media themselves and therefore has no horse in the race. And finally, run good experiments in your biggest channels, use platform tools that can help run good geo tests relatively easily. And invest a good chunk of your budget in allowing those experiments to happen.

Very good tips. And what would be one mistake that people should avoid?

Well, in researching this space, I hoped that there would be a great tech tool that solved this for everyone, like we all do. But in all the research I did, I found that those that had used a tech tool for attribution had been less satisfied with their attribution than those that hadn't. And so, so far it seems that no one has made a good tech solution. It's too much of a complicated problem. And so for now, I would avoid tech solutions to marketing mix modelling as I haven't seen any good reviews.

Fantastic. Thanks, Franky. Some amazing tips there. And plenty for people to think about.