Attribution Part 1: Attribution is Hard

When people talk about their specialty in marketing they will say things like “Branding”, “Social Media”, “Channel Marketing”, “Advertising”, “Digital Marketing”, maybe even “Customer Lifecycle Management”, but I have yet to hear someone say their specialty is “Attribution”. It’s too bad, because attribution is important. Very important.

Most of this book is going to be broken out into “marketing channels”. A marketing channel is effectively a source of customers. Different channels are of different importance for different businesses, but understanding the value of any given channel is vitally important (and the topic of Chapter xx). But before you can understand the value of a channel, you first need to understand the channel of origin of each of your customers. And that problem turns out to be a lot harder than it looks at first glance.

Before we dive into how you can do this the “Easy” way, let’s start with how it is actually done in most companies that attempt it at all. The most common method is called “30-day last touch”.

30-day Last Touch (30DLT) Attribution

How it works: Every time you identify a new customer you look back to determine what he did immediately before making that purchase. The easy part is looking at digital sales from your website. Here you look at what the customer did immediately before they arrived at your site not including traffic that comes from directly typing in your domain name into the browser bar (Direct-Type-In or DTI). If they came to your site as a DTI then you look back to see if you can identify a previous visit. How far do you look back? It says right in the name: 30 days. If you can’t identify them after going back 30-days then you just call the source, “DTI”. If you can identify that they came from somewhere else in the last 30-days then that ‘channel’ gets 100% of the credit.

Here’s an example:

I search on Kayak for a flight to Mexico and they tell me Expedia has a flight, so I click over and take a look. I’m not ready to buy so I bookmark the page and go home to talk to my wife. The next day I keep looking around. I search for “Flights to Mexico” on Google and click on an ‘organic’ result landing me on Expedia again. I look around and sign-up for their email program. The next day I search a little more specifically for “Packages to Cancun”. I click on the Google Ad, landing me on Expedia again. I look around the site and make a note of all the flight options for the two of us. The next day Expedia has got smart and they send me an email with some great flight deals to Cancun. I click on the Email and I finally buy the ticket.

30-Day last touch would say that I am an “Email Customer”, and that the Email Channel is 100% responsible for this sale.

Obviously from the script above that is not the whole story, but that is how 30-day last touch works.

Why would smart companies do this?

You don’t have to try too hard to come up with lots of stories like the one I describe above. So obviously 30DLT isn’t right – why not make it more sophisticated? The first answer is that 30DLT is easy. It nicely assigns every $ of revenue to a channel. Now you can assign channel owners who are responsible for the revenue their channel generates. Since figuring out the costs of their channel is pretty easy, you can now create Vice Presidents each with their own P&L they can optimize. And once you have a company structure set up this way, it’s really hard to change.

Truth be told, it’s not a bad method most of the time.

 

Another Easy Way: 30-Day First Touch

The basic principle of 30DLT is that even if the last channel didn’t do all the work to drive the sale, at least it was the channel that pushed the customer over the edge – it’s the channel that ‘closed the sale’. But there is another way of looking at it. What about the channel that first introduced the potential customer to your brand? That sounds pretty important too. It’s so important that the second most common attribution method is “30 Day First Touch” (30DFT).

30DFT works the same as 30DLT only instead of looking at the last 30 days and giving credit to the most recent channel, it looks at the same time period and gives all the credit to the first channel. In the story told above it would mean Kayak would get 100% of the credit for the flight purchase to Cancun (even if the original Kayak search was for Acapulco or even for something completely different like France)

There are pretty compelling arguments for both 30DLT and 30DFT. The arguments are so compelling in fact that different companies stand by their belief that one is better than another – even within the same parent company. I have never seen anyone articulate any real reason why either company should do one vs the other. The true answer is that both first and last-touch are important. In fact every touch is important, which leads us to the third most common method…

 

30-Day Any Touch

If every touch is important, and we aren’t sure how important each touch really is, why not give them all credit? That’s what this method does. For each transaction, it looks back 30-days and gives each channel that touched the customer a percent of the credit based on that channel’s share of the touches. So in earlier example it would say Kayak (25%), SEO (25%), SEM (25%), and Email (25%). The first obvious challenge with this method is that channels are given fractions of transaction credit. At the extreme a channel could be getting credit for fractions of a penny. In practice things are usually rounded to whole numbers for any reports, but it still makes things complicated when, for example, you want a report of all transactions driven by email marketing: Do you show all transaction where Email took part, or just the ones that Email owned 100% of the transaction? If you show the partial transactions, do you show them as whole transactions, or just the percentage that Email claimed? If only show the percentage, then how to you communicate average transaction size? If the entire transaction, then what happens when you sum all the channels and you end up double counting (or triple or more) transactions and reporting way more revenue than your company actually generated?

These complications are why most companies choose either first or last touch for their attribution model. They assign each transaction to one channel only and damn the consequences. The thinking is that for every transaction Email steals from Kayak, Kayak will steal a transaction from Email. While individual transactions will be in the wrong place, everything will balance out in the end.

They are wrong.

 

Channel and Lifecycles

There are two errors with that thinking. The first is that different channels influence customers at different points in the lifecycle. In cases of more than one channel touching a customer, some channels are more likely to touch them early and some are more likely to touch them late. The easiest to understand example of this is Email. It is (almost) impossible to sign-up for Email with a company without having visited that company’s website due to some other channel. Therefore every Email transaction only came about because of some other marketing channel. Or to put another way if you only have an email channel you will never have any transactions at all.

On a smaller scale this happens with all channels. Branded searched (where a customer types “Your Company” into a search engine and then clicks on the first link to your homepage) is unlikely to be their first channel of interaction with you. They were influenced somewhere else first (even if it was an untrackable channel like “Word of Mouth”). Kayak has great search functionality and many people may start their search there (making it an early channel). Most coupon sites on the other hand (also categorized as “Affiliates”) are used after the fact. People figure out what they want to buy and then search to see if there are any coupons they can use – so it becomes a late channel. If a company uses Last Touch, they will under-value Kayak and over-value coupon affiliates. If they use first touch, they will over-value Kayak and under-value coupons for closing the deal.

We see the same effects within channels. Some search terms are ‘early terms’. Terms people use when they are just starting to look around. Other terms are used after they get serious and are ready to close a transaction. What the early and late terms are will vary dramatically across industries. In my experience it is not intuitively obvious what the early and late terms are. Sometimes the early terms are specific (“Assisted Living near grocery store in Belltown, Seattle”) and sometimes  they are general (“Assisted Living Seattle”), but sometimes it is reversed. You will need to run tests in your own industry to be sure.

If you stick with one type touch-model you will have the issue of over-valuing and under-valuing different channels – which means you will likely find yourself under or over-spending vs where you should be.

 

The Simple Solution

Note that even the proportional method does not even pretend to get you the “right” answer. It just offers up a way to spread the value around. One way to think about it is to stop trying to get to the ‘right’ answer. Right answers are for academics. What you really want are methods that help you

  1. Run an effective business
  2. Spend your money more effectively than you do without the method (even if it isn’t perfect)

What do I mean by all that? Here is a simple example:

Ignore “free” channels.

One could use the standard 30-day last touch, but skip past any channel that was effectively free. If you don’t count your SEO or Email channels they will be under-reported, and your paid channels (SEM, Affiliates) would be over-reported. In the original example above that means you would not count the Email touch, you would not count the SEO touch, but you would count the SEM touch – and give the SEM channel 100% of the credit. From that example it would still mean that Kayak was ignored, but it also meant that (unless no paid channel touched the customer) you would always assign revenue to a channel where you were paying for the customer.

This has the advantage of ensuring you are not under-spending on the variable channels. Since SEO and Email were essentially free on the margin, you basically end of “doing” them as much as you can – or at least dollars would not be the limiting factor on doing them. Usually for Affiliates and SEM money IS the the marginal factor, and this method, since it gives more attribution to those channels, gives you more willingness to spend on them. There is always going to be a trade-off between how much you spent and how much traffic you generate. If you under-valued the traffic you generating you would rationally reduce the amount you are willing to spend. By giving the paid channels credit every time they are involved in a transaction you reduce the chances of underspending on those channels.

But wait Ed! You might reduce the under-spending, but could this method result in over-spending on the paid channels – since they are taking credit from the free channels. We can assume the free channels are doing some work, if we give 100% of that credit to the paid channels, aren’t the paid channels getting too much credit now?

Nope.

Let’s say that a transaction makes you $10 and was 100% due to SEM. You should be willing to spend up to about $9 to get that revenue.

What if SEM was only 50% responsible (or even 10% responsible)? If the other channel that was responsible was free, then we would still be willing to spend $9 on SEM to get that revenue. The percent the paid channel is responsible doesn’t matter.

Well… It matters to the Vice President that is running that channel who wants their numbers to look good, but it doesn’t matter when making individual spending decisions.

Doing it this way (I call it 30-Day Paid Last Touch – 30DPLT) does not solve all your problems, but it gets you some of the way there and keeps the simplicity of the standard 30DLT models. (Even with that simplicity you will have the problem on not being able to compare your Year-over-year numbers, but that should be solvable in most organizations).

Note that this model only works if you don’t double-count paid channels. If you need two different paid channels to complete the sale and you spend on both with 20% remaining margin, we would be spending 160% of the revenue you generate. So by giving an affiliate zero credit they will only optimize on the revenue where they generate a  ‘last touch’, ensuring that neither channel was over-spending.

 

The Second Problem: Synergy and Cannibalization

Now we get into two ugly words that will come up again and again in this book: Synergy and Cannibalization. They are two sides to the same coin. Together they consider that possibility that 1 + 1 = 2 is not true.

Synergy is what happens when two channels build on each other. I saw this often when I was working in telecom. My client was doing outreach to their customers using mail and phone. When they sent mail on its own the response rate (i.e., the % of people who signed up for the product they were promoting) was 1%. When they called the same type of people the response rate was 2%. But if they sent people mail and then followed up with a phone call the response rate was 1% + 3% = 4%. And if they send mail after calling the response rate was 2% + 1.5% = 3.5%. It turned out that a 1% response rate on mail was not profitable and a 2% response rate on phone was barely profitable, but a 4% response rate on both was very profitable. Looking at each in isolation would mean shutting down both programs, but by combining them they were able to make the program work great – even though the mail program by itself looked terrible without looking at its ‘synergistic effects’.

The opposite of synergy is cannibalization. Cannibalization is what happens when one and one equals less than two. Cannibalization is far more common than synergy. The best example of cannibalization is coupon sites. You partner with a coupon site and they start sending you 100 customers a day. It looks great, until you realize your overall business has only gone up by 10 customers. What happened? Was it a coincidence that your other marketing started under-performing at the same time as you launched your coupon partner? (Guess what the answer is?) What happened is that people who would have bought from you anyway instead went through the coupon site for a discount.

The only good way to know how much a channel cannibalizes from existing channels is to run A/B tests (testing is the subject of another chapter). Running A/B tests is not always possible, but without them you should be very cautious adding paid channels that ‘look’ like they might cannibalize.

What does a ‘cannibalizing channel’ look like?

It looks like a channel that is targeting people who might already be your customers or is offering discounts such that it would be beneficial for your customers to switch channels. This is one good reason to be very very careful of any discounting. Most discounting will drive incremental sales, but every discount will cannibalize some of your full paying customers. Understanding if discounting is a good idea and how to do it well is, you guessed it, the subject of another chapter.

Cannibalization matters a lot less when the channel is free (or just cheaper). If your SEO improves and starts cannibalizing your affiliate channel – that’s a great thing. Even if your volume does not go up, at least your costs will go down. If affiliate starts cannibalizing your SEO though, then you are in serious trouble.

When a company is set up with separate P&Ls for each channel it has set up incentives for channel owners to cannibalize from each other. When that happens you either need very honest channel owners who will not do that (even on marginal decisions) and very hands-on overall owner who will stop them, or some sort of attribution czar who ensures the cannibalization only happens towards the free (or cheaper) channels. Sometimes all three are not even enough, but they can help move in the right direction.

 

Some Examples of Cannibalization

Before I leave the topic of cannibals, I thought I would give a few very common examples of things I see companies doing all the time. Most of these we will come back to later in the book:

  • Coupons: I already gave this example above. Be very careful offering coupon codes on the internet. Many people will get to your purchase page and then stop and look for a coupon. You’ve just given your customers a discount AND paid an affiliate fee to the coupon website. Even if you think your coupon is secret (like you send it via email) it will often get out there and be put into the broader population. If you have a VP of Affiliates they will have an incentive to create coupon affiliates – since their channel gets the credit. This is why RetailMeNot was one of the most successful internet companies in 2013 – bad attribution decisions
  • Discounts: Similar to coupons, every time you discount your product you will be attractive to people who will purchase at the lower price that would not have purchased otherwise. But you will also discount for many people who WOULD have purchased at the full price. Beware of trying to measure the impact of any discount program without an effective control group.
  • Re-targeting: Display advertising tends to not work very well in “Tylenol” businesses. But there are lots of companies out there that will pitch you “re-targeting”. Re-targeting takes people who have visited your website (well, computers that have visited your website) and shows them ads to get them back to your website. There are definitely uses of retargeting (again, in another chapter), but most people who sell this to you and claim it works ignore all the cannibalization issues. It turns out that if you do proper A/B test you will find that many people that come back to you by clicking on re-targeted ads would have come back to you anyway. Maybe 80% of them. That’s a very high cannibalization rate, but it makes sense. By definition they are targeting people that were already interested enough in you they visited your website.
  • Really Big Companies:  The bigger your company and the higher your penetration the more at risk you are for cannibalization. If someone talks about a book on their website their readers that buy that book are likely to do so on Amazon – whether the page has a link to Amazon or not. But if that link goes to MyBook.com where I am selling it directly to the public and someone buys, there is a good chance that sale is incremental (if that link hadn’t been there the purchaser would just have bought from Amazon). So the bigger your company the more you need to worry about cannibalization from one channel to another (extreme case: If your company has zero sales then there is nothing to cannibalize…)

For more on online attribution, it’s worth reading Avinash Kaushik’s post on the subject

And then move on to the next chapter where I explain the easy way to implement an attribution system and explore how it interacts with offline channels.

Please note: I reserve the right to delete comments that are offensive or off-topic.