Monthly Archives: August 2011

Estimating The Causal Effect Of Online Display Advertising

To all my loyal readers, I apologize for the hiatus, but life happens. About four months ago, I started a new position at Yodle leading the team of quantitative developers that work on our bidding algorithm. The first few months have flown and I have learned a ton, but have been very busy and have left little time for writing.

Or for attending meetups, but I made it to a pretty good one earlier this evening titled “Estimating The Causal Effect Of Online Display Advertising,” presented by Ori Stitelman  of Media6Degrees. The main premise of the talk is that A/B testing, now an industry standard, can be costly or impossible, and it’d be nice if we could figure out causality based on something we already have lying around – our data. Of course, we are interested in display advertising at Yodle, but the use of the techniques Mr. Stitelman outlined have other applications: how do you measure what makes customers happy? how do you measure the effect organic and paid search have on each other?

He first started off by describing a methodology for doing this kind of quantitative analysis:

  1. State the Question: What is the business problem you are trying to solve?
  2. Define Causal Assumptions: Without getting into a statistical model, what is the causal relationship between events?
  3. Define Parameter: what is the parameter you actually care about?
  4. Estimate the Parameter.

Mr. Stitelman then described a few different way of estimating parameters:

  1. Inverse Probability-Weighted Estimates: a technique that weighs an event’s importance by how unlikely it was to happen on the particular example.
  2. Maximum Likelihood Estimator
  3. Targeted Maximum Likelihood Estimator

It’s obviously difficult to digest all this kind of material in a hour long talk, but the results presented show that this technique has promise.

What I have learned so far about meetup talks is that the ones that were worth attending add a whole new load of reading to your stack, and my stack is looking pretty high.