For the past 15 years web analytics has been namely about one thing, counting. Counting is very important and as an industry we have become infatuated with it. Never before could a medium really count what was happening with it. Attributable media was and continues to be truly revolutionary.
Attribution quickly formed the business models for the new medium — traffic buying, affiliate marketing (have we forgotten how Amazon got to be Amazon), SEO and Email are all channels that emerged based on one principle. You could count what you got in return.
In measured media, performance becomes its driver and thus optimization became the obligatory next progression. However for much of digital media, namely site content/user experience & display advertising, it has been a long slog towards optimization. A medium capable of being optimized to deliver persistent relevance may only be (by my best estimate) about 5% there.
Of course if we want to look for optimization inspiration we look at Google. Paid Search serves as our shining example of what optimization is capable of. Many are trying to emulate their models for success but search is only a small piece of our digital experience.
Still, there is one important lesson we can take away from search. We can optimize the medium up to the point of diminishing returns. This is simply amazing. Yet, most “experts” and certainly C-level execs don’t even realize this is possible. Most clients will not believe it. But it is an economic law (and one that data and ad exchanges will put to the test).
Not to be lost in all the dashboards and spreadsheets is what we are counting – the experience of people. Performance is based on the actions that control the medium – content, experience and people’s response. There is of course the ability to optimize the buy-side however if we have learned anything as performance marketers it is that the sell side optimization factors most highly into optimization. In this regard we can simply define our goals as optimizing the presentation and delivery of content.
This feedback loop is one reason the web experience has the amazing ability to be optimized and should improve for years to come…if we can get there. Optimization is hard. It requires some measure of testing and analysis — from bidding systems to creative performance. And data is a funny animal. There is no such thing as a perfect data set and often there are outliers. As we’ve seen this past year, even the world’s best systems are unable to prevent market forces from skewing predictive models to the point of rendering them useless.
There is also the “Predictive Paradox.” As we witness everyday in real-life people’s behavior is simply unpredictable over short periods of time. Yet incredibly and more often than not, over longer periods of time the differences in behavior normalize without significant variation. What this means to behavioral marketers is two things: 1) it is really hard to accurately assess wins that take place over short periods of time & 2) it is really hard to improve results by a large margin over a long period of time.
The other difficulty with optimization besides statistical significance is quality of data — how much noise is in the data set. Optimization of ads or any content has so much to do with the surroundings. Content, layout, visitor source, visitor familiarity with site, and probably about another hundred or so factors. With so much noise the more complex the testing the more false positives in the numbers. Yet for some reason we’ve been led to believe complexity is sexy when in fact it is accuracy that is sexy.
There is one last thing we should never forget as we spend more time optimizing digital media and it may be the most important aspect. Counting tells only part of the story – it tells you what people did. It does not tell you why. It does not provide answers or shed light on the actions people never realized they could take. To a large degree these answers can only be achieved through testing methodologies and observational research. Be wary of any predictive or targeting system not continually validating its assumptions with these methods…and good luck optimizing.
