Risk and A/B testing

ab-testing-problem-hypothesis-intro

A/B testing

As a marketer, I am fairly left brain oriented, some may say flakey and opinionated, and I have done well with that for many years.

Here is the paradox.

You can now test just about everything if you try hard enough. All sorts of ads, headlines, copy size, placement, colour, the best mix of paid and organic media, channel A Vs channel B, and so  on. There is no longer any excuse not to test, to quantitatively know what works best, to be able  to calculate with a pretty good level of confidence the outcomes of some marketing activity.

There are also some great resources to help think about the topic, Avinash Kaushiks blog being gold, as well as books like “A/B testing: The most powerful way to turn clicks into customers”.

There is a trap here however.

Reliance on data to inform decision making can become a crutch that stifles the left brain driven capacity to connect logically unconnected dots in some new way.

Years ago I was faced with a dilemma.

I just “knew” that rectangular 1kg yoghurt tubs would be better than the existing round ones, better for the retailers, better for consumers, just lousy for us as the producer, as margins were at risk from the higher costs, or volumes at risk from higher prices had we chosen to recover all the incremental costs.

Problem was that the round ones were industry standard, and were so for a reason, they were substantially cheaper, easier to print, and all the filling and collation equipment was designed for round tubs. I had to wait 6 years to do an A/B test by subverting a capex process of an equipment upgrade in a factory  by substituting rectangular tubs for round. Not a simple proposition when you consider all the supply and distribution angles that had to be covered.

Outcome: rectangular was vastly preferred by consumers (I somehow  “knew” that) and retailers as they achieved better shelf utilisation, which we were able to calculate and demonstrate to them. It turned out the cost premium was easily recovered in the incremental sales, and the dynamics of the market were changed in a fundamental way.

I could have, probably should have, lost my job for that piece of subversion, and had it tanked, I am sure I would have, but it would not have gone ahead with full disclosure in the capex process.

Some things are still really hard to digitally A/B test, you still need the market instinct and market risk taking mentality to have a go, but the personal cost has the potential to blow out in the wrong environment, but without the risk, there is no progress

About strategyaudit

StrategyAudit is a boutique strategy and marketing consultancy concentrating on the challenges of the medium sized manufacturing businesses that make up the backbone of our economy. The particular focus is on their strategic and marketing development. as well as the business and operational efficiency improvements necessary for day to day commercial survival. We not only give advice, we go down "into the weeds" to ensure and enable implementation.
This entry was posted in Change, Innovation, Social Media and tagged , , , , , . Bookmark the permalink.

One Response to Risk and A/B testing

  1. Pingback: Incremental change is a journey | StrategyAudit

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s