Big Data, Big Errors?

Taleb writes in a Wired article:

” Modernity provides too many variables, but too little data per variable. So the spurious relationships grow much, much faster than real information.”

Variables are what we analytics professionals call measures or numbers. Sales, conversion rate, average value, etc…But as our ability to collect, store and query this data improves, so does our ability to create new variables and new dimensions. Look at a typical marketer’s data toolkit – not only do they have to come up with all these metrics and dimensions, combine data from multiple sources, tools and processes….but they need to act!

The signal grows, as does the noise.

How can anyone make any sense of it all? Here are some guidelines we’ve developed after working with tons of data across decades of internal experience here at Delve – this pertains primarily to B2C clients, focused on making marketing decisions to both acquire new customers and retain existing ones:

1. Only use predictions that make sense – if an algorithm tells you to do something counter intuitive, use your brain.
2. Use a few critical dimensions, throw out the rest. Loyal customers vs. First time buyers – that’s a good distinction. 3 time buyers vs 4 time buyers, who cares.
3. Focus on optimizing traffic sources based on a single metric – conversion to a first time buyer is a great one for most growing SME’s. Larger businesses have different protocol but may be too steeped in bureaucracy and poorly designed incentives to make any optimization progress.
4. Don’t rely too much on A/B testing, instead focus on user experience testing, common sense and fast/optimized web experiences.
5. Collect feedback from your customers – integrate feedback solicitation mechanisms into the browsing experience AND post-purchase experience. Avoid overly structured surveys from big companies claiming you can benchmark satisfaction across industries, blah blah blah. Find problems, get free-form feedback and act.
6. Visualizing data rocks, but if a simple table in Excel will do, use it! Fancy doesn’t get you points…focus on making decisions.

Last but not least – USE A NULL HYPOTHESIS. Changing things for no reason is a crime many marketers are guilty of. Always say to yourself…”what’s wrong with the way things currently are”. If you can’t really concretely answer it, I think you’re barking up the wrong tree.  Tiny adjustments can create big changes…be careful calibrating when dealing with leverage of any kind.

Thoughts?

Jeff

View full article