Sunday, 20 November 2016

CRO statistical significance and beyond



How do you know when your AB test, split test, multivariate test or any other conversion rate optimization task is complete? When do you know you've found a winning experience that you can confidently apply to your web experience and expect the conversions to come rolling in?
Well the key thing is statistical significance of course, where essentially the volume of visitors to your test have voted with their feet, you've separated signal from noise and so on...
BUT, and it's a big but, have you allowed your test to play out through key trading times?
In my current company  we have a number of trading factors that mirror many other eCommerce businesses:

1. Day of week













2. Visits by hour




Once a CRO test has lived through several of these, plus a number of other factors specific to our industry (seasonality)  we can safely say the new experience has done it's tour of duty and is fit to implement.

Reaching a statistically significant result in itself is therefore not enough. Conversion optimization experts must ensure that their test audience takes into account critical factors that impact their tests.  This is important because one variation may appeal to one season or segment more than others and ultimately misguide the result.

Friday, 3 June 2016

Discount code testing on checkout

Been running ANOTHER personalisation test using VWO again on the checkout. This time targeting people who qualify for a discount code and presenting that code either just ahead of the promo code box or actually pre-loading the promo code box with an applicable discount code.

Result? 27% conversion when you put the promo code pre-loaded into the box versus 5.26% conversion when no code provided. 


Wednesday, 1 June 2016

Remove header & footer from checkout

I'm going for a different post style in a bid to post more frequently. I'm going to tell you what I'm testing and the test result. That's it....

We have an online checkout (basket etc). We tested removing the header with the primary navigation and the footer with its various site links. Why? Because these areas represent leakage points for customers and can also be distracting. Removing these escape points helps focus the user on the task at hand, ie, checking out!

Result = 22.7% conversion versus 10.29% conversion rate.