Tuesday, 29 July 2014

What's the process for deciding what to test?

I constantly have to remind myself of the process I should go through in identifying what I need to test, the best use of testing resource and how to get buy in. To this end I've done a mildly influenced by Mad Men presentation here:


P.S The online presentation tool I used was Emaze

Tuesday, 1 July 2014

The ethics & morality of A/B testing - #FacebookExperiment

I was recently sent this article from Techcrunch on the 'Morality of A/B testing'. In brief the article criticises Facebook for recently conducting an A/B test of its newsfeed.It manipulated a subset of it's users newsfeed so that some were shown a higher proportion of negative stories over positive to see if that would impact on these same users on posts; i.e. would they post more negative items as a consequence? It sounds like the outcome was yes, people were influenced to be more negative when exposed to negative news. Techcrunch however questions the morality and ethics of conducting such a test in the first place and that users should at least be offered an opt out citing that people with emotional issues had been dangerously exposed to this unwitting exercise. OK so you cannot lump all A/B testing exercises into the same boat with this one. Conventionally testing has been used to achieve a commercial or service outcome not an emotional change. Perhaps people are right that Facebook shouldn't take it upon themselves to manipulate peoples experiences in this way for their own ends. However a sense or perspective is needed here. It's Facebook we're talking about here after all. If your personality is so fragile that you need to be shielded from any potential bad experience, negative page content in this instance, should you even be on the internet in the first place? To me this is a veiled stab at big companies use of our data, which is fair enough in some instances where trust has been abused. I resent optimisation testing being used in this way, painted as some dark art that should not be deployed without peoples full consent. Secondly it smacks of the Nanny state. Why do we think we should cosset and protect adults in this way? I worry that if this starts to go down the whole cookie directive opt in/out route we'll damage online testing and experimentation and inhibit a whole industry in the process. If we start telling people that they are part of a test group or experiment online and that they can opt out of it you're going to compromise the fundamentals of testing for want of a disproportionate response to an innocuous practice. UPDATE: 3rd July 2014. Looks like the Facebook study has become a bigger issue now with the BBC now covering it here. UPDATE: 29th July 2014. This just in from the BBC, OK Cupid admit testing their subscribers to see if connects can be made with people not normally seen as a good match.