When Was The Last Time You Did An Experiment?

Experiments are the stuff of science. Subject of ‘Through the Wormhole’ or appearing on IFLS, we hear about them everyday in connection to a great discovery or major achievement, but never really that close to us. What we marketers miss is that experiments are all around us, everyday. And what untapped potential they have!

Source pixbay.com

Image Source pixbay.com

Even if you didn’t think of it as such, you probably did an experiment as recently as last week. The reality is, most of the things we do in online business are an experiment. There is no one guaranteed recipe for success because, although it might feel like the Internet has been around forever, in the big scheme of things, we are still just toddlers learning to take our first steps.

Online experiments are already starting to gain popularity. Here are three of the best from 2014:

We Experiment On Human Beings, Experiment Shows Up To 60% Of “Direct” Traffic Is Actually Organic Search, and 24 People, No Managers: Our New Experiment in Getting Work Done at Buffer (with more exciting ones being prepared right now!)

From how work gets done to how search engine optimization works, everything is open to experimentation and improvement. But here’s the kicker: Unless you plan some structure around your experiments, you’re not going to get the best results from them.

What Are Experiments And Why Are They So Important?

Marketing experiments work just like scientific experiments. The same basic rules apply in both cases. While this might sound worrying at first, in terms of time, effort and accuracy, it’s actually quite good news. Not only does science simplify matters for us, giving us the clear steps to perform successful experiments, it actually gives us a quick way to find right answers to our questions.

Essentially, the steps are these:

  • Note down what you are trying to put to the test.
  • Make sure the context and environment are not biased.
  • Establish the success metrics.
  • Document it so it can be replicated.
  • Run enough tests so the conclusions are valid and not “by chance”.

Feeling like you’ve gone back to school? Shake it off! You’re about to find out just how down-to-earth experiment-thinking actually gets.

Is A/B Testing An Experiment?

If I asked you to name any type of marketing experiment, would you say A/B testing? Take a moment, considering the steps above.

The answer might appear to be no, right? Indeed, if there is one key point to take from this article, it is this: Don’t rely solely on the results of an A/B test to make a change. Instead, ask yourself why users would act like that?

A/B tests are not experiments. They are methods to employ in experimenting. They allow you to test and validate your hypothesis, but they’re not the only method.

Confused? Let’s try a concrete example:

Let’s say we want to A/B test a button in our app, to see if red would work better. And then, say we discovered 70% of people clicked the red one. Great, so red works on our app. We change all similar buttons to red and re-measure the final CTR. Only this time, surprise surprise, the final CTR did not improve, it actually dropped. Well, now we’re confused! We lost time and valuable customers and gained nothing.

Or did we?

Experiments teach us how to think hacks and better execute them. Let’s rethink what we know for a fact. We learned that red worked better than blue in the account setup step.

Experiments teach us how to think hacks, and better execute them

Now, for the magical question. Why? That’s right, before you go making any more changes, ask yourself Why? And surprisingly, it can be anything: need for contrast, sensitivity to a color, a cultural thing, or even a mix of elements.

Let’s take contrast and say blue simply isn’t strong enough to get people started. It’s as if your users are in a hurry, not spending enough time there. It might also be that the text is faulty, too long or unclear, meaning they don’t have enough patience and when seeing a brighter colour, they just want to hit next.

On the other hand, when we changed the other buttons to red, we also changed the “Buy” option, and did more wrong than good. Red is an aggressive color for “Buy”, and dissuaded our users from purchasing our service.

Bonus, you gain a deep understanding of your persona

Don’t rush to conclusions. Having your hypothesis clear in your mind will get you better results and a clearer image of your user – in this case “users are confused in this step, and just want to hit next”. Simply by having this question in mind, your test results have more clarity and context.

You should test your hypothesis at least one more time before making a change. In time, it will not only save you a lot of time and customers, but it will also give you a clear and deeper understanding of your customers, without running surveys or guessing.

How to Be Off To a Great Start In Your Experiments

One great way to get started, especially if you’re a rookie in analytics, is to simply replicate something. The Internet holds a vast collection of free marketing experiments accurately documented so anyone can replicate them. Innertrends has some great stuff in its pipeline.

Here are some tips to narrow down your search:

  • Start with your customers. What is their journey to you? Pick an early starting point to get the most out of it.
  • Choose things which didn’t live up to your expectations. Just inserted a feature which you felt would attract usage, only to find it’s not so popular? Work on that.
  • I always add one more question, to make sure I’m playing by the ‘nice guy’ rule: Would people be upset if they found out about the experiment?

As time goes by, you will become addicted to experiment-thinking and you will start running your own experiments without even realizing it.

Image Source pixbay.com

It’s time to open our eyes to the experiments around us. Start small, take a second look at that result, play stubborn and never stop questioning “Why?” – and you will find the keyhole that taps directly into your user’s profile.

Do you have an idea for an experiment that you don’t dare start?

Tell us all about it. We’d love to help you get started or connect you to people who can help you out.

Author: Andra

Experienced product marketer, now discovering inbound.