Idea

This idea is about how you can improve your business strategy by applying learnings from the British government and the field of behavioral economics. I want to introduce one of the most important processes any CEO or CMO can pickup today, which is the process to “Test, Learn and Adapt” in conjunction with randomised controlled trials (RCT:s). For those of you who are unfamiliar with the concept of randomised controlled trials, this is when you test different conditions over randomly assigned groups of participants to benchmark and compare them to a control group which has not been subjected to any testing, hence finding out if the “intervention” created was effective or not. It scientific way to prove efficiency of different interventions.

The last few years paper after paper has been published about how inherently flawed human decision making is, or as Dan Ariely, known author on the subject would describe it: people (and consumers) “are predictably irrational”. I believe that within marketing and business we can create a better choice architecture by RCT experiments and by following the process of testing, learning and adapting.

Background Research

If we look at this from a marketing perspective IBM conducted over 1.700 interviews with CMO:s in 64 countries to find the most successful marketers and the results of the study is clear, the most successful CMO:s focuses on deep human insight instead of focusing on the actual product or service that they are selling.

“To succeed in the digital era, you have to be totally in sync with the behaviour and preferences of your customers in a fast-changing landscape. You have to be quick and adaptable.” – CMO, Retail, United States

How is this applicable to our subject? One way to reduce guesswork in what consumers truly care about and how they relate to it is to explore and test it, then learn and adapt to their preferences in an approach similar to that of public policy cases in Great Britain.

So how do we apply RCT to business? The Behavioural Insights Unit in the UK suggests a 9 point process as follows (slightly modified below to suit marketing and business needs):

Test

1. Identify two or more business interventions to compare (e.g. old vs new marketing message; different variations of a similar marketing strategy).
2. Determine the outcome that the marketing message is intended to influence (e.g more sales, brand uplift etc) and how it will be measured in the trial.
3. Decide on the randomisation unit: whether to randomise intervention and control groups at the level of individuals (online marketing for instance can track down to specific IP addresses) or geographical areas (e.g. similar sales districts).
4. Determine how many units (people, companies, or selling areas) are required for robust results (make sure the sample is large enough to be representative).
5. Assign each unit to one of the marketing messages (conditions), using a robust randomisation method.
6. Introduce the marketing messages (or what you actually are testing) to the assigned groups.

Learn

7. Measure the results and determine the impact of the marketing message (tested condition).

Adapt

8. Adapt your marketing message to reflect your findings.
9. Return to Step 1 to continually improve your understanding of what works

Potential Experimental Design

What I suggest is that businesses should adopt the use of RCT and create a culture of test, learn and adapt to enable a better decision making architecture regarding everything from product investments to marketing strategy. I would argue that this should already be a hygiene factor within businesses today; to be able to adapt future decisions efficiently. We might have a hunch about what works, but as human beings we consistently over evaluate our decision making capabilities, RCT:s and testing the gathered data will however tell the story of the most efficient solution.

Value

If we can enable businesses to adopt an approach of test, learn and adapt in combination with regular RCT:s according to the 9 key points above we can significantly increase value in organizations by creating a learning, data driven and cost efficient organisation that is well suited for the future however it might look.

Andreas Roos, The Choice Architect

Sources:

 Stepping up to the challenge: CMO insights from the Global C-suite Study

Dan Ariely: we are predictably irrational

Randomized Controlled trial

Test, Learn,Adapt: Developing Public Policy with Randomised Controlled Trials, Haynes, Service, Goldacre, Torgerson

(2012)Test policies in randomised controlled trials.