Posts Tagged ‘Cabinet Office’

Test, Learn, Adapt

Michael Sanders

If there is a phrase more likely to attract rolled eyes than “Behavioural Economics”, it is probably “Evidence based policy” – one has in the past been derided for not really being economics, and the other for being more apt with the first and third words reversed. Supporters of both have often despaired. There is now, however, a glimmer of hope.

The Cabinet Office’s Behavioural Insights Team, charged by the coalition with bringing behavioural science to policymaking, this week launched a paper with the title “Test, Learn Adapt: Developing Public Policy with Randomised Controlled Trials”[1], which is part manifesto for randomised controlled trials (RCT), and half handbook on how to conduct them. Although not comprehensive, its nine steps offer a simple guide to the basics of designing and running a trial.

Co-authored by the Ben Goldacre of the Guardian[2], and Professor David Torgerson, director of the University of York’s Trial Unit, the paper gives every indication that government is, or at the least is trying to be, committed to the idea of running trials wherever possible, often with the help of the academic community – which will hopefully lead to a better understanding of which policies are beneficial and which are not. Although there are other[3], more comprehensive[4] and technical guides[5] to trial design, this is an important step for government.

The second step may be even more difficult – making sure that successful intervention are rolled out more widely when proved to be beneficial.  For example an RCT by the University of Cambridge’s experimental criminologists has provided the first robust evidence of the benefits of ‘hotspot’ policing in the UK[6]. Another, by Robert Metcalfe of the University of Oxford, provides clear implications about how to reduce household energy consumption in both the short and the long term[7]. Both of these trials, although excellent, will be of very limited practical use if their findings are not used to inform future policy – this will require continued work both by policymakers, and by those running the trials[8].

[1] Haynes et al (2012): “Test, Learn Adapt: Developing Public Policy with Randomised Controlled Trials” Cabinet Office, available from:

[3] Angrist & Pischke (2008): “Mostly Harmless Econometrics” Princeton University Press

[4] Gerber & Green (2012): “Field Experiments: Design, Analysis and Interpretation” W. W. Norton & Company

[6] Presented by Barak Ariel at “Beyond Nudge” conference at the British Academy, June 2012

[7] Presented by Robert Metcalfe at “Beyond Nudge” conference at the British Academy, June 2012