Randomised controlled trials are a proven tool for understanding the causal effects of policies. Esin Serin asks whether they can play an increased role in informing the development of effective green innovation policies in the UK at this critical time for climate action.
Technology, alongside behaviour change, is essential for the UK to bring its emissions to net zero by 2050. Progress is needed to develop new technologies for reducing emissions where current technologies fall short. The UK is already implementing policies to drive the required innovation in low-carbon technologies, with significant capital deployed along the way. But how can policymakers know whether their policies are working? Or working better than alternatives? Or that they are creating additional benefits and not just subsidising innovation that would have happened anyway? Randomised controlled trials offer a rigorous way of answering these questions.
Understanding the efficacy of green innovation policies
Evidence on the efficacy of green innovation policies is needed to identify the most cost-effective pathways to net zero, ensuring that public resources are not wasted but are spent on policies that create real positive impacts. When investigated thoroughly, there may be many policies not working as intended. For example, a review of studies evaluating direct funding programmes for innovation finds eight papers showing positive impacts of such programmes on patent outputs but another six showing either mixed or zero impact. Another paper – drawing on evidence from randomised controlled trials that go beyond a sole focus on innovation – goes further to suggest “80% of things don’t work”.
Evidence on what works can help unlock significant growth opportunities for the UK if it can build comparative advantage in and capture a valuable share of the growing global market for green technologies. The UK government has already adopted innovation-led growth as a long-term ambition and embedded it in its net zero strategy. Research from the LSE’s Centre for Economic Performance and Grantham Research Institute has shown that the UK has a strong base, being specialised in green technologies overall and in some specific technologies where the estimated returns on public investment in innovation are especially high, such as in offshore wind, tidal stream and carbon capture and storage.
Using randomised controlled trials to inform green innovation policies
Randomised controlled trials involve assigning subjects such as individuals, households or firms (in a randomised manner) to either a treatment group that receives the policy being studied (e.g. a research and development grant) or a control group that does not.
Given subjects are randomly assigned to one group or the other, the control group should, in theory, be similar to the treatment group, and so the difference in average outcomes between the two groups (e.g. research and development spending, patent applications) can be associated with the policy itself rather than individual characteristics or wider trends or influences.
Despite their potential, randomised controlled trials have seen very limited use in the green innovation context so far. Indeed, we identified only 29 trials globally (five being in the UK) which are either planned, conducted or under way. This is unsurprising given the challenges of implementing randomised controlled trials, ranging from cost and time requirements to the ethical implications of allocating a policy on a randomised basis, even if it is for learning purposes.
Views on randomised controlled trials are also varied: some argue they represent a methodological ‘gold standard’ for proving causal impact, while others view them as insensitive to the importance of context as they can measure impact with high confidence but on their own, tend to provide little information about how and why the measured impact came about. There are also unique challenges to implementing randomised controlled trials in the specific context of green innovation.
Can randomised controlled trials have a greater role?
Among the priority areas of green innovation for the UK, we found that energy storage and flexibility, homes and buildings, and hydrogen may present fertile grounds for randomised controlled trial-based evidence building on innovation policies. This is primarily because these areas involve some domestic-scale, consumer-facing technologies that are likely to be developed by a large number of innovators, making them conducive to randomised controlled trial-based evidence-building.
Randomised controlled trials’ contribution to achieving green outcomes can be maximised if they are used as a way to make innovation policy more innovative, shifting the focus from ‘evaluating’ if a single policy design is working to approaching the policy as an experiment in its entirety – that is, exploring a range of policy ideas, testing promising ones at small-scale and basing subsequent decisions on these lessons.
The Business Basics Fund from the Department for Business, Energy and Industrial Strategy uses an experimental approach, by supporting trials of different ideas which encourage small and medium enterprises to adopt productivity-boosting technologies and management practices. Larger trials seeking access to the fund need to have an evaluation methodology such as a randomised controlled trial that will make it possible to draw robust conclusions on what does or does not work. The recently launched Green Homes Finance Accelerator takes a similar approach, allocating a portion of its £20 million funding to pilot a range of innovative financing mechanisms which encourage domestic energy efficiency and low carbon heating retrofit.
Randomised controlled trials as part of a broader culture of evidence-based policymaking
Of course, randomised controlled trials are not the only source of evidence. Policymakers can draw on years of learning by doing and other rigorous methods of policy evaluation on innovation broadly (see this toolkit) as well as on green innovation specifically. For example, we know that progress made to date with some key green technologies was driven not by technology-neutral, generic research and development investments or carbon pricing but instead by innovation policy packages involving deliberate technology choices.
The UK’s Contracts for Difference mechanism was not tested under a randomised controlled trial, but no one can deny its role in bringing down the costs of offshore wind in the UK by around 70% over a decade. The investment signal that the policy sent also had wider and longer-term impacts on the development of the sector that may not have been captured under a randomised controlled trial design.
Randomised controlled trials are one important enabler of evidence-based policymaking rather than an end in themselves. We cannot randomise every green innovation policy, but we can be more proactive about recognising opportunities for using randomised controlled trials to learn about the role that different policy instruments can play for achieving specific technological objectives. As a result, we may learn that not every policy is having its intended impact, but in so doing, we might build an evidence base to accelerate net zero-aligned innovation and growth.
___________________
About the AuthorsEsin Serin is a Policy Analyst at the LSE’s Grantham Research Institute on Climate Change and the Environment.
Featured image credit: L.Filipe C.Sousa on Unsplash
