To help chart a course through the next stages of the pandemic, the government needs to weigh up, and balance, a huge range of issues (such as those raised by the Scottish public) which all feel very different, and cover topics like health, the economy, social issues and the environment.
The good news is that, at least some of the thinking has already been done to help us tackle complex, multi-faceted challenges, and this blog highlights some of the tools available to help policy makers make decisions.
How the Scottish Government is framing its thinking
The Scottish Government set out its thinking in its framework for decision making (23 April) and provided further analysis on 5 May. Broadly, there are six aspects to its approach. Those most relevant now include
- suppressing the virus,
- caring for those who need it and
- supporting people, business and organisations.
Thinking further ahead are
- measures to recover to a new normal,
- protecting against this and future pandemics and
- renewing the country.
The government says that the principles underlying its approach are: being safe, lawful, evidence-based, clear, realistic, and collective. The framework also helps policy-makers to think about how different types of impacts could be assessed, and it identifies four harms from COVID-19, whilst a subsequent document (7 May) set out the supporting evidence to be used in making decisions
- Direct health impacts: Measures could include the number of new cases, the number of new and total hospital admissions, Intensive Care Unit admissions, and deaths
- Indirect health impacts: Measures could include the number of excess deaths, some measures of health service performance and capacity, (including accident and emergency attendances), evidence of people avoiding going to a hospital or a GP practice with non-COVID immediate medical concerns, and other effects on physical health and mental health and wellbeing.
- Social impacts: The government identifies six dimensions of societal impact (for which there are numerous indicators): safety and security, skills learning and development, social capital and community cohesion, loneliness, anxiety and fear of social isolation, economic security and welfare, and finally social contract and trust in government.
- Economic impacts: Could include higher unemployment, financial insecurity and hardship for many businesses, individuals and households. The damaging effect on poverty and inequality may be profound and there will also be gender and age-related dimensions of economic harms. The extent to which the economy can restart is an issue whilst impacts will intensify the longer the lockdown continues.
These harms clearly include a wide variety of issues. In the end, balancing these impacts means making some subjective judgement, and requires leadership. But several tools exist to help us marshal the evidence, structure our thoughts and make the best possible decisions in the changing circumstances.
Some of the policy tools that can help us make decisions
Cost Benefit Analysis
For half a century, civil servants have made use of different versions of the HM Treasury Green Book, more recently backed up by the Aqua book (modelling techniques) and the Magenta book (evaluation techniques) in appraising and making decisions on major policy areas. These tools help to make such decisions as transparent, objective and evidence-based as possible. The Green Book describes appraisal as:
“the process of assessing the costs, benefits and risks of alternative ways to meet government objectives. It helps decision makers to understand the potential effects, trade-offs and overall impact of options by providing an objective evidence base for decision making.
A key aspect of appraisal is that it should be applied proportionately. The resources and effort employed should be related to the scale of the proposals under consideration. A social Cost Benefit analysis aims to assess all impacts, including the socio-economic, environmental and financial impacts against doing nothing (as background, see previous SPICe blogs on economic assessments, or more specifically on the dualling of the A9).
How are costs and benefits valued?
The Green Book suggests that, if possible, a pound sign should be attached to costs and benefits “in order to provide a common metric”. Market prices of the best alternative would be the usual starting point, but the Green book recognises that, in many cases, a suitable market price does not exit. In these cases, non-market price methods such as “valuation techniques or a range of specific standard values” can be used. Ultimately however, standard values may not be possible for everything, in which case the cost/benefit should still be recorded and clearly described as part of the results of the analysis.
Winners and losers
Distributional analysis should also inform decisions, and this may relate to groups of people (for example as currently being considered by the Equalities and Human Rights Committee), or it could relate to the impact on different parts of the country geographically (for example in this COVID-19 community vulnerability analysis whilst the Scottish Government also uses an Inclusive growth diagnostic tool to help make regional local policy decisions).
Optimism, risk and time
Care needs to be taken to avoid optimism bias (particularly in relation to under-estimating costs or duration of new investments). Risk and sensitivity analysis (the degree to which changes to assumptions can knock plans off course) must also be considered. Another issue to factor in is the time taken to deliver different outcomes. There is an assumption that benefits sooner are better than benefits later, and the technique of discounting to create a “present value” can build that into the calculations.
Quality adjusted life years (QALYs)
In health settings, one of the key metrics used to help make decisions is the quality adjusted life year (QALY). QALYs allow for an assessment of the value of health interventions by calculating both the quantity and quality of life generated by a particular intervention. One QALY is equal to 1 year of life in perfect health, but for someone living with less than perfect health, the value is set at specific points between 0 and 1. The amount of QALYs generated by an intervention can then be compared with the associated cost to provide a cost-utility ratio. Within the UK, QALYs are usually only routinely applied to funding assessments for medicines but they can be used to aid decisions in other areas of competing choices.
In many areas there is an increasing focus on wellbeing as the way to frame our thinking and decision-making. Examples include the OECD Framework for Measuring Well-Being and Progress, New Zealand’s approach to wellbeing budgeting, and wellbeing as the central focus of Scotland’s National Performance Framework)
In a recent report, the Centre for Economic Performance suggests that policy-makers have to balance the impact of decisions on how and when to move away from lockdown upon the many factors that feed into wellbeing. One of the report’s authors, Lord Gus O’Donnell, also suggests that the evidence shows that the value of QALYs is too low in the current circumstances.
CEP suggests it is helpful to decision-making to forecast each factor using a single metric.They use the number of Wellbeing-Years (aka Wellbys) that result from ending the lockdown on different dates. This new metric, they argue, makes it possible to compare the impact of each factor in a way that is relevant to all public policy decisions.
Based on their “illustrative” figures and assumptions their analysis shows that the longer the lockdown is in place the greater the negative impacts, until a tipping point is reached.
Scotland’s National Performance Framework is aligned with the UN Sustainable Development goals. The themes of: a strong, healthy and just society; living within environmental limits; a sustainable economy; sound science; and good governance, can also provide a means of taking a cross cutting and holistic view, for example by using a Scottish Parliament sustainable development impact assessment tool.
Garbage in/garbage out: making decisions with limited information
There are well-documented uncertainties on much of the social, economic and epidemiological statistics, and indeed the pandemic itself makes data collection harder, However Lord Gus O’Donnell points out that decisions do need to be made quickly one way or another. He suggests that a cost benefit analysis is needed and that a “rough estimate of the right concept” would be better than “precise figures … on only part the picture”.
Just doing the analysis is not enough
Carrying out the analysis is the easy part and, as the Institute for Government points out, public support for any decision is crucial and any changes, and the trade-offs behind them, need to be explained clearly and repeatedly. Indeed, levels of public knowledge and compliance with Government advice are among the factors the Scottish Government will assess in its framework for decision making.
Let’s get used to it?
As recently as January 2020, the World Economic Forum set out what it considered to be the major risks facing the world in the coming years. Although “infectious diseases” was indeed amongst the risks identified, there were several others thought more likely to happen, and with a greater impact. For example, at the front of the queue of global risks over the next ten years (in terms of scale of impact) are:
- failure of climate change mitigation and adaptation
- weapons of mass destruction
- major biodiversity loss and ecosystem collapse
- extreme weather events (e.g. floods, storms, etc.)
- water crises.
With this in mind, and considering we also still do not yet know what longer-term consequences will arise from the pandemic itself, perhaps we all need to sharpen up our analytical tools so we can respond to more big impact, system wide events quickly, effectively and fairly.
Simon Wakefield, SPICe