Economic impact assessments generally show how organisations, events, sectors, or publically funded policies and projects have contributed to the economy at either a local, regional or national level.
What’s economic impact assessment about?
The aim is to measure additional economic value in the economy – i.e. “net economic impact.” Assessments can measure the economic benefits of interventions, including changes in economic growth (Gross Value Added (GVA)) and the number of jobs created.
A range of techniques, such as input-output modelling, capture direct, indirect (relates to supply-chain effects resulting from the direct effect), and induced (pertains to the increases in wage and salary spending by both the direct and indirectly affected) effects – these are often referred to as “multiplier effects”. Increasingly, many assessments also look at tax contributions, wellbeing, productivity benefits and environmental improvements.
The best economic impact calculations follow HM Treasury Green Book standards. The Green Book provides guidance for public sector bodies on how to appraise proposals before committing funds to a policy, programme or project. It is advised by HM Treasury that all new policies, programmes and projects, whether revenue, capital or regulatory, should be subject to comprehensive and proportionate assessment. This is recognised by both the Scottish and UK Governments and their associated public bodies.
The key term here is ‘proportionate’ – the effort applied to appraisal and evaluation should be proportionate to the funds involved, outcomes at stake, and the time available. Scottish Enterprise, which has a publicly accessible database of appraisal and evaluation evidence called Evaluations Online, illustrates the use of Green Book methodology by a Scottish public sector organisation.
The art of going from gross to net
Regularly large economic impact or value figures are reported in the media for different events, organisations, or sectors. Reading such reports often prompts at least one of the following reactions:
- how realistic is this figure – it seems incredibly big
- how is it even calculated
- this aggregated to other economic figures I know results in an amount that’s bigger than the total Scottish economy
- there is something of a blackbox of assumptions around this.
Much of these puzzlements result from the art of going from gross to net in economic impact assessments, which can often seem like an exercise in smoke and mirrors. Figure 1 shows the steps that need to be applied to go from the total monetary value of all economic activity (gross) to the true new money in the economy because of an intervention (net).
Please note Figure 1 provides a simplified overview of the adjustment steps required. Assessments in areas with public sector funding often require more modifications, whereas wholly private sector studies can be less complicated.
Decrypting economic impact reports
A recent report for Highlands and Islands Enterprise on the pan-Scotland Entrepreneurship Support Programme (ESP) 2013-16 illustrates some of the perplexity in deciphering economic impact reports. The report’s conclusion presents four different sets of figures, which can create quite a challenge to interpret.
- In gross direct terms the Programme had resulted in £7.7 million GVA to date.
- Net additional GVA to date was £3.3 million, including multiplier effects.
- Net additional forecast GVA was £49 million, including multiplier effects.
- Total net additional GVA was £114 million (this is an aggregate of to date and future GVA grossed up to all participants and includes multiplier effects).
To get to grips with these estimates you need to be familiar with the concept of gross to net; the difference between direct, indirect, and induced impacts; grossing up from survey sample population to total population; and forecasting future impacts which brings complexities around real pricing and discounting. Thus, is it any wonder the presentation of economic impact results leaves people confused?
A tool for informed decision making
Much of the mystification around economic impact assessment stems from the following factors:
- The lack of consistent methodology and language across assessments reduces the potential to make meaningful comparisons, as concluded by a major review of UK and overseas assessments in the cultural sector for the Department for Digital, Culture, Media & Sport (DCMS). It highlighted specific weaknesses that undermine the usefulness of economic impact studies and criticised the limited exploration of key economic concepts. It called for future economic impact assessments to seek a more “systematic consideration” of the consequences of investment.
- It is not always clear how the different factors of adjustment have been determined and if and how these adjustments have been applied. This is particularly true for ‘deadweight’ – the extent to which the measured impacts would have been attained even without investment. Also the use of varying sources for multipliers can have a significant influence on the final net impact figures.
- Communication of results from assessments can often be misleading. Following the 2014 Ryder Cup in Scotland the media ran headlines about the event adding £106 million to the Scottish economy. However, the detailed impact report shows the £106 million represented gross economic activity, including multiplier effects. Thus, a proportion of this money would have been in the Scottish economy anyway without the event. The actual net economic impact of the Ryder Cup event, including multiplier effects, was £41 million.
Despite the inconsistencies and challenges around economic impact assessment, there is agreement on the importance of it in aiding the decision making process for policymakers.
It is necessary to strengthen economic impact assessment as a tool for informed decision making, and for supporting scrutiny by the Parliament and other organisations. It should be possible to encourage the systematic, consistent and transparent use of economic impact assessment by Government and its agencies. The question is how this can be incentivised to improve the current, often unintentional, landscape of smoke and mirrors.
Alison O’Connor, Senior Analyst, Financial Scrutiny Unit