COVID-19 statistics are everywhere, and can change on a daily basis. Some of the statistics can be interpreted in misleading ways. This blog explains which statistics can tell us how effective COVID-19 vaccines are, and which statistics can’t.
Are COVID-19 vaccines effective?
Since the vaccination programme began in December 2020 almost 12 million COVID-19 vaccine doses have been administered in Scotland (Public Health Scotland data, as of 25 January 2022):
- Ninety two percent of the population aged 12 years and over have received at least one dose.
- Everyone aged 18 years and over has been offered a third or booster dose, and 73% have received this.
We know that the vaccines are highly effective at protecting individuals from getting severely ill with COVID-19. Evidence for this comes from vaccine effectiveness studies – such as the EAVE II study in Scotland – which have been specifically designed for this purpose. There is consistent evidence emerging from multiple studies that, compared with unvaccinated people, the risk of hospitalisation or death due to COVID-19 is more than 90% lower for those who have received two or three vaccine doses.
No vaccine is 100% effective, however. COVID-19 cases, hospitalisations and deaths can still occur among vaccinated people. We also know that vaccine effectiveness decreases over time. This means that a vaccinated individual’s risk of getting severely ill with COVID-19 will increase with time since their last vaccine dose.
COVID-19 statistics for the Scottish population
Routine statistical reports such as the weekly reports from Public Health Scotland tell us how many COVID-19 cases, hospitalisations, and deaths there have been in the Scottish population, and often report the figures for vaccinated and unvaccinated people separately. These simple rates tell us about population impact and help to inform the health service response, for example, but comparing the rates cannot tell us how effective COVID-19 vaccines are.
For example, Public Health Scotland report that, in the second week of January 2022, the COVID-19 case rate in Scotland was lower for unvaccinated people (age-standardised rate of 424 per 100,000) than for those who had received two doses of a COVID-19 vaccine (886 per 100,000). But – and this is the critical point in interpreting these statistics – this does not mean that the vaccines are ineffective.
Underlying these simple rates are important differences in risk, behaviour, and testing between the vaccinated and unvaccinated groups. These ‘statistical biases’ can skew any comparisons between these groups, and could lead to misleading conclusions being drawn.
The reasons are explained in detail in blog posts by Public Health Scotland and the UK Health Security Agency. We summarise these below.
1. The size of the unvaccinated population is known to be over-estimated: this will result in under-estimated rates.
Rate calculations need health outcome counts (cases, hospitalisations, or deaths) and population size data. For example: 5 COVID-19 cases occurring within a population of 1000 would give a rate of 500 cases per 100,000 people. The GP registration database (known as the Community Health Index, or CHI) is used for Public Health Scotland’s rate calculations, because it includes an individual’s vaccination status as well as their health outcomes. People who have interacted with the NHS in Scotland recently (e.g., when getting vaccinated) are likely to still be in the country when the health outcome data were collected. This results in reliable population estimates and rates for groups that have been vaccinated, particularly those who have received a booster dose. In contrast, some people in the database will have left the country without de-registering from their GP. But, because there is no record in the database of them having left the country, they will still be included in the population count for the unvaccinated group, leading to this population being over-estimated, and therefore its rates being under-estimated.
2. Vaccinated and unvaccinated groups behave differently.
The two public health agencies state that, compared with unvaccinated people, vaccinated people may be more health conscious so more likely to regularly test for COVID-19, and more likely to be identified as a case. More testing increases the likelihood of getting a positive test result, and results in higher case rates. There may also be differences in their social interactions, which would affect their exposure to COVID-19.
3. Vaccinated groups are likely to be more vulnerable to COVID-19.
Vaccination was prioritised for individuals considered to be more susceptible or more at risk. Those that were first in line included many at higher risk from COVID-19 because of their age, occupation, or underlying health conditions. Differences in the vaccinated proportion of each age group help to demonstrate this prioritisation. Additionally, any individuals in the priority groups who have only had one or two doses would have received these over six months ago, and the protection offered by these doses will have waned.
The take-home message from this blog is that COVID-19 vaccines are effective at protecting individuals from serious illness. Vaccine effectiveness studies will continue to provide us with evidence about this. Any claims to the contrary are likely to be based on a misinterpretation of the statistics, particularly if the statistics were not designed to give information about vaccine effectiveness.
Elizabeth Richardson, Senior Researcher, Health and Social Care