This blog looks at the work of Dr Adam Chalmers from the University of Edinburgh who has been examining how the Parliament can trace public engagement through the parliamentary process.
As with all guest blogs, what follows are the views of the author and not those of SPICe, or of the Scottish Parliament.
Parliamentary context
Citizen participation, or citizen involvement in public decision making, is one of the Scottish Parliament’s aims as being part of “a modern, dynamic parliamentary democracy.” In Spring 2022 the Citizen Participation and Public Petitions Committee launched an inquiry into how people’s voices are heard in the work of the Parliament. A Citizens’ Panel – a diverse group of members of the public – was established to support this inquiry. One of the recommendations from the Citizens’ Panel included the development of a framework for measuring the impact of deliberative democracy.
In September 2023 the Committee published a report on the inquiry: Embedding Public Participation in the Work of the Parliament. As part of its report the Committee agreed with the Citizens’ Panel recommendations which look todevelop a framework for measuring the impact of deliberative democracy, and to provide feedback on the outcome of any engagement.
In Summer 2023, Dr Adam Chalmers from the University of Edinburgh undertook an investigation to see if it is possible to track public engagement through the parliamentary process. The aim of this project was to see how we can support the Parliament’s use of participation by demonstrating that people’s voices are heard, and to understand how we can report this back to participants and the public.
When we talk about the parliamentary process we are talking about a range of different things. When primary legislation (introducing new laws), goes through the Parliament it goes through three stages. This includes committee meetings, reports and debates in the Chamber. For secondary legislation (updating existing laws), this will be simpler as there are not multiple stages. Committees also hold inquiries and scrutinise Government policy and decision-making outside of the legislative process, including by exploring Public Petitions.
The rest of this blog will look at the results of this work.
Brief outline of the process
The Scottish Parliament has been making increased use of digital engagement tools such as Citizens Space, to facilitate committee inquiry engagement work. This has given the Parliament access to large amounts of structured data on public responses to committee inquiries.
The study examines these written responses and assesses if and how these responses appear in committee meetings, reports, chamber debates and laws including the content of SPICe papers.
For the study, ten pieces of scrutiny work with different characteristics were chosen, ranging from primary and secondary legislation to petitions. Examples include topics like Gender Recognition Reform, which received about 12,000 responses, and the Community Empowerment (Scotland) Act 2015, which got 165.
The research used a method called “cosine similarity”, from a school of study called natural language processing, to measure textual similarity. This technique checks how similar the public’s consultation responses are to official parliamentary documents. The similarity scores are then sorted into categories ranging from “very low” to “high” to give a clear understanding of the impact.
Brief look at results
So, what has this analysis told us about tracking public engagement through the parliamentary process? The study had three major findings:
- Not all consultations show the same levels of similarity with parliamentary proceedings.
- Responses from organisations tend to have a higher similarity score than those from individuals.
- There is less similarity with consultations as you progress through the legislative stages
We will now investigate these findings in a bit more depth.
Levels of similarity: The analysis of ten cases, as shown in Figure 1, revealed a range in similarity levels. None of the cases had a “Very low” level of similarity. Two were classified under “Low”, four under “Moderate,” and four under “High.” Four out of the ten cases demonstrated a “Moderate” or “High” level of similarity.
Difference in similarity scores between individuals and organisations: A key discovery was the difference in similarity scores between individuals and organisations. Organisations’ similarity scores were nearly three times greater than those of individuals in the consultations. This is perhaps unsurprising as organisations are more likely to use language which is more like Parliamentary language. However, there was one case, specifically the Licensing of short-term lets, where both individuals and organizations exhibited similar levels of similarity.
Decline of Similarity Scores Over Legislative Stages: When examining the similarity scores of public consultations across three legislative stages, there was a noticeable trend. The public’s scores were highest in Stage 1 but saw a decline by Stage 3. Notably, Stage 2 exhibited the lowest similarity scores for the public, categorized as “Very low”. Within Stage 1, the public had the greatest scores on Petitions and Committee Inquiries, followed by Secondary Legislation, and then Primary Legislation. Again, this is perhaps unsurprising as later stages of the legislative process focus on amendments to legislation and the language becomes a lot more technical than at stage 1.
Conclusion
This project started out to see if we could answer the question: can we track public engagement through the parliamentary process? As can be seen, by looking at the similarity between the text of consultations and parliamentary proceedings, we can find some evidence to inform this. But it has also provided evidence of the issue around the use of language within the Parliament. Higher similarity of language is seen for organisations as compared to individuals as they are more likely to use Parliamentary language. However, the concept of Parliamentary language feeling exclusionary is not a new concept, as highlighted in a recent debate on the Citizen Participation and Public Petitions Committee report.
This research has given us a basis to develop a mechanism for measuring the impact of deliberative democracy. While impact is quite a vague concept, looking at text similarity has given us a good start. But it has also provided us with a way to gain insight into how the Parliament uses language.
If you are interested in finding out more about the Parliaments work on participation SPICe have publishes several blogs exploring the topic:
- Turning the lens inwards – the Citizen Participation and Petitions Committee’s Public Participation Inquiry
- Recommending a more participatory future – the Citizen Participation and Petitions Committee’s Public Participation Inquiry
- A shopping list for Scottish Parliament citizens’ assemblies? – the Citizen Participation and Petitions Committee’s Public Participation Inquiry
- Embedding Deliberative Democracy in a Participatory Parliament
Dr Adam Chalmers, University of Edinburgh and Andrew Aiton, SPICe