When I started out working with data in Wales, way back in 2006, it was a very different environment!
My first needs assessment, to support the 2008-2011 Merthyr Tydfil Children’s and Young People’s plan, was a fairly rudimentary affair. It contained lots of flowing prose (which added nothing!), and a whizz through all the performance measurement data I could include. It showed how Merthyr compared to the other 21 local authorities in Wales for that year, and a little bit of timeline data to show direction of travel. There was no secondary evaluation, no situation analysis, no context, and almost no low-level geography data. Fortunately, I’ve got a bit better at this data stuff since then!
As has everyone else, including those who provide leadership and guidance in this area. Everyone with responsibility to complete Well-being assessments, or population assessments, or any form of assessment or analysis, is now well-versed in the art of accessing, analysis and contextualising data to better understand the situation or population they are looking at.
This shift in technique and focus equally applies to how we measure impact. Previously, we’d use much the same data as we’d used to complete a needs assessment (how antiquated does that seem now!), but now we are far more sophisticated in how we measure impact. The use of qualitative data is now routine and compliments the quantitative data that has always been available.
There has also been a gradual, but fundamental shift, in what we measure. The introduction of the Well-being of Future Generations Act and the Social Services and Well-being Act ask that we consider how we measure impact. What is it that we are trying to achieve, what is it that people want from services and service providers and how can we best show impact.
Perhaps the biggest shift has been the development of ideas around what our end goals are. Why are we here? What are we trying to achieve with all our planning and effort and services? Do we want to make things ‘better’ or do we want people to be ‘happier’? These are fairly big philosophical questions, but obviously pretty central to ensuring we are doing the right thing for people. It is an interesting tension between organisations who are responsible for strategically planning and providing the services (sometimes required to provide by legislation), and the public who are supported by, and consume the services. Which of those two groups gets to decide which services should be provided and what the outcome of those services should be? And what success should look like?
This position does raise an interesting philosophical question – it is entirely possible to run a good service that makes no impact on people’s well-being. Using all sorts of standard service performance measures (attendance numbers, raised awareness, attendee satisfaction, etc.), an intervention can be a success. But the trickier thing to measure is what difference an intervention has made to people’s well-being, are people actually better off because of what you have done? And is that difference tangible and long-lasting, or just a fleeting emotion? All challenging stuff, but if we are really trying to understand what makes a difference to people’s lives, are these the sorts of questions we need to answer?
So, these questions are now shaping how we decide what data to collect and disseminate. Our traditional range of indicators covering performance and health, education, transport, the economy, etc. remain absolutely relevant and important to our partners. But we have been expanding our range to meet their changing need. So, we now hold far more survey data and we are looking to develop our ability to hold other sources of qualitative data as well.
In April 2018, we launched the first version of Thriving Places Wales. Developed in partnership with Happy City, it was our first venture into well-being data, and it looks at the prevalence of conditions for well-being, which are focused on health, community and happiness rather than income and services.
The tool is based on several years of development work by Happy City, who provided the expertise and methodology. We, along with the Public Services Boards in Gwent, provided a distinctive Welsh perspective, by sourcing and including data from Welsh sources that gave the tool a different feel to the Thriving Places Index for England.
It keeps company with lots of other resources that are developing and becoming more useful to partners, such as the ONS National Well-being dashboard, the Well-being of Wales indicators and the Welsh Index of Multiple Deprivation, which looks at the absence of well-being rather than the prevalence of well-being.
And in addition to the increased focus on well-being, the issue of loneliness and isolation is also having a raised profile now. The Office for National Statistics (ONS) are bringing together several sources of administrative and survey data to look at loneliness, and Welsh Government has recently closed a consultation on tackling loneliness and isolation, which will led to the publication of a national strategy.
This does not begin to look at what well-being data might be available to you locally from services, community groups, schools and other sources where surveys and other data gathering exercises may have been commissioned. All these additional sources of data give partners options in terms of the data available to them. And whilst having too much data to choose from is obviously better than the opposite scenario, it does mean you have to make choices!
So, the real kicker to all this is that there is probably not one right answer, no single course of action that you can take. As with all these things, a combination of data and data sources probably works best to give you the fullest picture possible, and (as always) your ideal solution will look just a little different from everyone else’s!
Perhaps the key is to try different things, to speak to colleagues and partners and to have the time internally to make sure you know what your end goals are and what questions you need to ask to find out if you are getting there! Which sounds like the start of a Theory of Change model, but we’ll leave that for another blog!
Since writing this blog, we have published a 2019 update to Thriving Places Wales.
About the author