Attribution: In evaluation, attribution refers to the actors and interventions responsible for progress and change.
Co-production: This is a way of working that actively involves service users and citizens in the design, planning, delivery and improvement of public services. See the Co-production Network for Wales.
Counter-factual: The counter-factual asks “what would have still happened” or “what would have happened anyway” to groups, communities or individuals if the programme, policy or intervention being evaluated was not in place. It supports a Theory of Change approach to evaluation.
Empirical studies/evaluations: These studies use scientific methods and experiments to understand the impact of an intervention or programme, including Randomised Control Testing and Quasi-Experiments.
Mixed methods research: This typically involves using research methods that allow qualitative and quantitative data to be collected.
Primary data: Primary data is the research and evidence gathered specifically for the current evaluation or study. It can be qualitative or quantitative and include: surveys; focus groups; interviews; and experimental and creative methods - there are many options. These types of data support all evaluation types. Primary data supports all types of evaluations, especially when there is no pre-existing (secondary) data available to show change, impact and progress. In Impact evaluation, where long and complex change needs to be conveyed, primary data can be an effective way of showing emergent and unplanned change.
Qualitative data: Qualitative data describes rather than measures. Typically, it describes the quality of something, or individuals’ and groups’ experience or perception of something. Words, imagery, and creative means are typically used to convey findings.
Quantitative data: Quantitative data is any data that can be measured numerically. It can measure people (for example, the number of people in one area), things, characteristics (e.g. the number of people claiming Job Seekers Allowance, the temperature of something), and experience (the number of people who rated a service highly). Numbers are used to convey findings.
Quasi-Experiments: Quasi-Experiments are a proxy way of determining the counter-factual, which is “what would have happened anyway” without the existence of the programme or intervention under evaluation. Often, administrative and existing data is drawn on to see the difference a programme has made in a population or group where the programme has been trialled in comparison to a population or group without the intervention. Where it is not possible to carry out Randomised Control Trials, Quasi-Experiments are often used.
Randomised Control Testing: Randomised Control Trials are an empirical and experimental approach to testing the effectiveness or impact of an intervention or programme. People with similar characteristics are randomly assigned to an intervention or non-intervention group. Both groups are assessed to determine the impact of a programme or intervention on the intervention group in comparison to the non-intervention group. This is a widely accepted “gold standard” way of establishing the counter-factual, which conveys what would have happened without an intervention or programme in place.
Secondary data: Secondary data is any pre-existing information you can use to help you answer questions and can include: administrative data, programme monitoring, attendance figures and registers, official statistics, programme reports, etc. These types of data can be helpful in any type of evaluation, but especially economic, outcome and process evaluations which often need to draw on existing information to evidence money saved, improved outputs and outcomes.
Theory based studies/evaluations: These designs are used to qualitatively assess the ways in which a programme or intervention has influenced outcomes and impacts. It assesses the extent to which a programme has influenced results and looks at the reasons for this, usually referring to a Theory of Change.