The key aim of the Chance UK mentoring programme is to help primary school children with behavioural difficulties reduce those difficulties so they can look forward to a brighter future.
We can only know if we’re achieving this aim by carefully monitoring and evaluating our impact. Since we were founded in 1995, our charity has used a tried-and-tested behaviour measurement tool called the Goodman Strengths and Difficulties Questionnaire (SDQ) for evaluation purposes. A child must have an SDQ score of at least 16 to be eligible for the mentoring programme, and the intention is the score should reduce over the course of the mentoring year. By performing a simple before-and-after analysis of SDQ scores, Chance UK staff are able to see whether the mentoring has made a positive difference. Staff also interview children, mentors, parents/carers and referrers at the end of the mentoring year to collect their opinions about the quality of the programme, and any ideas for improvement. This ensures our evaluation is based not only on statistics, but also on the views of individuals who have direct experience of our services.
In 2008, academics from Goldsmiths, University of London, carried out an in-depth piece of research into Chance UK mentoring; this developed and supported in-house findings. Academics looked at the SDQ scores of 100 children after their year of mentoring with Chance UK and found that:
SDQ scores are divided into five subscales. Four of them, the ‘difficulties’ subscales, measure conduct problems, hyperactivity/inattention, peer relationship problems and emotional symptoms. Importantly, Goldsmiths found that all four of these subscales decreased following the mentoring year. The fifth subscale measures pro-social behaviour – the opposite of anti-social behaviour. Scores on this scale increased, indicating that mentored children had become better able to think of others. A year long longitudinal study of five children still being mentored and a retrospective study of 40 cases also formed part of the Goldsmiths research. Both produced encouraging results.
In May 2012, the Big Lottery Fund named Chance UK as one of 25 ‘outstanding’ projects that would receive five years’ support and £948,000 to develop its evaluation work as part of the ‘Realising Ambition’ programme. A year later, the Fund again selected Chance UK, this time to receive expert support from a respected research charity, the Social Research Unit (SRU), to carry out a randomised controlled trial. In an RCT, participants are divided into two groups. One group receives an intervention, in this case the Chance UK mentoring programme, while the other, the control group, does not. At the end of the trial, results for the two groups are compared to see if there is any statistically significant difference between them.
The Chance UK RCT, known as ECHO (Evidence for Children’s Outcomes), started in July 2014. The Social Research Unit report on the trial will appear in 2017 and provide the most detailed evaluation of the charity’s mentoring programme and outcomes so far. It may also highlight changes that would improve the programme. RCTs are considered to be the ‘gold standard’ for evaluation and Chance UK is looking forward to taking part.
Chance UK has prepared a detailed theory of change (also known as a logic model) for its core mentoring programme to support the Randomised Controlled Trial. This document is available to commissioners and funders on request. Please contact firstname.lastname@example.org if you would like to receive a copy.
A second theory of change is in preparation for Chance UK’s Parent Programme. It should be complete by the end of 2014 and will then also be available to commissioners and funders.
At Chance UK we want to know we’re able to achieve our aims. But we also need to know we can achieve those aims in a way that enables children to live better in society as adults and save society money. We therefore asked Pro Bono Economics (PBE), an organisation that matches volunteer economists with charities, to evaluate this aspect of our mentoring programme.
By analysing data from other studies, PBE found that conduct problems in young children lead directly to an extra £22,000 public service costs per child between the ages of 10 and 28. This pays for extra support at school or social care, for example. The economists also found the total cost to each individual resulting from conduct problems in childhood is £142,800 over their lifetime e.g. because of low educational attainment.
PBE’s report launched in November 2013 and concluded that the benefits of Chance UK mentoring would outweigh its costs if just one of every 42 mentored children avoided the negative outcomes and costs associated with conduct problems. Put another way, Chance UK’s work would be cost-effective if it reduced negative outcomes in only 2.5% more children than would have improved without mentoring. Since the Goldsmiths research (see above) showed 51% of children who completed Chance UK mentoring no longer had an abnormal level of behavioural difficulties, it seems clear that the Chance UK programme more than pays its way.