Category Archives: Policy on Charity Impact

Using Evidence to inform policy and practice: notes from a session at #locality16

Leave a reply

Some note from the Panellists ay the Locality Conference in York in 2016.

Richard Harries.  I know how hard it is for community groups to get their voice heard.

Emma Stone:  Every time we fund research it feels like a hatchling – we have no idea which one will survive to make a difference.  We spent 40k on some research on the impact of the social fund, it came at a good time and led to a £250m boost to the fund – which was then scrapped.

At JR Housing – we invested quite a lot in scoping what we could do on dementia by working with people who deal with it or have it.  Academic research was less important than what was being done to address dementia and address loneliness.  Across the country lost of people are having the same conversation.  So the research leads to informal collaborations.

Last example is a long term partnership with Loughborough University around minimum income standards, because it’s only over time that you build up a cumulative effect where people start to trust the relationship and the approach.  How do you keep research grounded in the real world?  Communications is important –  we have had to try and build up new conversations in terms of poverty.  A tool to help people see where thye sat on the scale crashed the BBC website.

We can solve poverty in the UK

We can solve poverty in the UK – click to download report.

Various form of evidence can work – including impact measurement used by the teams at the time.  They are very good at influencing at a local level – but to influence at a national levele you may need more detailed academic research

Work out the value of what you have found out to influence where you want to see change in your world.  (This is just what we work with our clients to do –  to capture the difference they are making and use that to influence the change they want to see)

The key is to be clear on the value. The inference you draw from lighter touch measurement may be different from more in depth research. Be open about what you think it means.

Communicate the finding of your work.

  • Understand what you need to record.
  • Identify the key people to communicate with.
  • Amplify your reach.
  • Invite people to see you at work.
  • Celebrate what you achieve.
  • Share what hasn’t worked so well.

You can also use a wider range of media. Techniques such as story telling can make a huge difference (the stories you can collect through the Impact APP

Click to see the toolkit

Click to see the toolkit

Dan Corry

Evidence needs to be strong. It helps if it’s independent, it also helps if it fits the direction the government is going in.

Mentioned Feinstein about evidence of investment in young people by showing how crucial early years were and amplified surestart.

National Minimum wage and national Living wage policies born out of evidence.  Other work in the 1980’s showed the impact of long term unemployment on people.  Further evidence that short term prison sentences are a waste of time – which included work from Downing St Delivery unit.  This couldn’t change policy because politicians believed it was at odds with what the public want.  Evidence  of the impact of sugar on obesity is slow to take hold because of the power of the sugar lobby.

We are also doing some work on social prescribing to show DR’s how loneliness is effecting health and that introducing people to local community groups will improve help. NSPCC changed it’s tactics by using evidence to influence professionals.

One thing that is crucial is cost effectiveness. Something may work, but if it is to expensive it wont be adopted.

What do we want from evidence

  • Helps us understand problems
  • Causal drivers
  • What are the options to solve problems
  • Robust methods for evaluating
  • We want things that will ‘help ministers achieve their  objectives’

I’ve always wanted to know the ‘right’ answer to the problem we’re tackling. That may not fit with what government or the public wants, but it helps you respond as a civil servant in the most effective way.

What do police makers need:

  • Don’t forget policy takes place in a political environment, shaped as much by that as evidence.
  • Public opinion means we shouldn’t expect an automatic hearing for evidence – so present it well, clear of waffle and jargon and at the right time
  • Ministers are more interested in evidence than you might think.
  • Change may take time so persevere.
  • If you want to influence you need to work through a wide range of challenges, engage with civil servants, build relationships, get a secondment to a department to understand their world.
  • Civil servants keen for new ideas, especially on cheaper ways to do things.
  • We need policy and systems which encourage experimentation and trial and error.

Don’t just engage with civil servants, talk to the ministers or other people advising them.  Civil servants alone may be to late in the policy making process – often decided when politicians are in opposition. So talk to them.   At the moment our present prime minister wasn’t in opposition or able to develop that before becoming PM – chance now to influence what and how.

Also mentioned Ministry of Justice data models which makes it easier for small organisations.

Dan Corry – caught in a trap, between public servants needing simpler information coming back fro the sector and the third sector looking to find fresh ways to measure their impact.

How do charities capture and measure Impact? Notes on the House Lords Charities Committee enquiry.

Leave a reply

On Tuesday the House of Lords Charities Select Committee held the first of a set of of evidence collecting sessions for their enquiry into how charities measure impact. There were two sessions.  We watched and made some notes, mainly of the things that struck us as pertinent during the first, which revolved mainly around Impact.

Here they are.

Session one focussed on Impact, the panel was.

There were a few points made around measuring impact, that unsurprisingly we wholeheartedly agree with.

Paul Streets  thought it was unrealistic to expect their to be universal ways to measure impact.

To paraphrase

Every funder will need a different set of metrics which work on a different timescale.  As a funder we shouldn’t be asking for bespoke measurements… we need to find out what charities are measuring first.   If they’re good at it let them do it their way and if they’re not, maybe think twice about funding them.

This make huge sense to us.  Part of what we try and do is encourage charities or social enterprises to be proactive about how they measure and report impact.   We think that getting ahead of your commissioners and funders and being clear about what you do and how you measure that will ultimately helps funders.

What info should donors expect from Charities on the difference / Impact their funding has made?

Paul Streets:

We should start by looking at the organisation – have a conversation with them about how they will report outcomes, Tell us how you are listening to your beneficiaries – and how you’ll measure outcomes. If you have 50 beneficiaries and they are hard to reach that’s o.k just tell us how you’ll listen to them.

You have to measure the soft outcomes, as some projects the first outcome might just be getting someone to catch a bus.”

Dan Corry:

There is a a cost to administrate impact. There is a problem where some smaller charities do not wanted to push up admin costs or  they can’t afford it. What they need is head space to think it through. If you’re are going to collect soft data, that’s ok, but don’t just talk about the success stories.

Gen Maitland Hudson:

You shouldn’t just be collecting data, stories and feedback, you should be acting on it to make change, to make things better, there’s no point collecting it otherwise. Value the information.

Paul Streets;

As a funder we want honesty, if you have under performed tell us, we’ll still want to hear of the impact. As funders we need to allow charities to frame their own impact. Different charities have different Impact. some of it will be qualitative, some quantitative, some financial – but we need to understand this.

Gen Maitland Hudson;

We can help smaller charities measure Impact, we just need to find what tools can be provided – for collection and analysis – and it’s often the analysis that gets left out.”

The analysis side of story capture is what we recognise as the hard part. You can collect stories, but where do you store, them and how to you analyse them?  That’s one thing the impact app does. You collect the stories, it sorts, analyses and begins to to make sense of the content you’re collecting. Your stories becomes searchable, you can begin to spot trends, gaps in service and best practice. You can build case studies and evidence for funder’s old (and new). And we’re not the only ones to recongnise this strength .

The talk then moved on to comparison;

Can you compare the work of charities working in different circumstances?

Dan Corry; 

If you work in the same area – having the same metrics is useful, working together means you can work together, within reason, but you can’t compare different sectors. It’s apples and pears.

There’s been a danger that you try and work out which charity offers the greatest returns on investment and comparing across sectors doesn’t work

Gen Maitland Hudson;

What are you measuring for and what do you want your benchmark to do. Different projects need different benchmarks. It comes back to knowing what you want to use the data for.

We understand that comparing apples and pears isn’t realistic, but we also know that some funders may want to keep track of how they’re money affects people’s lives across a range  providers. The observer account makes it easy for a funder to collect and analyse live stories from a range of organisations they have funded.

The conversation continues;

Paul Streets;

Bench marking can work for funders, instead of for the charities How many people apply to you that don’t get a grant – if that number is high then are we wasting our time and theirs”

Dan Corry:

When you’ve been commissioned you need to remember that you are there to service your beneficiaries, not the funders.

The pay by results model has made charities think about measurement and then you have to think about value for money. PBR can pull charities away from their mission, but they have to know the contract is right. PBR can also make charities skim the easy cases for quick results.

Paul Streets;

Some PBR contracts can work, Meals on wheels, Domiciliary care etc, but contract commissioning can be highly destructive to other charities – as the shift from grants to contracts happen it’s highly unfair to the multiple disadvantaged.”

Cherry picking, or skimming the easy cases, is something some of our clients have recognised across their sectors.  We think that good use of impact measurement can help the best organisations show that they don’t do that.

It’s not simply where you’ve got a client to, but where they started from that important.  The best organisations may do dozens of things to help someone – and by routinely capturing the voice of the people you help we think that builds of the evidence of the myriad small things you do and why they matter

By consistently collecting, and by trusting your clients voice you can show the impact you’re making whether they are the easy to work with, or the hard to reach.