5 ways to get amazing Charity Commission data

Earlier this month the Charity Commission for England and Wales launched it’s “Statement of Strategic Intent“. One of the five strategic objectives for 2018 to 2023 is “Informing public choice” about charities – and they identify data as one of the ways to use this. As they say:

Today, we collect and display basic data about charities. The public and other stakeholders can check that a charity is registered, but our data isn’t easy to access, share or compare with other datasets. We will use our data and our expertise to make it easier for the public to find the information that matters to them, to assess charities and the difference they make. We are committed to making sure our data is truly open. [Link]

This is a fantastic commitment, and it’s great to see the Commission recognise the value of its data to the public and to charities.

I’ve recently written a report, commissioned by an advisor to NCVO’s Charity Tax Commission, looking at how charity regulators across the world collect and use their data [comments welcomed in this Google Doc; pdf also available]. Based on this report, and my experience of using the Charity Commission’s data for research, analysis and building tools, I have identified five ways the Commission can improve its data to meet its strategic objective.

1. Build on what’s there already

It’s important to recognise that the Commission is not starting from scratch. While the information is by no means perfect, the Commission already publishes a large dataset covering the organisations it regulates. This dataset goes above and beyond what other similar regulators publish in some areas, for example:

• It includes historical data. This includes financial details – you can track income and expenditure of charities across at least five years. But it  always amazes me that the Commission database goes back decades and includes details of charities like this one – registered in 1965 and removed in the 1970s.

• It goes beyond headquarters. While the data isn’t perfect, the Charity Commission does ask charities to record their “area of operation”. This means you can start to look at where charities actually operate, not just where they are registered. This extends internationally – you can see which countries charities are operating in.

So any plans the Commission has for its data should recognise what they’ve already achieved, and make sure they don’t lose that.

2. Use and create open standards

Data standards turbocharge open data, making it compatible with other datasets and more useful to users. They mean that people can use and build on data released without needing to pay for proprietary tools or get locked in to one vendor. Any data strategy that the Charity Commission designs should have open standards at its core.

A good starting point for that is file formats. At the moment the Charity Commission’s data downloads are in BCP file format. This is the standards equivalent of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard’. Switching to a standard open file format like CSV would instantly make this data more accessible and useable.

There are simple improvements that can be made like using standard date formats. The Charity Commission can build on the work that the Government Digital Service is doing on open standards in government.

But using existing standards probably won’t be enough. There is no standard for data about non-profit organisations, so creating this standard would be a big step forward. The Commission can learn from the experiences of standards like 360Giving or Open Contracting in this area.

3. Learn from the rest of the world

While researching charity regulation and transparency in other countries, it became apparent that the challenges facing the Charity Commission are not unique and that other regulators are looking at the same issues. Some examples for the Commission to take note of include:

• The US, where the “Form 990” gives a much more comprehensive view of a charity’s activities. It’s a pretty daunting bureaucratic form, but the rich information it gives has kicked off a cottage industry of data-driven organisations helping the sector improve – after a long campaign to open up the data.

• New Zealand, which has recently started mandating outcomes reporting for the largest organisations.

• Australia has experimented with a “Charity Passport” allowing different state and federal government departments to share data about charities, reducing the burden on those organisations. The recent collapse of plans for joint HMRC/Charity Commission processes show the challenges of implementing something similar here in the UK.

4. Stop, collaborate and listen

The Charity Commission isn’t alone in this, and it’s vital that its data strategy reflects the network that they are part of. The other UK charity regulators (OSCR and CCNI), Companies House, other regulators like the Fundraising Regulator or Care Quality Commission, are all part of the data picture for charities. This means that any data products designed by the Charity Commission should link with and complement the data of those other regulators.

Another important link for charity data is with HMRC – a large number of charities are registered with HMRC for tax purposes, but there is no public data on their activities or even a list of the names of organisations. A comprehensive data strategy would consider what data HMRC could release to add to the base of knowledge on charities.

But more importantly, the Charity Commission needs to listen to the views of charities themselves and their beneficiaries. I believe that some of the greatest gains in data about charities could come from producing annual accounts in more machine-friendly formats, and getting rid of PDFs. But these changes could make producing accounts more complex, and so any process that replaces PDFs needs to ensure that the needs of small charities producing them are met.

5. Build and improve, build and improve

I was excited when the Charity Commission first launched its “beta” version of the register. It wasn’t perfect, but it offered a great base to build a better way of showcasing the data the Commission holds about charities. But it’s disappointing that the tool has remained largely the same since it launched in 2015 – though perhaps not surprising given the constrained finances at the Charity Commission over that time.

Whatever data strategy the Commission produces, it won’t get it right first time. A key feature of any strategy is making sure there is space built in to reflect, listen and learn from mistakes and change to reflect those lessons.