What does the EU do with 292,9 Billion? Trace it here!
Data
Subsidystories.eu is a joint project between Open Knowledge International and Open Knowledge Foundation Germany. The technological architecture of the website is provided by OpenSpending, which is an OKI project.
Download
The concatenated dataset combines the datasets of all 28 EU member state countries. We have collected all available ERDF, ESF and CF datasets for the 2007-2013 and 2014-2020 period. However, major gaps still exist for the 2014-2020 period, because not all data has been released. The data was collected throughout October 2016 - February 2017. If you find discrepancies in the data, please check the source description (.yaml) files on Github. They provide additional information on the data, its the latest version etc. If you have found new datasets or updated datasets that are not in our files yet, do let us know.
The original files are described here For creating the joined datasets out of these files, we mapped every single dataset to common denominator variables that can be compared between the countries. More information on how we did this can be found under the Resources tab.
Resources
The following informs you about our methodology, how we collected the data and provides helpful documentation for understanding the data.
Working with the data
Please find a guideline on how to work with the data and open spending here.
Policy and Data Background
Here you can read more on the policy and data background.
Methodology
Detailed Documentation can be found here
- Finding the raw data: searching on the respective member states’ websites for the datasets. If not available online, we contacted the bureaus directly.
- Cleaning the data: the data we found often was not in an open data format - it came in different formats such as PDFs, webapps or Excel sheets and had to be scraped to a machine-readable format. Scraping was done either with scripts or using Tabula, while OpenRefine was used for cleaning.
- Standardising the data: for the different datasets to be merged, all single datasets needed to be mapped to one overall schema. This was based on the Open Fiscal Data Package.
- The datasets are hosted on OpenSpending, a fiscal transparency platform developed by Open Knowledge International. It provides an extensive and ever-growing fiscal database with visualization and analyzation tools.
- Further information including all the code, raw data, models and schemas can be found on Github
The Goal
The European Structural Investment Funds (ESIF), together with agricultural subsidies, provides the biggest source of funding within the EU.
However, how exactly these funds are being spent within the member states itself is not transparent. Since the ESIF datasets are hosted on the respective websites of the member countries, it hasn’t been possible to have a consolidated overview of all the available sources and the distribution of the money down to the transactional level. In that sense, this project is unique for it unifies the available datasets of fund distribution on one website.
This data collection aims to increase fiscal transparency in the European Union.
We invite data journalists, civil society organisations and interested citizens to analyse and track money flows within the EU.
If you have any questions on the methodology, please reach out anna.alberts@okfn.de
The Teams Behind this Project
This is a joint collaboration between the teams of Open Knowledge Germany and Open Knowledge International (OKI). The technological architecture is provided by OpenSpending, which is an OKI project. The project would not have been possible without the tireless efforts from our volunteers:
Luigi Reggi, Sem vd Brink, Rui Barros, Zoltan Sipos, Mathias Huter, Bertrand Delvaux, Maria Manolova, Olaya Argueso, Adrian Ortiz, Marta Rey, Delphine Reuter, Fernando Blat, Friedrich Lindenberg, Benjamin Knoedler, Veronika Voelinger, Christian Joergensen, Monisha Martins, Eliseo Antonini, Xavier Seingard, Alain Jenotte, Balazs Krich.