Sustainability of (Open) Data Portal Infrastruc-tures reports pt. 5

Since the summer of 2020, the European Data Portal (EDP) team has been summarising the six reports included in the “Sustainability of (Open) Data Portal Infrastructure” as featured highlights. This particular report will focus on the fifth report “Open Data Portal Assessment Using User-Oriented Metrics”. The report highlights ten different ways in which open data portals can be structured to ensure sustainability and added value, and offers indicators and guidelines for portal owners that help guarantee the quality, realise active use, and improve user experience.

As open data portal are central access points to a plethora of datasets, it is vital that the quality of these portals is meaningfully evaluated. In addition, as a limited number of countries assess their open data strategies, current open data initiatives have to be monitored as well. To that end, the report defines several Key Performance Indicators (KPIs) and benchmarks that allow measurements over time and comparisons with other portals.

Ten user-oriented sustainability principles

This report builds on the EDP’s 8th Analytical Report titled “The Future of Open Data Portals”. The report presented 10 ways in which open data portals can organised for sustainability and added value:

1. Organise for use of the datasets (rather than simply for publication);
2. Learn from the techniques utilised by recently emerged commercial data marketplaces; promoting use via the sharing of knowledge, co-opting methods common in the open-source software community;
3. Invest in discoverability best practices, borrowing from e-commerce;
4. Publish good quality metadata to enhance reuse;
5. Adopt standards to ensure interoperability;
6. Co-locate documentation so that users do not need to be domain experts in order to understand the data;
7. Link datasets to enhance value;
8. Be measurable, as a way to assess how well they are meeting users’ needs;
9. Co-locate tools so that a wider range of users and re-users can be engaged with;
10. Be accessible by offering both options for big data, such as Application Programme Interfaces (APIs), and options for more manual processing, such as csv-files, thus ensuring a wide range of user needs are met.

In order to assess these 10 user-oriented sustainability principles, the analytical report investigates several metrics and methods from a variety of sources. This includes published academic papers, white papers, independent reports, and initiatives from the European Commission amongst other institutional entities.  For each of the ten principles, one or more metrics were selected. The metrics are for instance checklists for web quality, certificates as put forward by the Open Data Institute, tools for web accessibility, or scales developed by researchers. Subsequently, several open data portals were investigated as examples to assess their adherence to the principles. This selection includes EU government data portals, local portals such as Open Data Trentino, and specific open data initiatives that seemed of interest, such as the London Datastore.

Applying the metrics to these examples, the study finds that:

1. All portals have datasets accompanied by descriptive records and most of them allow to preview extracts of the datasets;
2. The principle of promoting use is achieved to the maximum extent by the majority of the portals, with the exception of the Belgian portal;
3. All portals perform comparable in terms of discoverability, i.e. the publisher of the data has an open portal and publishes an updated, searchable list of datasets;
4. The EU Open Data Portal and the Cyprus National Portal perform best in terms of applying linked metadata policies;
5. In terms of promoting standards, there is extensive variation across portals;
6. All portals provide some kind of supporting documentation to co-locate but either as a document separate from the data, or unable to directly access from within the dataset;
7. In terms of linked data, the majority of portals perform well meaning they use RDF standards;
8. There is quite some variation between portals in how measurable they are, though overall the performance is average;
9. Co-location of tools enabled finding different types of visualisation tools (e.g. maps, graphs, and tables) and collaboration tools (e.g. users posting their reuses, discussion groups and community resources). In this regard, Portugal and Luxembourg score relatively high, meaning that these portals offer visualisation and collaborations tools for user to work interactively and innovatively;
10. All portals use human and machine-readable, non-proprietary formats indicating high accessibility of the portals;

In conclusion, though there is variety amongst the various portals in performance, overall the report finds the assessed open data portals to be successful, of high quality, and generally sustainable. Several portals are already well developed with high levels of maturity for open data, while others are still in a process of development. These results support portal owners in assessing the current sustainability of their portal, and identifying what is needed to improve usability.

This article focused on the key findings of the Fifth Sustainability Report. For more information on user-oriented metrics and open data portal assessment, explore the full report on the EDP website.

Keep an eye out for our next featured highlight on 17 February 2021 that will focus on “A Distributed Version Control Approach to Creating Portals for Reuse“. Interested in learning more about the topic? Join the EDP’s webinar series on “The Future of Open Data Portals”!