Open Data infrastructure Schleswig-Holstein

[jze]

In the German federal state of Schleswig-Holstein we use Frictionless extensively for the open data publication. There are a lot of datasets with Frictionless Tabular Data Resource in our data portal: https://opendata.schleswig-holstein.de/dataset?q=frictionless

During the last month we have developed a lot of software in this area. The Frictionless schema is in the heart of the workflow. This is a diagram showing an overview:

Similar to the French https://schema.data.gouv.fr we have developed a schema repository https://schema.odi.schleswig-holstein.de/schemas (You can switch the language to English in the upper right corner where it says "Deutsch".) It gives you a nice overview of the existing schemas and allows you to define your own schemas using a web UI or directly the JSON of the schema in your browser.

The next component is an upload helper. For a specified schema you can upload CSV files that will be validated against that schema. You don't even know if there already is a schema for the kind of data you upload. The system will look for table schemas that match your data structure and suggest them. You can accept that suggestion of define your own table schema.

The data is then transformed into a variety of formats and services:

• GeoJSON,
• Frictionless Tabular Data Resource,
• Apache Parquet,
• Web Feature Service (WFS),
• Web Map Service (WMS),
• RDF.

Especially for the RDF transformation we have added some additional properties to the Frictionless schema. I am going to write another discussion item for that. It would be great if they could make their way into the official Frictionless schema specification.

A logical last step would be to embed the table schema in the DCAT metadata. This would allow it to be transported across all levels. Then you could also perform a SPARQL query for datasets containing a specific field (with its semantic meaning) based on information in the DCAT metadata. However, this would require an RDF representation of the Frictionless schema. It seems the topic was started a long time ago but was never finished

The complete software is open source. You can find it in the code repository of the German public administration https://gitlab.opencode.de/sh/zit/opendata/open-data-infrastruktur. There is also a (German) documentation website https://docs.odi.schleswig-holstein.de. However, that documentation is still work in progress.