Skip to content

volunteeringdata/open-data-infrastructure

Repository files navigation

Volunteering Open Data Infrastructure

This repository holds code for modeling and accessing open data for volunteering.

Data Model

  1. Volunteering Data Model
  2. Data Model Visualisation

API

  1. OpenAPI description document
  2. Swagger UI
  3. JSON-LD context document
  4. SPARQL UI

How To

Set up and understand the infrastructure

The infrastructure consists of a Container Registry and two app services: 1 for the API and 1 for the database.

Some infrastructure as code (IaC) in the form of ARM templates is here to help out.

See the infrastructure README for details.

Deploy the API and Data

Both Data and API are continuously deployed via GitHub actions.

Transform Data

Run automated data transformation for DoIt Data sample

dotnet run ./transform/DoIt/activities.json ./data/doit-data.ttl --project ./transform/DoIt/

Publish Data

You can simply build the data container (fuseki) using this Dockerfile and publish it to a container registry of your choice.

A WebHook can automatically deploys the latest data container version available to an Azure App Service for example.

The data container is a readonly triplestore with a public sparql endpoint containing the latest data (including this repos' .ttl files).

Login

source .env
az login
az acr login --name $INFRASTRUCTURE_CONTAINER_REGISTRY_NAME

Publishing the container

docker build --tag $INFRASTRUCTURE_DATA_CONTAINER .
docker push $INFRASTRUCTURE_DATA_CONTAINER

Run the Data Container Locally

source .env
docker run --rm -p 3030:3030 $INFRASTRUCTURE_DATA_CONTAINER

SPARQL endpoint accessible at http://localhost:3030/sparql.

Example Query (URI encode the query and use it as parameter to http://localhost:3030/sparql?query=):

  • Get all activities CONSTRUCT { ?s <https://ns.volunteeringdata.io/activityTitle> ?o } WHERE { ?s <https://ns.volunteeringdata.io/activityTitle> ?o }
  • Get all statements CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }
  • Get the vocabulary CONSTRUCT { ?s ?p ?o . } WHERE { ?s ?p ?o ; <http://www.w3.org/2000/01/rdf-schema#isDefinedBy> <https://ns.volunteeringdata.io/> . }

Run the API Locally

Configuring User Secrets

Manage user secrets (right click Query.csproj > Manage User Secrets) and set the SparqlEndpointUri.

The secrets.json file should look like this:

{
  "QueryService": {
    "SparqlEndpointUri": "https://sparql.volunteeringdata.io/sparql"
  }
}

Running the Project

dotnet run --project Query

Go to the swagger endpoint on localhost. Or go to the SPARQL UI on localhost.

Configuring the MCP Server

See for example Claude's Documentation), and add the following local config:

{
    "mcpServers": {
        "volunteering": {
            "args": [
                "mcp-remote",
                "https://api.volunteeringdata.io/mcp"
            ],
            "command": "npx"
        }
    }
}

Endpoints Examples

Organisations:

Activities:

Taxonomy:

License

This work is licensed under MIT.

SPDX-License-Identifier: MIT



TODO

Enable CORS:

  • Test CORS is correctly configured for our purpose (MCP, API)

IO optimisation:

  • JSON input
  • CSV input
  • Optimised RDF formats (TriG/TriX/BinaryRDF)

Validator service:

  • Based on a SHACL document
  • Give it a URI for the SHACL doc
  • Give it a URI for the data

Data Portal endpoints:

  • DCAT
  • Croissant (TODO: Fix endpoint !important)
  • DCAT-AP
  • MLDCAT-AP

Infrastructure as Code:

  • Create service principal for CD
  • Export full arm template from Resource Group, or
  • Improve modular ARM templates

Documentation:

  • Serve LODE documentation
  • Serve WebVOWL

Modeling:

  • Improve model
  • Map ontologies (alignment)

Search:

  • Improve geo location search (GeoSPARQL Query not supported)
  • Improve Lucene indexing

Endpoints modeling:

  • Improve response formats of endpoints
  • Improve OpenAPI description

About

Volunteering Data Infrastructure

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors