publiccode.yml v1 roadmap

[bfabio]

tl;dr:

• v1 milestone
• new 1.0-rc working branch
• help welcome

Time for v1

We've gone through a few 0.x releases. 🙂

The spec is used in Italy, Germany, Sweden, Switzerland and the EU OSS Catalogue to name a few and we have a good sense of what works and what doesn't. It's time to start thinking about v1.0.

This post just keeps track of things at a high level, the details are in the individual issues and PRs as usual. Issues and PRs labeled standard-breaking-change can only land in a new major version, so they are good candidates for v1.0.

Here's what needs to happen, not comprehensive and can be expanded later:

General stuff

Remove all deprecations (🔴 help welcome)

Anything deprecated in v0 (eg. usedBy, genericName, lowercase country codes, it.conforme.*, etc.) should be removed.

Multi repo projects

Maintainers are understandably unsure how to handle them. There are a few possible approaches, each with tradeoffs, so supporting this directly needs some design work.

Get rid of country specific section

They are slow to evolve, weird, and also a design smell.

Add a generic supports field

A generic key to declare what standards, regulations, or frameworks the software supports or complies with. (proposal incoming)

Useful, for example, for CRA (discussion #242) which could be:

supports:
    - thing: urn:cra

and to get rid of country specific sections.

URNize where possible

Some fields currently rely on free text values that are hard to validate or compare across catalogues. Where it makes sense, we should move towards URN identifiers and provide some guidance on how to use them.

Core tools

On the core tools side:

Multi-version support in parsers and crawler (🔴 help welcome)
The core tools mostly handle multiple schema versions already. This is just about verifying everything works and making sure v1.0 will be supported cleanly.

Track the effort here:

• publiccode-parser-go

• publiccode-parser-php

• publiccode-crawler

• software-catalog-api

• publiccode-parser-action

• publiccode-parser-gitlab-ci

• publiccode-validator-api

• publiccode-issueopener

Multi-version documentation & migration guides (🔴 help welcome)

The built documentation needs to show both v0 and v1. We should keep improving the docs structure and ideally write a cross-version migration guide.

See #299.

Work towards v1.0 will happen on the v1-rc branch. That will not be a release version, it's a branch where we can iterate and commit breaking changes without affecting v0 while not releasing v1 straight away.

Once we're happy with its state, we'll release it as v1.0.

[ruphy]

amazing!

[olibrian]

nice!

[tomootes]

great news @bfabio!

What do you think of the idea to work towards the generation of a JSON-Schema for every new version? If we have a coherent JSON-schema we could use this as the single source of truth and use this as a starting point for parsing? This could eliminate a lot of work imho. In my understanding projects like publiccode-parser-go are obsolete if we just use the JSON-Schema for parsing.

Like i issued here: #303. This does require a proper versioning of the JSON-schema.

[bfabio]

[...] use the JSON-Schema [as the single source of truth] for parsing [and validation].

This one pops up every now and then 😄

People seem drawn to it like moths to a flame, it's got some kind of pied piper charm. So let me drop a practical note here we can use as a reference for next time.

I already covered the schema vs parser split in #275 and in #277.

Short version: the JSON Schema is great for editor (vscode, etc.) autocomplete and a first "does this file look roughly right" pass.
It just isn't the validator and it can't really be one.

People have been drawn to this siren since day one: #42 is from February 2019, seven years ago!, and the author very quickly backed out, they can probably tell you what a pain it was.

Look at the diff and watch the moment it starts drowning in if/else branches: notice how unreadable it gets, and keep in mind that was a schema for a publiccode.yml with way fewer fields than today.

The reason is that JSON Schema is just not expressive enough.

Let's start with something that looks deceptively simple: releaseDate. The spec says it's date in YYYY-MM-DD form (aka RFC 3339 full-date).

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "releaseDate": {
      "type": "string",
      "format": "date"
    }
  },
  "required": ["releaseDate"]
}

Looks fine. Except format validation enforcement is optional in the JSON Schema spec, so a lot of libraries don't implement it. Try it yourself on https://jsonschema.dev/ with:

{ "releaseDate": "2026-00-29" }

Passes clean.

Ok, let's drop format and use a regex:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "releaseDate": {
      "type": "string",
      "pattern": "^[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01])$"
    }
  },
  "required": ["releaseDate"]
}

Now try:

{ "releaseDate": "2026-02-29" }

Passes. 2026 isn't a leap year. And this is a date, the simplest thing right after a string (well actually... but let's not digress).

Then there's BCP47, an RFC ~6000 lines long that on top of its own grammar tells you to use the "Extended Filtering" algorithm (see RFC4647, another ~1000 lines). Not something you validate with a regex, and therefore not with JSON Schema. SPDX license expressions: same story, it's basically a micro DSL.

And we haven't even gotten to the cross-field stuff (localisation.availableLanguages vs description.*), the things that need to touch the filesystem (logo, screenshots, legal.authorsFile must actually exist in the repo), the network checks (url has to point to a real reachable repo, it.riuso.codiceIPA has to exist in some Italian registry), and the conditional requireds (maintenance.contractors only when maintenance.type: contract).

So yeah: schema for shape and IDE support, parser for actual validation and automatic generation/updating of the JSON Schema would be nice, just not the SSoT.

[libremente]

I can confirm @bfabio! It was a dead end back in the days and I think it... still is today 😄 It can definitely be useful for some use cases but we should really consider the tradeoff of writing/maintaining it on the long term.

[tomootes]

@bfabio thanks for enlightening me on this issue! I was totally living in the understanding that JSON Schema could be the SSoT but i didn't grasp its nature totally.