Skip to content

Ship versioned native XSD schemas with governance CI#6528

Open
duncdrum wants to merge 3 commits into
eXist-db:developfrom
duncdrum:dp-schema-governance
Open

Ship versioned native XSD schemas with governance CI#6528
duncdrum wants to merge 3 commits into
eXist-db:developfrom
duncdrum:dp-schema-governance

Conversation

@duncdrum

@duncdrum duncdrum commented Jun 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Adds mvn validate checks that canonical config instances (5 files) stay valid against their XSDs, and that any semantic XSD edit is accompanied by a version bump on xs:schema/@version (enforced by .github/workflows/ci-schema-checks.yml)
  • Bumps xs:schema/@version on all 9 native XSDs for post-2023 semantic changes
  • Adds schemaVersion attribute to each canonical config instance, mirroring the XSD version — lets tooling detect schema/instance drift at a glance
  • Ships schema/ in every distribution layout (tarball, zip, Docker image, IzPack installer) as $EXIST_HOME/schema/, a stable location for external tools (eXide, IDE plugins) to resolve native config grammars

What changed

Area Files
Governance CI .github/workflows/ci-schema-checks.yml, .github/scripts/prepare-governance-context.sh, schema/governance.xsl, schema/schema-version-type.xsd
XSD version bumps schema/*.xsd (9 files)
schemaVersion attribute 5 canonical config instances in exist-distribution/src/main/config/
SchemaVersion.java Hand-written constants (replaced by codegen in #6530)
Distribution packaging exist-distribution, exist-docker, exist-installer pom changes
Documentation schema/README.md, AGENTS.md

Spec references

  • XML Schema 1.1 Part 1 §2.4 (xs:schema/@version)
  • Governance policy: schema/README.md

Test plan

  • mvn -N xml:transform@schema-governance passes on this branch (CI green)
  • mvn validate (canonical instance checks) passes
  • SchemaVersionSyncTest — constants match XSD @version attributes
  • SchemaVersionTest — constant format/parsing

Related issues

Closes #6189 — ships native XSD schemas in every distribution layout as $EXIST_HOME/schema/.
Closes #6190 — bumps xs:schema/@version on all 9 native XSDs; governance CI enforces version bumps on future semantic edits.
Partially addresses #6008 — fixes the abrev → abbrev typo in expath-pkg.xsd; the namespace attribute/element discrepancy is not addressed in this PR.

Notes

This is PR-A of a 5-PR split from #6505. Merge order: this PR → #6530 → #6531 → #6550 → #6551.

🤖 Generated with Claude Code

@duncdrum duncdrum requested a review from a team as a code owner June 26, 2026 12:04
This was referenced Jun 26, 2026
Closed
Open
Open
Open
dizzzz
dizzzz reviewed Jul 4, 2026
View reviewed changes

@dizzzz dizzzz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dizzzz dizzzz requested review from a team, joewiz, line-o and reinhapa July 4, 2026 18:11
This was referenced Jul 6, 2026
Open
Open
line-o
line-o reviewed Jul 7, 2026
View reviewed changes
Comment thread .github/scripts/prepare-governance-context.sh
line-o
line-o reviewed Jul 7, 2026
View reviewed changes
Comment thread .github/workflows/ci-schema-checks.yml
line-o
line-o reviewed Jul 7, 2026
View reviewed changes
Comment thread .github/workflows/ci-schema-checks.yml
line-o
line-o reviewed Jul 7, 2026
View reviewed changes
Comment thread exist-core/src/main/java/org/exist/util/MimeTable.java
line-o
line-o reviewed Jul 7, 2026
View reviewed changes
Comment thread schema/expath-pkg.xsd
</xs:annotation>
</xs:attribute>
<xs:attribute name="abrev" type="xs:NCName" use="required"/>
<xs:attribute name="abbrev" type="xs:NCName" use="required"/>

@line-o line-o Jul 7, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

abbreviated form of abbrev :)

line-o
line-o reviewed Jul 7, 2026
View reviewed changes
Comment thread pom.xml

phase=none is deliberate: this must NOT run on an ordinary `mvn install`/
`mvn test` (no context.xml will exist outside CI). It only runs when
invoked directly: `mvn xml:transform@schema-governance`.

@line-o line-o Jul 7, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we extend the documentation to explain what one has to do in order to create the context file in order to run this outside CI?

@duncdrum duncdrum Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a readme inside schema later on which goes into greater detail. To generate the context file, you run the shell script on nix. The idea is that you shouldn't have to ever do this. If you edit a schema file, you should know that you are editing a schema, this is only there to catch version drift, i.e. you edit a schema or instance without changing its version.

Validation will always run as part of normal builds now.

@line-o line-o Jul 7, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to be able to check that locally and could not find the infrormation.\

can I call .github/scripts/prepare-governance-context.sh on my local machine?

@line-o line-o Jul 7, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or is the new check enough?

@duncdrum duncdrum Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can call the script locally

line-o
line-o reviewed Jul 7, 2026
View reviewed changes

@line-o line-o left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good and comprehensive.
I would like two things documented answering these questions:

  • What needs to be done when a new schema is added?
  • How can I check the schema governance on my local machine?

@duncdrum

duncdrum commented Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

The canonical location for schema and instance pairs is the pom's validate-canonical-instances validationSets. In practice new schema and instance pairs require:
a) a schema inside schema/
b) an instance file very likely in exist-distribution/src/main/config/

both are picked up automatically. No further edits are necessary, CI will do its thing.

If the instance (or the schema) live outside these paths, you need to add the paths in these locations, to
c) also have the version drift protection do its thing.
d) wire schema-version-codegen execution in the consumer module's pom to generate Version constants instead of hand editing. (codegen is in the next PR)

I will add these steps to the readme inside schema.

@duncdrum duncdrum force-pushed the dp-schema-governance branch from 5984222 to ab032c1 Compare July 7, 2026 08:37
@line-o

line-o commented Jul 7, 2026

Copy link
Copy Markdown
Member

Can you also address the codacy warnings about assert in Junit tests?

@duncdrum duncdrum force-pushed the dp-schema-governance branch 2 times, most recently from 04e5431 to 8fd2efd Compare July 7, 2026 12:56
@duncdrum
[feature] Add schema governance CI; enforce XSD @Version bumps on sem…
69108b3 
…antic changes

Add a mvn validate execution (xml:transform@schema-governance) that catches
semantic edits to the seven native XSD schemas without a matching @Version
bump; bump all seven XSDs to reflect post-2023 semantic changes.

Closes eXist-db#6190
@duncdrum duncdrum force-pushed the dp-schema-governance branch from 8fd2efd to 290acfa Compare July 7, 2026 13:10
duncdrum added 2 commits July 7, 2026 15:32
@duncdrum
[feature] Add schemaVersion attribute to canonical config instances; …
05904fd 
…consolidate governance CI

Add schemaVersion="x.y.z" to all five canonical config instances so
administrators can see at a glance which XSD version a config file targets,
and so startup code can log a DEBUG or WARN when it doesn't match.
Consolidate the four separate governance shell scripts into a single
governance.xsl, making the version-bump check fully Maven-native. Add
SchemaVersionSyncTest to guard the generated constants against drift.
@duncdrum
[feature] Ship native XSD schemas in all distribution layouts
b867e64 
Include schema/ (conf.xsd, collection.xconf.xsd, descriptor.xsd,
controller-config.xsd, mime-types.xsd, users.xsd, server.xsd,
security-manager.xsd, expath-pkg.xsd and its extensions) in the
tarball, zip, Docker image, IzPack installer, and macOS .app bundle
as $EXIST_HOME/schema/. Also fix the "abrev" -> "abbrev" typo
in expath-pkg.xsd.

Closes eXist-db#6189
Partially addresses eXist-db#6008
@duncdrum duncdrum force-pushed the dp-schema-governance branch from 290acfa to b867e64 Compare July 7, 2026 13:33
@duncdrum

duncdrum commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

@line-o done also squashed commits and did another round of rebasing based on other review comments.

@duncdrum duncdrum mentioned this pull request Jul 8, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@line-o line-o line-o left review comments
@dizzzz dizzzz dizzzz left review comments
@joewiz joewiz Awaiting requested review from joewiz
@reinhapa reinhapa Awaiting requested review from reinhapa

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[BUG] exist native *.xsd schema files have outdated version attributes [feature] add exist-db specific schema files to distribution output

3 participants

@duncdrum @line-o @dizzzz