[feature] Add ModuleFactory and IndexFactory SPI for auto-discovery of bundled modules and indexes#6551
Open
duncdrum wants to merge 19 commits into
Open
[feature] Add ModuleFactory and IndexFactory SPI for auto-discovery of bundled modules and indexes#6551duncdrum wants to merge 19 commits into
duncdrum wants to merge 19 commits into
Conversation
reinhapa
requested changes
Jul 7, 2026
…antic changes Add a mvn validate execution (xml:transform@schema-governance) that catches semantic edits to the seven native XSD schemas without a matching @Version bump; bump all seven XSDs to reflect post-2023 semantic changes. Closes eXist-db#6190
…consolidate governance CI Add schemaVersion="x.y.z" to all five canonical config instances so administrators can see at a glance which XSD version a config file targets, and so startup code can log a DEBUG or WARN when it doesn't match. Consolidate the four separate governance shell scripts into a single governance.xsl, making the version-bump check fully Maven-native. Add SchemaVersionSyncTest to guard the generated constants against drift.
Include schema/ (conf.xsd, collection.xconf.xsd, descriptor.xsd, controller-config.xsd, mime-types.xsd, users.xsd, server.xsd, security-manager.xsd, expath-pkg.xsd and its extensions) in the tarball, zip, Docker image, IzPack installer, and macOS .app bundle as $EXIST_HOME/schema/. Also fix the "abrev" -> "abbrev" typo in expath-pkg.xsd. Closes eXist-db#6189 Partially addresses eXist-db#6008
…generate SchemaVersion constants Extend the Maven validate phase to check every schema/*.xsd against the W3C XSD 1.1 meta-schema (upgrading the bundled XMLSchema.xsd/dtd to the 2009 XSD 1.1 revision); caught and removed five xsi:type="dcterms:W3CDTF" appinfo annotations with no backing schema, bumping the affected schema @Version values. Wire a generate-sources execution that reads each governed XSD's xs:schema/@Version and emits SchemaVersion.java constants so they can never drift from the schemas. Closes eXist-db#5541
…line Wire the XML Catalog API into Saxon's XsltCompiler so xsl:import, xsl:include, and document() URIs resolve via the eXist-db entity catalog (Closes eXist-db#350). Generalize MutableCollection's at-store-time validation to route through a JAXP 1.1 SchemaFactory when the schema or its catalog- resolved meta-schema requires XSD 1.1; extract the detection logic into Xsd11SchemaDetection so validation:jaxp() and org.exist.validation.Validator share the same probe-compile path. Suppress the spurious xmlresolver WARNING for the non-existent "default" catalog entry. Closes eXist-db#5541
Set a Jetty stop timeout so shutdown does not block indefinitely on active connections; move ShutdownListenerImpl.shutdown() to a daemon thread so it does not deadlock the Jetty shutdown-hook thread that also holds the BrokerPool write lock.
…nical at build time Add a Maven exec plugin execution that transforms the canonical exist-distribution conf.xml through module-specific XSLT strip-sheets, writing per-module fixture conf.xml and controller-config.xml files into src/test/resources-filtered at process-test-resources time. Eliminates ~35 hand-maintained fixture copies that drift from canonical on every schema edit.
…o URN imports Register the repo-root schema/catalog.xml in the Maven xml-maven-plugin catalog chain so XSD-against-XSD validation resolves locally without network access. Migrate all 35 XSLT strip-sheets from bare http:// imports to URN-mapped imports via the catalog. Document the new-native-schema checklist and fix the profile location note in the README.
…X parser features in conf.xml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…yes", scheduler jobs as live enabled="no" entries, @enabled on <parameter> and <property> in conf.xsd; bump schema to 2.5.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…2.6.0 - Add memorySizeType (pattern [0-9]+[KkMmGgTt]?|-1); apply to cacheSize, collectionCache, minDiskSpace, recovery/@SiZe - pool/@min, @max: xs:integer → xs:positiveInteger - lock-table/@trace-stack-depth: xs:int → xs:nonNegativeInteger - indexer/create/@type: xs:anySimpleType → xs:string - transformer/@caching: xs:string → yes_no with default="yes" - scheduler/job/@period: xs:string → xs:positiveInteger - vector-models/model/@Dimension: xs:integer → xs:positiveInteger - Fix swapped docs on raise-error-on-failed-retrieval / enforce-index-use - Fix "group-comit" typo in recovery/@group-commit doc - Fix rpc-server in-memory-size default 4196 → 4096 - Fill in TODO docs: indexer/create attrs, flushAfter, n, xquery module attrs (class/uri/src), xupdate/@growth-factor Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ure triggers
ParametersExtractor.parseProperties() used case-sensitive .equals("no") while
every other enabled check in Configuration.java uses .equalsIgnoreCase(). An
enabled="NO" parameter would be silently included instead of skipped.
generate-conf-fixture.xsl stripped @enabled from kept module/index nodes but
not from surviving triggers, so BouncyCastle and URLStreamHandler emitted an
explicit enabled="yes" in generated fixtures — inconsistent with the module
treatment and noise in fixture diffs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nonical conf.xml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…abled to <vector-models> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…canonical conf.xml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…f.xml element
AbstractIndex.configure() only sets name from config.getAttribute("id") when
config != null. IndexFactory SPI registration passes config=null, leaving name
null. IndexController.getWorkerByIndexName() matches on name, so SPI-registered
indexes (range-index, ngram-index, sort-index, lucene-index) were never found,
causing NPE at every range:index-keys-for-field() call.
Fix: after configure(), if name is still null, call setName(id) with the
configuration id that was used to key the indexers map.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…es covered by existing imports Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 tasks
line-o
reviewed
Jul 8, 2026
line-o
left a comment
Member
There was a problem hiding this comment.
Resolving modules to load by default over SPI received pushback on the community call on 2026-07-06
Security concerns need to be addressed before we can consider pulling this in.
@dizzzz specific concern was: "I see autoloading modules that are not mentioned in the configuration as a risk."
My concern is: if bundled modules are left out of the configuration I have to know they are there in order to disable them again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces
ServiceLoader-based SPI for both XQuery modules and index modules so that all bundled implementations are auto-discovered at startup without explicitconf.xmlentries. The canonicalconf.xmltemplate is trimmed accordingly — new installations get a leaner config by default. Existing hand-maintained configs continue to work unchanged: explicit entries always take precedence over SPI discovery.Builds on the
@enabledattribute from #6550 (itself depending on #6528, #6530, #6531).Closes #3062 — delivers both parts of the issue: the
@enabledattribute (in #6550) and the ServiceLoader/CDI-style autodiscovery requested here.What Changed
ModuleFactory SPI
org.exist.xquery.ModuleFactoryinterface:getNamespaceURI()+create(String, Map<String,List<?>>, XQueryContext)matching the existingModulefactory contractServiceLoader<ModuleFactory>wired intoConfiguration.configureModules(): SPI runs first, then theconf.xmlloop; an explicit<module>entry for the same namespace URI overwrites the SPI entry, andenabled="no"suppresses itModuleFactoryimplementations inMETA-INF/services/org.exist.xquery.ModuleFactoryConfiguration.class.getClassLoader()used explicitly so the loader survives embedding scenariosIndexFactory SPI
org.exist.indexing.IndexFactoryinterface:getId()+create(BrokerPool, Path, Element)ServiceLoader<IndexFactory>wired intoIndexManagerinitialisation: SPI entries whoseiddoes not appear as a live<module>entry in conf.xml are registered automatically;enabled="no"suppresses without requiring JAR removalnamedefaults to the configidwhen registered via SPI without a conf.xml elementVector model registry
Configuration;@enabledon<vector-models>children so individual models can be disabled; suppressed WARN log for unavailable models (logged at DEBUG instead)exist-distribution/src/main/config/conf.xml(canonical template)<builtin-modules>section trimmed — the 27 bundled modules no longer listed individually<modules>(index) section trimmed — the bundled index modules no longer listed individuallyCompatibility
<module>entries inconf.xmlalways take precedence over SPI discovery for the same namespace URI or indexid— zero upgrade action required for hand-maintained configsenabled="no"to its entryMETA-INF/services/files are JAR-internal; no classpath changes required by consumersTest Plan
mvn validate -pl exist-distribution -Ddependency-check.skip=true— canonicalconf.xmlstill validatesmvn test -pl exist-core -Ddependency-check.skip=true -Ddocker=false— full unit suite passesenabled="no"to a module's conf.xml entry; confirm it is not loaded even though SPI would discover it🤖 Generated with Claude Code