Merge branch 'main' into docs/slsa-provenance-spdx-sboms

dervoeti · web-flow · commit 300339945f51 · 2026-07-13T15:32:26.000+02:00
diff --git a/README.adoc b/README.adoc
@@ -70,5 +70,5 @@ The design & layout comes from our https://github.com/stackabletech/documentatio
 == More useful links
 
 * The https://github.com/stackabletech/documentation-ui[documentation-ui] repository.
-* The https://github.com/stackabletech/crddocs[crddocs] repository from which the https://crds.stackable.tech/[CRD reference] is generated.
+* The https://hub.stackable.tech/crds[CRD reference] on the Stackable Hub.
 * The Stackable https://docs.stackable.tech/home/stable/contributor/[contributor's guide] containing more info on how to contribute to the documentation.
diff --git a/antora-playbook.yml b/antora-playbook.yml
@@ -31,6 +31,7 @@ antora:
       mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs
       script_stem: mermaid-scripts
     - ./supplemental-ui/lib/stackable-operator-helpers.js
+    - ./supplemental-ui/lib/llms-txt.js
 content:
   sources:
     - url: .
diff --git a/local-antora-playbook.yml b/local-antora-playbook.yml
@@ -16,6 +16,7 @@ antora:
       mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs
       script_stem: mermaid-scripts
     - ./supplemental-ui/lib/stackable-operator-helpers.js
+    - ./supplemental-ui/lib/llms-txt.js
 content:
   sources:
     - url: ./
diff --git a/modules/concepts/pages/overrides.adoc b/modules/concepts/pages/overrides.adoc
@@ -86,10 +86,10 @@ The `jsonPatch` is more complicated than the `jsonMergePatch`, but offers more f
 ----
 configOverrides:
   config.json:
-    jsonPatches:
-      - '{"op": "test", "path": "/0/name", "value": "Andrew"}'
-      - '{"op": "add", "path": "/0/happy", "value": true}'
-      - '{"op": "remove", "path": "/0/birthDate"}'
+    jsonPatch:
+      - {"op": "test", "path": "/0/name", "value": "Andrew"}
+      - {"op": "add", "path": "/0/happy", "value": true}
+      - {"op": "remove", "path": "/0/birthDate"}
 ----
 
 ==== User provided JSON
@@ -117,7 +117,8 @@ configOverrides:
   config.json:
     userProvided:
       myString: test
-      myList: [test]
+      myList:
+        - test
       myBool: true
       my:
         nested.field.with.dots: 42
diff --git a/modules/contributor/images/project-overview-documentation.drawio.svg b/modules/contributor/images/project-overview-documentation.drawio.svg
diff --git a/modules/contributor/pages/docs/crd-documentation.adoc b/modules/contributor/pages/docs/crd-documentation.adoc
@@ -1,10 +1,10 @@
 # CRD documentation
-:crds-docs: https://crds.stackable.tech/
-:crddocs-repo: https://github.com/stackabletech/crddocs
+:crds-docs: https://hub.stackable.tech/crds
+:stackable-issues: https://github.com/stackabletech/issues
 
-The {crds-docs}[CRD documentation{external-link-icon}^] is generated from the CRD manifest files, which are in turn generated from the operator source code.
+The {crds-docs}[CRD reference{external-link-icon}^] on the Stackable Hub is generated from the CRD manifest files, which are in turn generated from the operator source code.
 All the documentation strings are doc strings in the Rust source code.
 If you want to contribute documentation for a particular field, this needs to be done in doc string of the property on the struct that makes up that part of the CRD.
 
-To change the UI, adjust the HTML template and CSS files in the {crddocs-repo}[crddocs repository{external-link-icon}^].
-Also consult the README in that repository to learn more about how the site is generated.
+The Hub ingests the CRDs of every operator automatically, so no manual steps are needed to publish the reference.
+If you spot a problem with how the reference is rendered, file an issue in the {stackable-issues}[issues repository{external-link-icon}^].
diff --git a/modules/contributor/pages/docs/overview.adoc b/modules/contributor/pages/docs/overview.adoc
@@ -5,11 +5,10 @@
 :antora-zulipchat: https://antora.zulipchat.com/
 :antora-distributed-components: https://docs.antora.org/antora/latest/distributed-component-version/
 :antora-content-branches: https://docs.antora.org/antora/latest/playbook/content-branches/
-:crddocs-site: https://crds.stackable.tech/
+:crd-reference: https://hub.stackable.tech/crds
 :diataxis: https://diataxis.fr/
 :netlify: https://www.netlify.com/
 :pagefind: https://pagefind.app/
-:stackable-crddocs-repo: https://github.com/stackabletech/crddocs
 :stackable-docs-readme: https://github.com/stackabletech/documentation/blob/main/README.adoc
 :stackable-docs-repo: https://github.com/stackabletech/documentation
 :stackable-docs-ui-repo: https://github.com/stackabletech/documentation-ui
@@ -46,7 +45,7 @@ The guide has to account for different use-cases (i.e. the user is using their o
 Since this kind of information is typically product specific, it is located in the usage guide section of individual operators.
 
 **Reference** information for the Stackable platform entails all the settings and Options in our YAMLs, which we generate.
-The reference is found at {crddocs-site}[{crddocs-site}{external-link-icon}^] and generated from the {stackable-crddocs-repo}[crddocs repository{external-link-icon}^].
+The reference is found at {crd-reference}[{crd-reference}{external-link-icon}^] on the Stackable Hub, which ingests the CRDs of every operator automatically.
 
 === Style guide
 
diff --git a/modules/contributor/pages/project-overview.adoc b/modules/contributor/pages/project-overview.adoc
@@ -56,8 +56,8 @@ The UI for the documentation is found in the https://github.com/stackabletech/do
 The documentation pulls in operator documentation files from the operator repositories.
 The documentation is found at https://docs.stackable.tech/[https://docs.stackable.tech/{external-link-icon}^].
 
-There is also https://crds.stackable.tech/[https://crds.stackable.tech/{external-link-icon}^] where you can find generated documentation for all the CustomResourceDefinitions on the platform.
-The code to generate this page is found in the https://github.com/stackabletech/crddocs[crddocs{external-link-icon}^] repository.
+There is also the https://hub.stackable.tech/crds[CRD reference on the Stackable Hub{external-link-icon}^] where you can find generated documentation for all the CustomResourceDefinitions on the platform.
+The Hub ingests the CRDs of every operator automatically.
 
 [[tooling-repositories]]
 === Tooling repositories
diff --git a/netlify.toml b/netlify.toml
@@ -4,7 +4,7 @@
 
 [[redirects]]
   from = "/home/stable/reference/*"
-  to = "https://crds.stackable.tech/"
+  to = "https://hub.stackable.tech/crds"
   status = 301
 
 [[redirects]]
diff --git a/only-dev-antora-playbook.yml b/only-dev-antora-playbook.yml
@@ -16,6 +16,7 @@ antora:
       mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs
       script_stem: mermaid-scripts
     - ./supplemental-ui/lib/stackable-operator-helpers.js
+    - ./supplemental-ui/lib/llms-txt.js
 content:
   sources:
     - url: ./
diff --git a/supplemental-ui/lib/llms-txt.js b/supplemental-ui/lib/llms-txt.js
@@ -0,0 +1,70 @@
+// Generates /llms.txt following https://llmstxt.org/ so that AI agents can
+// discover the documentation without scraping HTML.
+//
+// For every component, only the version that 'stable' points to is indexed
+// (the latest release, or nightly when building without release branches).
+// Each entry carries the published page URL, the page description and a link
+// to the AsciiDoc source on GitHub, so agents can both read a page and find
+// the file to change.
+//
+// Useful links:
+// Extensions: https://docs.antora.org/antora/latest/extend/extensions/
+// Types of events: https://docs.antora.org/antora/latest/extend/generator-events-reference/
+module.exports.register = function () {
+  this.once('beforePublish', ({ playbook, contentCatalog, siteCatalog }) => {
+    const siteUrl = (playbook.site.url || '').replace(/\/$/, '')
+    const lines = [
+      '# Stackable Data Platform Documentation',
+      '',
+      '> Documentation for the Stackable Data Platform (SDP), a curated selection of',
+      '> open source data apps like Apache Kafka, Apache Druid, Apache Trino and',
+      '> Apache Spark, all working together seamlessly on Kubernetes.',
+      '',
+      'Machine-readable release, component and CRD data is available from the',
+      'Stackable Hub: https://hub.stackable.tech/llms.txt',
+      '',
+      'Every entry below links the published page and, in parentheses, its AsciiDoc',
+      'source on GitHub. To suggest a change to a page, edit that source file.',
+    ]
+
+    for (const component of contentCatalog.getComponents()) {
+      const version = component.latest
+      const pages = contentCatalog
+        .findBy({ component: component.name, version: version.version, family: 'page' })
+        .filter((page) => page.out)
+        .sort((a, b) => a.pub.url.localeCompare(b.pub.url))
+      const byModule = new Map()
+      for (const page of pages) {
+        const module = page.src.module
+        if (!byModule.has(module)) byModule.set(module, [])
+        byModule.get(module).push(page)
+      }
+      for (const [module, modulePages] of byModule) {
+        lines.push('', `## ${component.title}: ${module === 'ROOT' ? 'general' : module}`, '')
+        for (const page of modulePages) {
+          const title = (page.asciidoc && page.asciidoc.doctitle ? page.asciidoc.doctitle : page.src.stem)
+            .replace(/<[^>]+>/g, '')
+          const description = page.asciidoc && page.asciidoc.attributes['description']
+          const source = sourceUrl(page.src)
+          let entry = `- [${title}](${siteUrl}${page.pub.url})`
+          if (description) entry += `: ${description.replace(/\s+/g, ' ').trim()}`
+          if (source) entry += ` (source: ${source})`
+          lines.push(entry)
+        }
+      }
+    }
+
+    siteCatalog.addFile({
+      contents: Buffer.from(lines.join('\n') + '\n', 'utf8'),
+      out: { path: 'llms.txt' },
+    })
+  })
+}
+
+function sourceUrl (src) {
+  const origin = src.origin
+  if (!origin || !origin.url || !origin.url.startsWith('https://github.com/')) return undefined
+  const repo = origin.url.replace(/\.git$/, '')
+  const start = origin.startPath ? `${origin.startPath}/` : ''
+  return `${repo}/blob/${origin.refname}/${start}${src.path}`
+}
diff --git a/truly-local-playbook.yml b/truly-local-playbook.yml
@@ -16,6 +16,7 @@ antora:
       mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs
       script_stem: mermaid-scripts
     - ./supplemental-ui/lib/stackable-operator-helpers.js
+    - ./supplemental-ui/lib/llms-txt.js
 content:
   sources:
     - url: ./
diff --git a/ui b/ui
@@ -1 +1 @@
-Subproject commit d72dc30dfea69d4b15fc9d05c03554ca5cbbe03b
+Subproject commit ba3a4d5f7ca754d157667edc1104c3ef52156546
