Skip to content

Commit 41c315d

Browse files
zeevdrzeevdr
authored
feat(schemas): publish meta-schemas to schemas.opendecree.dev via Pages (#263)
Adds the GitHub Pages deploy workflow that serves schemas/v*/decree-{schema,config}.json at https://schemas.opendecree.dev/schema/v*/. Uses an explicit-allowlist artifact build — only the JSON files + index.html + robots.txt are exposed; the rest of the repo stays private. Includes hosting runbook and resolves the design-brief hosting open question. Closes #125.
1 parent 9b7e39b commit 41c315d

4 files changed

Lines changed: 327 additions & 1 deletion

File tree

‎.agents/context/schema-spec.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,11 @@ Meta-schema encodes this via `allOf` with 4 `if/then` branches keyed on `type`.
167167
}
168168
```
169169

170+
## Hosting
171+
172+
- **Resolved:** GitHub Pages on the `opendecree/decree` repo itself, with custom domain `schemas.opendecree.dev` via a Cloudflare CNAME (DNS-only, gray cloud). The `.github/workflows/deploy-pages.yml` workflow builds an explicit `_site/` artifact from `schemas/v*/...` and uploads only that to Pages — no other repo content is exposed. CORS comes for free (`access-control-allow-origin: *` on GitHub Pages static files); content-type is `application/json` (the issue's acceptance criteria treat this as an acceptable fallback for `application/schema+json`).
173+
- **One-time setup steps** are documented in [`docs/development/schemas-hosting-runbook.md`](../../docs/development/schemas-hosting-runbook.md). May extract to a dedicated `opendecree/schemas` repo later if decree's Pages slot is needed for something else (docs landing, API reference). The public URL stays stable across that move thanks to the CNAME.
174+
170175
## CI
171176

172177
- **Primary tool:** `check-jsonschema` (Python, wraps `jsonschema` library, excellent error messages, built-in YAML support, default Draft 2020-12)
@@ -213,7 +218,6 @@ Meta-schema encodes this via `allOf` with 4 `if/then` branches keyed on `type`.
213218

214219
## Open questions
215220

216-
- **Hosting target for `schemas.opendecree.dev`** — dedicated GitHub Pages repo? Cloudflare redirect to raw GitHub content? Needs DNS + CORS setup.
217221
- **Bundling tool** — hand-rolled Python script vs off-the-shelf (e.g. `json-dereference-cli`). Go with off-the-shelf if one exists and is maintained.
218222
- **Does the CLI emit `$schema`/`$id` on export?** — `decree schema export` should probably inject `$schema` by default, make `$id` opt-in.
219223
- **Post-v1.0.0 URL migration** — when the spec promotes to 1.0.0, keep `/v0.1.0/` live forever or redirect? Preserve forever matches OpenAPI's dated-URL practice.

‎.github/workflows/deploy-pages.yml‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# Deploy meta-schemas to GitHub Pages at schemas.opendecree.dev.
2+
#
3+
# Publishes JSON Schema 2020-12 documents under schemas/v*/ to
4+
# https://schemas.opendecree.dev/schema/v*/decree-{schema,config}.json so
5+
# editors (via schemastore.org), CI linters, and other tooling can fetch
6+
# the meta-schemas by URL. Closes #125.
7+
#
8+
# Allowlist publishing model:
9+
# The job builds _site/ from scratch, copies only the JSON files, runs
10+
# an audit step that fails on unrecognized files, and uploads exactly
11+
# _site/ as the Pages artifact. The rest of the repo is not exposed —
12+
# Pages serves only what's in the artifact.
13+
#
14+
# Triggers:
15+
# - push to main when schemas/** or this workflow itself changes
16+
# - workflow_dispatch (manual republish for debugging or post-DNS fixups)
17+
#
18+
# Bootstrap requirement:
19+
# Custom domain schemas.opendecree.dev must be configured in the repo's
20+
# Settings → Pages with Source: GitHub Actions. Cloudflare CNAME
21+
# schemas → opendecree.github.io must be in place. See
22+
# docs/development/schemas-hosting-runbook.md for one-time setup.
23+
24+
name: Deploy Pages
25+
26+
on:
27+
push:
28+
branches: [main]
29+
paths:
30+
- 'schemas/**'
31+
- 'scripts/generate-schema-index.py'
32+
- '.github/workflows/deploy-pages.yml'
33+
workflow_dispatch:
34+
35+
permissions:
36+
contents: read
37+
pages: write
38+
id-token: write
39+
40+
concurrency:
41+
group: pages
42+
cancel-in-progress: false
43+
44+
jobs:
45+
deploy:
46+
name: Deploy meta-schemas
47+
runs-on: ubuntu-latest
48+
timeout-minutes: 5
49+
environment:
50+
name: github-pages
51+
url: ${{ steps.deployment.outputs.page_url }}
52+
steps:
53+
- name: Checkout
54+
uses: actions/checkout@v6
55+
with:
56+
persist-credentials: false
57+
58+
- name: Set up Python
59+
uses: actions/setup-python@v5
60+
with:
61+
python-version: "3.x"
62+
63+
- name: Install jsonschema + PyYAML
64+
run: pip install --no-cache-dir 'jsonschema>=4.21' 'PyYAML>=6'
65+
66+
# Reuses the script that ci.yml already runs on PRs — same checks,
67+
# second pass on main before publish.
68+
- name: Validate meta-schemas
69+
run: python3 scripts/validate-meta-schemas.py
70+
71+
# Belt-and-braces: publish-time check that the URL inside the
72+
# artifact matches the URL it will be served at. Catches the
73+
# rare case where a rename slips through without updating $id.
74+
- name: Validate $id matches publish URL
75+
run: |
76+
set -euo pipefail
77+
fail=0
78+
for f in schemas/v*/decree-schema.json schemas/v*/decree-config.json; do
79+
ver=$(basename "$(dirname "$f")")
80+
base=$(basename "$f")
81+
expected="https://schemas.opendecree.dev/schema/${ver}/${base}"
82+
actual=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("$id",""))' "$f")
83+
if [ "$actual" != "$expected" ]; then
84+
echo "::error file=$f::\$id mismatch — expected $expected, got $actual"
85+
fail=1
86+
fi
87+
done
88+
exit $fail
89+
90+
- name: Build _site/
91+
run: |
92+
set -euo pipefail
93+
mkdir -p _site/schema
94+
for d in schemas/v*/; do
95+
ver=$(basename "$d")
96+
mkdir -p "_site/schema/${ver}"
97+
cp "${d}decree-schema.json" "_site/schema/${ver}/"
98+
cp "${d}decree-config.json" "_site/schema/${ver}/"
99+
done
100+
python3 scripts/generate-schema-index.py _site
101+
102+
# Allowlist guard: refuse to upload anything that isn't on the
103+
# explicit list of expected files. A workflow bug that copies
104+
# extra paths into _site/ (e.g. .git, .github, internal/) gets
105+
# caught here, not at the Pages edge.
106+
- name: Audit _site/ contents
107+
run: |
108+
set -euo pipefail
109+
unexpected=$(find _site -type f \
110+
! -path '_site/index.html' \
111+
! -path '_site/robots.txt' \
112+
! -regex '_site/schema/v[0-9]+\.[0-9]+\.[0-9]+/decree-\(schema\|config\)\.json' \
113+
-print)
114+
if [ -n "$unexpected" ]; then
115+
echo "::error::Unexpected files in _site/ — refusing to publish:"
116+
echo "$unexpected"
117+
exit 1
118+
fi
119+
echo "Files to be published:"
120+
find _site -type f | sort
121+
122+
- name: Setup Pages
123+
uses: actions/configure-pages@v5
124+
125+
- name: Upload artifact
126+
uses: actions/upload-pages-artifact@v3
127+
with:
128+
path: _site
129+
130+
- name: Deploy
131+
id: deployment
132+
uses: actions/deploy-pages@v4

‎docs/development/schemas-hosting-runbook.md‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Meta-schema hosting runbook
2+
3+
Operations guide for `https://schemas.opendecree.dev/` — the Pages site that serves the JSON Schema 2020-12 meta-schemas under `schemas/v*/`.
4+
5+
## Architecture
6+
7+
The decree repo is the publish target. `.github/workflows/deploy-pages.yml` builds a `_site/` directory containing only the JSON files (plus `index.html` and `robots.txt`) and uploads it as the Pages artifact. The rest of the repo is not exposed.
8+
9+
DNS routes `schemas.opendecree.dev` → `opendecree.github.io` via a Cloudflare CNAME (DNS-only). GitHub Pages provisions the Let's Encrypt cert.
10+
11+
## One-time setup
12+
13+
### 1. Cloudflare DNS
14+
15+
In the `opendecree.dev` zone:
16+
17+
| Field | Value |
18+
|-------|-------|
19+
| Type | `CNAME` |
20+
| Name | `schemas` |
21+
| Target | `opendecree.github.io` |
22+
| Proxy status | **DNS only** (gray cloud) |
23+
| TTL | Auto |
24+
25+
Why gray cloud and not orange (proxied): with the proxy on, Cloudflare terminates TLS at its edge and GitHub can't validate domain ownership for Let's Encrypt. Workarounds exist (Cloudflare's "Full" mode + advanced certs) but add complexity for no current benefit. We can switch to proxied later if response-header tuning or caching becomes useful.
26+
27+
`.dev` is on the Chromium HSTS preload list, so HTTPS is mandatory at first request — there's no period where the domain is reachable over plain HTTP.
28+
29+
### 2. GitHub Pages
30+
31+
In the decree repo: **Settings → Pages**.
32+
33+
1. Source: **GitHub Actions** (not "Deploy from a branch").
34+
2. Custom domain: `schemas.opendecree.dev`. Save.
35+
3. If GitHub asks for a TXT verification record, add it on the apex via the Cloudflare DNS panel and re-verify.
36+
4. Wait for the cert to provision. Typical 15–60 minutes, sometimes longer. Once complete, tick **Enforce HTTPS**.
37+
5. Verify before announcing the URL anywhere:
38+
39+
```sh
40+
curl -I https://schemas.opendecree.dev/
41+
```
42+
43+
Expect `HTTP/2 200` and a valid cert. If the cert is still pending, retry; do not link the URL externally yet.
44+
45+
## Publishing a new version
46+
47+
1. Author meta-schema YAML under `schemas/vX.Y.Z/decree-{schema,config}.yaml`. The `$id` field of each file must equal `https://schemas.opendecree.dev/schema/vX.Y.Z/<filename>` — the deploy workflow asserts this.
48+
2. Run `python3 scripts/yaml-to-json.py schemas/vX.Y.Z/decree-schema.yaml schemas/vX.Y.Z/decree-schema.json` (and the same for `decree-config`). Both YAML and JSON copies are committed.
49+
3. `make validate-meta-schemas` to confirm canonical files validate and known-invalid fixtures don't.
50+
4. Open a PR. CI runs `Meta-schemas check` on every PR; it fails loud on validation regressions.
51+
5. After merge to main, `Deploy Pages` triggers automatically (path-filtered on `schemas/**`). It republishes the entire `_site/` artifact — adds the new version, regenerates the index.
52+
6. Verify:
53+
54+
```sh
55+
curl -s https://schemas.opendecree.dev/schema/vX.Y.Z/decree-schema.json | jq -r '.["$id"]'
56+
```
57+
58+
The output must equal the requested URL.
59+
60+
## Immutability policy
61+
62+
Once a version is announced (linked from external systems, schemastore.org catalog, READMEs, etc.), **do not edit `schemas/vX.Y.Z/` in place**. Bugs require a new SemVer dir. Fix-ups are allowed up until the version is referenced externally; after that, third parties may have cached the file and may not refetch.
63+
64+
The workflow does not enforce immutability — it allows overwrites because pre-release fix-ups are common. Discipline is on the author.
65+
66+
## Manual republish
67+
68+
If a Pages run fails (transient action error, cert wasn't ready, etc.), trigger a republish without commit:
69+
70+
```sh
71+
gh workflow run deploy-pages.yml --repo opendecree/decree
72+
```
73+
74+
This dispatches the workflow on the current `main` HEAD. The same validation steps run; if any fail, no deploy.
75+
76+
## Troubleshooting
77+
78+
- **Cert pending past 60 minutes.** Re-check the CNAME (gray cloud, target `opendecree.github.io`). Toggle the custom domain off and back on in Settings → Pages to nudge re-provisioning.
79+
- **`curl -I https://schemas.opendecree.dev/` returns 5xx or connection refused.** Custom domain not yet active. Wait for cert provisioning. Don't announce the URL until this passes.
80+
- **Workflow fails with "Pages site not found".** Custom domain hasn't been saved in Settings yet, or the Pages source is still set to a branch. Set Source = GitHub Actions and re-dispatch.
81+
- **`$id` mismatch error from the deploy workflow.** A file under `schemas/v*/` has a `$id` that doesn't match its publish URL — usually a stale rename. Fix the YAML, regenerate the JSON, push.
82+
- **Audit step refuses unexpected files in `_site/`.** A workflow step copied something unintended. Inspect the run log for the file list; typical cause is a `cp -r` that pulled in too much.
83+
- **Schemastore.org caches a 404.** If schemastore fetched the URL during the cert-pending window, their crawler may cache the failure. Re-trigger their fetch by editing the catalog entry's `description` or filing a quick "please re-fetch" issue on their tracker.
84+
- **Browser HSTS pinning issues.** Once a browser has loaded `schemas.opendecree.dev` over HTTPS with a valid cert, it pins the HSTS for at least the cert lifetime. Don't worry — `.dev` is preloaded so this is the default state.
85+
86+
## Future migration
87+
88+
If decree's Pages slot is ever needed for something else (project docs landing, API reference site), extract this configuration to a dedicated `opendecree/schemas` repo:
89+
90+
1. Create the repo with the same `_site/` build pipeline.
91+
2. Move `schemas/v*/` to the new repo.
92+
3. Re-target the Cloudflare CNAME (target stays `opendecree.github.io`; the new repo is now the source).
93+
4. Disable Pages on the decree repo, enable on the new repo with the same custom domain.
94+
5. The public URL stays stable — the CNAME continues to resolve, and consumers see no change.
95+
96+
The current single-repo design is cheap to extract; this is documented for future reference, not a planned move.

‎scripts/generate-schema-index.py‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
#!/usr/bin/env python3
2+
"""generate-schema-index: build the Pages root index.html.
3+
4+
Walks <site>/schema/v*/ to discover published versions and writes a static
5+
HTML page at <site>/index.html that lists each version's published files
6+
with download links. Also writes <site>/robots.txt to keep crawlers off
7+
the JSON (we don't need SEO for raw schema docs).
8+
9+
Usage:
10+
generate-schema-index.py <site-dir>
11+
12+
Vanilla — stdlib only. Called from .github/workflows/deploy-pages.yml.
13+
"""
14+
15+
from __future__ import annotations
16+
17+
import html
18+
import re
19+
import sys
20+
from pathlib import Path
21+
22+
VERSION_RE = re.compile(r"^v(\d+)\.(\d+)\.(\d+)$")
23+
24+
PAGE_TEMPLATE = """<!DOCTYPE html>
25+
<html lang="en">
26+
<head>
27+
<meta charset="utf-8">
28+
<title>OpenDecree Meta-Schemas</title>
29+
<meta name="viewport" content="width=device-width, initial-scale=1">
30+
<style>
31+
body {{ font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; max-width: 720px; margin: 2rem auto; padding: 0 1rem; line-height: 1.5; color: #1f2937; }}
32+
h1 {{ font-size: 1.6rem; margin-bottom: 0.25rem; }}
33+
a {{ color: #2563eb; text-decoration: none; }}
34+
a:hover {{ text-decoration: underline; }}
35+
table {{ width: 100%; border-collapse: collapse; margin-top: 1rem; }}
36+
th, td {{ padding: 0.5rem 0.75rem; text-align: left; border-bottom: 1px solid #e5e7eb; }}
37+
th {{ font-weight: 600; background: #f9fafb; }}
38+
footer {{ margin-top: 2rem; font-size: 0.875rem; color: #6b7280; }}
39+
</style>
40+
</head>
41+
<body>
42+
<h1>OpenDecree Meta-Schemas</h1>
43+
<p>JSON Schema 2020-12 documents that describe the OpenDecree configuration format. Source of truth: <a href="https://github.com/opendecree/decree/tree/main/schemas">opendecree/decree</a>.</p>
44+
<table>
45+
<thead><tr><th>Version</th><th>File</th></tr></thead>
46+
<tbody>
47+
{rows}
48+
</tbody>
49+
</table>
50+
<footer>Apache 2.0 · OpenDecree is alpha — all artifacts subject to change.</footer>
51+
</body>
52+
</html>
53+
"""
54+
55+
56+
def main() -> int:
57+
if len(sys.argv) != 2:
58+
print(f"usage: {sys.argv[0]} <site-dir>", file=sys.stderr)
59+
return 2
60+
61+
site = Path(sys.argv[1])
62+
schema_dir = site / "schema"
63+
if not schema_dir.is_dir():
64+
print(f"error: {schema_dir} not found", file=sys.stderr)
65+
return 1
66+
67+
versions: list[tuple[tuple[int, int, int], str, list[str]]] = []
68+
for d in schema_dir.iterdir():
69+
if not d.is_dir():
70+
continue
71+
m = VERSION_RE.match(d.name)
72+
if not m:
73+
continue
74+
files = sorted(p.name for p in d.iterdir() if p.is_file())
75+
versions.append(((int(m.group(1)), int(m.group(2)), int(m.group(3))), d.name, files))
76+
versions.sort(reverse=True)
77+
78+
rows: list[str] = []
79+
for _, ver, files in versions:
80+
for fname in files:
81+
rows.append(
82+
f' <tr><td>{html.escape(ver)}</td>'
83+
f'<td><a href="schema/{html.escape(ver)}/{html.escape(fname)}">{html.escape(fname)}</a></td></tr>'
84+
)
85+
86+
body = "\n".join(rows) or ' <tr><td colspan="2">No versions published yet.</td></tr>'
87+
(site / "index.html").write_text(PAGE_TEMPLATE.format(rows=body))
88+
(site / "robots.txt").write_text("User-agent: *\nDisallow: /\n")
89+
print(f"Wrote {site / 'index.html'} ({len(versions)} versions)")
90+
return 0
91+
92+
93+
if __name__ == "__main__":
94+
raise SystemExit(main())

0 commit comments

Comments
 (0)