Skip to content

Make docs image generation deterministic and enforce committed docs/images in CI#46

Open
Copilot wants to merge 26 commits intomainfrom
copilot/add-image-canonicalization-step
Open

Make docs image generation deterministic and enforce committed docs/images in CI#46
Copilot wants to merge 26 commits intomainfrom
copilot/add-image-canonicalization-step

Conversation

Copy link
Copy Markdown

Copilot AI commented May 8, 2026

just docs currently produces PNGs whose binary encoding can vary across machines, creating noisy diffs. This PR makes generated docs images deterministic at the byte level and adds CI enforcement so PRs cannot pass with stale or uncommitted docs/images output.

  • Deterministic PNG canonicalization in just docs

    • Added a canonicalization step to the docs-generation script, now extracted to generate_docs_images.py (invoked by just docs).
    • Re-encodes rendered PNGs into a stable form by:
      • decoding IDAT scanlines,
      • reconstructing pixels from PNG filters,
      • rewriting scanlines with filter type 0,
      • writing deterministic store-only DEFLATE blocks.
    • Preserves functional image content while normalizing encoding-level variability.
  • Render failure handling hardening

    • Kept existing known-failure sentinel handling (OpenSCAD occasionally emits a fixed-size broken PNG).
    • Reworked retry flow to be bounded and explicit (with max retry constant + clear failure error), while still canonicalizing successful outputs.
  • CI workflow to gate docs image drift

    • Added .github/workflows/docs-images.yml for PRs/workflow_dispatch.
    • Workflow regenerates required generated inputs and docs images via just clean-docs paths docs, then fails if docs/images differs from committed files.
    • Updated CI to install an OpenSCAD snapshot/nightly build (via scadm) instead of Ubuntu’s stable openscad package, so docs rendering uses syntax/features required by current sources.
    • Added explicit minimal token permissions (contents: read).
- name: Install dependencies
  run: |
    sudo apt-get update
    sudo apt-get install -y just libfuse2 libegl1 libxcb-cursor0 libglu1-mesa
    uv sync --frozen
    python -m pip install --user scadm==0.6.3
    tmpdir="$(mktemp -d)"
    printf '{"dependencies":[]}\n' > "${tmpdir}/scadm.json"
    (
      cd "${tmpdir}"
      ~/.local/bin/scadm install --openscad-only
    )
    echo "${tmpdir}/bin/openscad" >> "$GITHUB_PATH"
    "${tmpdir}/bin/openscad/openscad" --version

- name: Regenerate docs images
  run: |
    just clean-docs paths docs

- name: Verify docs images are committed
  run: |
    if [ -n "$(git status --porcelain -- docs/images)" ]; then
      echo "::error::docs/images is out of date. Run 'just docs' and commit the result."
      git --no-pager status --short -- docs/images
      git --no-pager diff -- docs/images
      exit 1
    fi

[!WARNING]

Copilot AI linked an issue May 8, 2026 that may be closed by this pull request
Copilot AI and others added 12 commits May 8, 2026 17:22
Agent-Logs-Url: https://github.com/yawkat/GridFlock/sessions/712e802b-110d-46e3-b582-73e61cab0a46

Co-authored-by: yawkat <2395297+yawkat@users.noreply.github.com>
Agent-Logs-Url: https://github.com/yawkat/GridFlock/sessions/712e802b-110d-46e3-b582-73e61cab0a46

Co-authored-by: yawkat <2395297+yawkat@users.noreply.github.com>
Agent-Logs-Url: https://github.com/yawkat/GridFlock/sessions/712e802b-110d-46e3-b582-73e61cab0a46

Co-authored-by: yawkat <2395297+yawkat@users.noreply.github.com>
Copilot AI changed the title [WIP] Add canonicalization step for documentation images Make docs image generation deterministic and enforce committed docs/images in CI May 8, 2026
Copilot AI requested a review from yawkat May 8, 2026 17:30
Copilot finished work on behalf of yawkat May 8, 2026 17:30
@yawkat
Copy link
Copy Markdown
Owner

yawkat commented May 9, 2026

@copilot extract the generation script from the justfile

Copy link
Copy Markdown
Author

Copilot AI commented May 9, 2026

@copilot extract the generation script from the justfile

Done in commit 848dd3a (with follow-up polish through a9eb3a0). The just docs generation logic is now extracted from justfile into generate_docs_images.py, and just docs now invokes that script.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • dl.google.com
    • Triggering command: /usr/lib/apt/methods/https /usr/lib/apt/methods/https (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of yawkat May 9, 2026 10:04
@yawkat
Copy link
Copy Markdown
Owner

yawkat commented May 9, 2026

@copilot PTAL at the failure in the new workflow

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 9, 2026

🚀 Preview deployed to https://f169ee29.gridflock-yawk-at.pages.dev

Copy link
Copy Markdown
Author

Copilot AI commented May 9, 2026

@copilot PTAL at the failure in the new workflow

Fixed in 69f1e16. The new workflow was failing because paths/puzzle.scad wasn’t generated before just docs (--hardwarnings then aborted OpenSCAD with “Can't open include file 'paths/puzzle.scad'”). I updated the workflow to run just clean-docs paths docs so the generated include exists before docs rendering.

Copilot finished work on behalf of yawkat May 9, 2026 10:08
@yawkat
Copy link
Copy Markdown
Owner

yawkat commented May 9, 2026

@copilot CI still fails. I think you have to use a snapshot build of openscad. Also, you will probably have to run the canonicalization as part of this PR to get the check to pass. If you can't do that, please inform me and I will run it locally.

Copy link
Copy Markdown
Author

Copilot AI commented May 9, 2026

@copilot CI still fails. I think you have to use a snapshot build of openscad. Also, you will probably have to run the canonicalization as part of this PR to get the check to pass. If you can't do tha...

Addressed in f355201: docs-images CI now installs a nightly OpenSCAD snapshot (via scadm) instead of Ubuntu’s 2021.01 package, then runs just clean-docs paths docs. I couldn’t regenerate/canonicalize docs/images locally in this sandbox because files.openscad.org is DNS-blocked here (snapshot download fails), so if the workflow next fails only on image drift, please run just clean-docs paths docs locally and push the updated docs/images.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.launchpad.net
    • Triggering command: /usr/bin/add-apt-repository add-apt-repository -y ppa:openscad/snapshots git conf�� get --global ve-keyring.gpg pull.rebase (dns block)
  • files.openscad.org
    • Triggering command: /usr/bin/python3 python3 - /var/lib/dpkg/info/python3-dateutil.list /var/lib/dpkg/info/xdg-user-dirs.list conf�� get r0_1.22.0-2.1build1_amd64.deb /home/REDACTED/.cargo/bin/git committer.name (dns block)
    • Triggering command: /home/REDACTED/.local/bin/scadm /home/REDACTED/.local/bin/scadm install --openscad-only git rev-�� HEAD libxcb-xinput0_1/var/lib/dpkg/info/libkrb5support0:amd64.list plit architectures (dns block)
  • openscad.org
    • Triggering command: /usr/bin/python3 python3 - /var/lib/dpkg/info/python3-dateutil.list /var/lib/dpkg/info/xdg-user-dirs.list conf�� get r0_1.22.0-2.1build1_amd64.deb /home/REDACTED/.cargo/bin/git committer.name (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of yawkat May 9, 2026 10:40
@yawkat yawkat marked this pull request as ready for review May 9, 2026 10:48
yawkat and others added 4 commits May 9, 2026 13:01
Use LIBGL_ALWAYS_SOFTWARE=1 + GALLIUM_DRIVER=softpipe in both the
justfile docs recipe and the CI workflow. This pins both local and CI
rendering to Mesa's pure-CPU softpipe rasterizer, avoiding pixel
differences caused by GPU vs. CPU rendering or different GL backends.
Instead of failing when docs/images differ, the CI workflow now
commits the regenerated images back to the branch. This is necessary
because OpenGL rasterization (even with the same OpenSCAD version)
produces pixel-level differences across different Mesa versions and
GPU drivers, making bit-identical local reproduction impractical.

CI becomes the canonical source for committed image content. Local
'just docs' runs still serve as regression detection: after pulling
CI-committed images, re-running locally will show a diff for any
model change that visibly affects the renders.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve documentation build

2 participants