Skip to content

feat(ci): cross-reference tag review via post-build lint + workflow_run#39666

Closed
kim-em wants to merge 3 commits into
leanprover-community:masterfrom
kim-em:crossref-tooling-ci
Closed

feat(ci): cross-reference tag review via post-build lint + workflow_run#39666
kim-em wants to merge 3 commits into
leanprover-community:masterfrom
kim-em:crossref-tooling-ci

Conversation

@kim-em

@kim-em kim-em commented May 21, 2026

Copy link
Copy Markdown
Contributor

Cross-reference tag review for PRs adding @[stacks ...], @[kerodon ...], or @[wikidata ...] attributes — implemented as a post-build lint step plus a workflow_run-triggered companion that posts the bot comment.

Why this shape. The previous iteration of this PR ran an independent pull_request_target workflow that text-parsed PR source files. That was a workaround for not having a built Mathlib environment available in CI. With a post-build lint step we do have the elaborated environment — so we read Mathlib.CrossRef.tagExt directly via importModules (loadExts := true) and cut ~250 LOC of bespoke byte-level scanner.

The three pieces:

  • scripts/crossref.lean — the extract subcommand is gone. New check --diff <range> --tsv <path> subcommand walks tagExt, looks up each tagged declaration's source module via env.getModuleFor?, filters by git diff --name-only of the PR range, and writes a machine-readable TSV. No Markdown, no network, no snippet fetching — that's the privileged workflow's job. Note: withImportModules passes loadExts := false and would leave tagExt empty for imported modules; we use importModules (loadExts := true) instead.

  • .github/workflows/build_template.yml — three new steps in post_steps, after the existing import-graph upload:

    1. lake env lean --run scripts/crossref.lean check --diff "${BASE_SHA}..HEAD" --tsv crossref-added.tsv
    2. Write crossref-context.json carrying the PR number and head SHA (so the workflow_run job doesn't have to rely on the unreliable workflow_run.pull_requests[0].number).
    3. Upload both as a single crossref-context artifact.
  • .github/workflows/crossref_review.yml — rewritten. Trigger flips from pull_request_target to workflow_run on the build workflows. Downloads the artifact, resolves the PR number through a three-stage fallback (artifact JSON → workflow_run payload → gh pr list head SHA match), calls the mathlib-ci orchestrator (which now consumes the TSV and fetches snippets in Python directly with parallel ThreadPoolExecutor for Stacks/Kerodon), and posts via update_PR_comment.sh. Fails the check on any missing tag.

Privilege model. The lint step runs with the PR's permissions and produces only a machine-readable TSV. The workflow_run job runs in the base ref's permission scope (with pull-requests: write), never executes anything from the artifact, fetches snippets server-side from three fixed upstream APIs, and builds the Markdown comment in trusted code.

🤖 Prepared with Claude Code

@github-actions

github-actions Bot commented May 21, 2026

Copy link
Copy Markdown

PR summary 292d3968f8

Import changes for modified files

No significant changes to the import graph

Import changes for all files
Files Import difference
Mathlib.Tactic 2
Mathlib.Tactic.CrossRef.Fetch (new file) 54

Declarations diff

+ CheckArgs
+ CrossRefHoverPanel
+ CrossRefHoverPanel.rpc
+ CrossRefHoverProps
+ Database.fetch
+ Database.gerbyBase?
+ Database.ofName?
+ Database.ofName?_name
+ Database.ofString?
+ Database.ofString?_name
+ Result
+ Result.emit
+ SnippetOutcome
+ WikidataParse
+ attachCrossRefWidget
+ cacheFile?
+ cacheLoad
+ cacheStore
+ checkUsage
+ fetchGerbyAll
+ fetchSnippet
+ gerbyBase?
+ gitChangedFiles
+ moduleToFilePath
+ parseCheckArgs
+ parseWikidataResponse
+ renderOutcome
+ snippetMain
+ snippetUsage
+ usage
+ wikidataBatchSize
+ wikidataResultOf
++ Database.name
++ afterFirst?
++ fetchGerby
++ fetchUrl
++ fetchWikidata
++ flattenWhitespace
++ jsonStrPath?
++ parseGerbyTitle
++ stripHtml
++ takeUntilChar
++ userAgent
+-+ Database

You can run this locally as follows
## from your `mathlib4` directory:
git clone https://github.com/leanprover-community/mathlib-ci.git ../mathlib-ci

## summary with just the declaration names:
../mathlib-ci/scripts/pr_summary/declarations_diff.sh <optional_commit>

## more verbose report:
../mathlib-ci/scripts/pr_summary/declarations_diff.sh long <optional_commit>

The doc-module for scripts/pr_summary/declarations_diff.sh in the mathlib-ci repository contains some details about this script.


No changes to strong technical debt.
No changes to weak technical debt.

⚠️ Workflow documentation reminder

This PR modifies files under .github/workflows/.
Please update docs/workflows.md if the workflow inventory, triggers, or behavior changed.

Modified workflow files:

  • .github/workflows/build_template.yml
  • .github/workflows/crossref_review.yml

⚠️ Scripts folder reminder

This PR adds files under scripts/.
Please consider whether each added script belongs in this repository or in leanprover-community/mathlib-ci.

A script belongs in mathlib-ci if it is a CI automation script that interacts with GitHub (e.g. managing labels, posting comments, triggering bots), runs from a trusted external checkout in CI, or requires access to secrets.

A script belongs in this repository (scripts/) if it is a developer or maintainer tool to be run locally, a code maintenance or analysis utility, a style linting tool, or a data file used by the library's own linters.

See the mathlib-ci README for more details.

Added scripts files:

  • scripts/crossref.lean

@mathlib-dependent-issues mathlib-dependent-issues Bot added the blocked-by-other-PR This PR depends on another PR (this label is automatically managed by a bot) label May 21, 2026
@mathlib-dependent-issues

mathlib-dependent-issues Bot commented May 21, 2026

Copy link
Copy Markdown

@kim-em

kim-em commented May 22, 2026

Copy link
Copy Markdown
Contributor Author

Here's what the bot's comment renders as, generated by feeding the orchestrator a synthetic file with one valid + one bogus tag from each database. (No quoting trick — this is the literal contents of comment.md from a real run, dropped into a GitHub comment so you can see how the table actually displays.)


Cross-reference tags added by this PR

Wikidata

Tag Declaration Title Snippet Author comment
Q11518 theorem pythagoras : True Pythagorean theorem relation in Euclidean geometry among the three sides of a right triangle named after Pythagoras
QFAKE99999 theorem bad : True NOT FOUND intentionally bogus

Stacks

Tag Declaration Title Snippet Author comment
09FS theorem cohen_first (n : ℕ) [NeZero n] : True Example 9.5.2 Example 9.5.2. The characteristic of $\mathbf{F}_ p$ is $p$, and that of $\mathbf{Q}$ is $0$. First part. We don't require p to be a prime in mathlib.
ZZZZ theorem worse : True NOT FOUND also bogus

Kerodon

Tag Declaration Title Snippet Author comment
0009 theorem first_notation : True Notation 1.1.0.1 Notation 1.1.0.1. For every nonnegative integer $n$, we let $[n]$ denote the linearly ordered set ${ 0 &lt; 1 &lt; 2 &lt; \cdots &lt; n-1 &lt; n } $.

⚠ One or more tags above could not be found upstream. Please verify the tag identifier or remove the attribute.

@kim-em kim-em force-pushed the crossref-tooling-ci branch 4 times, most recently from 69c93c6 to 1eb21ea Compare May 22, 2026 10:05
kim-em and others added 2 commits May 22, 2026 10:13
Add `scripts/crossref.lean`, a single standalone tool to support review of
PRs that add `@[stacks ...]`, `@[kerodon ...]`, or `@[wikidata ...]`
attributes. It exposes two subcommands:

* `lake env lean --run scripts/crossref.lean snippet <db> <tag>...` fetches
  a one-line label/description for each tag from the upstream database.
  Wikidata uses the `wbgetentities` API (with recovery when one bad QID
  poisons a batch); Stacks and Kerodon use the documented Gerby
  `/data/tag/<TAG>/content/statement` endpoint with a tolerant HTML→text
  strip that survives `<` inside math. Exit codes distinguish all-resolved
  (0) from missing (2) from transient network failure (3). When
  `CROSSREF_CACHE_DIR` is set, responses are memoised per `(database, tag)`
  so repeat lookups are instant.

* `lake env lean --run scripts/crossref.lean extract --file <path>...` (or
  `--diff <range>`) walks Lean source and emits TSV of every
  cross-reference attribute it finds, paired with the declaration it
  decorates. A byte-level scanner correctly skips string literals and
  line/block comments, handles multi-attribute blocks
  (`@[simp, stacks 01AB]`), multi-line attribute blocks, doc comments
  between attribute and declaration, and modifier keywords (`private`,
  `noncomputable`, …). Signatures are collapsed to one line for downstream
  consumption.

The two subcommands share a single `Database` enum, with a roundtrip
theorem `Database.ofString?_name` proving `name ∘ ofString?` is a section
— so adding a fourth database fails to compile until both projections are
updated together.

The script imports only Lean core (plus `Std.Data.HashMap` for batching),
so it can be invoked directly with `lake env lean --run` without any
Mathlib build, and a future CI workflow can drive it on untrusted PR
files without elaborating PR code.

Companion follow-ups (separate PRs) will add an LSP widget that surfaces
these snippets in the editor and a GitHub Actions workflow that posts a
once-per-PR comment with tag / signature / snippet tables and fails CI on
unresolved tags.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a Mathlib declaration carries `@[stacks ...]`, `@[kerodon ...]`, or
`@[wikidata ...]`, the info view now shows the upstream label / description
when the cursor sits on the attribute. The fetch is on-demand via an RPC
call from the widget to the Lean server, which shells out to `curl`; we
deliberately don't fetch at attribute-elaboration time so offline builds
stay clean and the only network access is when an editing session opens
the panel.

Two new modules and a small refactor to the existing one:

* `Mathlib/Tactic/CrossRef/Fetch.lean` (new) holds the `Database` enum and
  the snippet-fetch logic. The companion `scripts/crossref-snippet.lean`
  in the previous PR reimplements the same logic standalone for CI usage;
  the two are kept in sync deliberately so the CI script doesn't need a
  Mathlib build.
* `Mathlib/Tactic/Widget/CrossRefHover.lean` (new) defines the widget
  itself: an `RpcEncodable` props record, a `RequestM` RPC method that
  calls `fetchSnippet`, and a `mk_rpc_widget%` Component that renders
  the result as Html.
* `Mathlib/Tactic/CrossRefAttribute.lean` (modified) imports the two new
  modules, removes the now-duplicated `Database` / `databaseURL` /
  `databaseLabel` definitions, and calls `Widget.savePanelWidgetInfo` in
  each attribute's `add` handler. Tag storage, docstring rewriting, and
  the trace commands are untouched.

If `curl` is missing or the upstream site is unreachable, the widget
renders an inline error instead of blocking the LSP. The existing
`MathlibTest/CrossRefAttribute.lean` suite continues to pass — none of
the behaviour visible to it changes.

Stacked on leanprover-community#39662
(scripts only). A third follow-up will add a GitHub Actions workflow
that posts a once-per-PR comment with tag / signature / snippet tables.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kim-em kim-em force-pushed the crossref-tooling-ci branch from 1eb21ea to 9d70518 Compare May 22, 2026 10:17
kim-em added a commit to kim-em/mathlib-ci that referenced this pull request May 22, 2026
…t step

The mathlib4 cross-reference review pipeline is moving from a text-based
extractor running independently in a `pull_request_target` workflow to an
elaboration-based lint step that runs in the build pipeline's `post_steps`
job after `lake exe cache get`. The lint step emits a machine-readable
TSV (db / tag / declName / file / comment) as an artifact, and a
`workflow_run`-triggered companion downloads it here.

This commit rewrites `scripts/crossref_review/crossref-pr-comment.py` so
it:

* Takes `--tsv` (the path to the artifact) instead of two checkout paths.
* Defensively parses the TSV (unknown databases dropped, missing fields
  padded). Nothing from PR-derived input is executed.
* Fetches snippets directly via Python's `urllib` — Wikidata via the
  batched `wbgetentities` API, Stacks/Kerodon via the documented Gerby
  `/data/tag/<TAG>/content/statement` endpoint. Gerby fetches are
  parallelised with `ThreadPoolExecutor(max_workers=8)` to keep the
  workflow under a minute on PRs that touch large files.
* Constructs the Markdown comment in trusted code (no longer relies on
  PR-derived Markdown), still using the `### Cross-reference tags added
  by this PR` sentinel so `update_PR_comment.sh` can dedupe.
* Exits 0 / 1 / 2 / 3 with the same grammar as before.

Stacked on leanprover-community/mathlib4#39666
upcoming force-push, which adds the post-build lint step and rewires
`crossref_review.yml` as `workflow_run`-triggered.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kim-em kim-em force-pushed the crossref-tooling-ci branch from 9d70518 to d49feaa Compare May 22, 2026 11:39
@kim-em kim-em changed the title feat(ci): per-PR comment summarising cross-reference tags feat(ci): cross-reference tag review via post-build lint + workflow_run May 22, 2026
Replace the text-based cross-reference extractor with an elaboration-
based lint step that runs in the build pipeline's `post_steps` job
and a `workflow_run`-triggered companion that posts the bot comment.

**Why.** The previous text-based extractor (a ~250 LOC byte-level
scanner in `scripts/crossref.lean`) was a workaround for not having a
built Mathlib environment in the pull_request_target workflow. Now
that we run as a post-build lint step, we have a fully elaborated
environment and can read `Mathlib.CrossRef.tagExt` directly — cutting
~250 LOC and eliminating the fidelity gap with the Lean parser.

**Pieces:**

* `scripts/crossref.lean` — the `extract` subcommand is gone. New
  `check --diff <range> --tsv <path>` subcommand uses
  `importModules (loadExts := true)` (NOT `withImportModules`, which
  passes `loadExts := false` and would leave `tagExt` empty), walks
  `tagExt`, looks up each tagged declaration's source module via
  `env.getModuleFor?`, and filters by `git diff --name-only` of the
  PR range. Output is machine-readable TSV
  (`db\ttag\tdeclName\tfile\tcomment`). No Markdown, no network,
  no snippets — those are the privileged workflow's job.

* `.github/workflows/build_template.yml` — adds three new steps to
  `post_steps`, after the import-graph upload:
  - Run `lake env lean --run scripts/crossref.lean check ...`
    against the diff between the PR base and HEAD.
  - Write a `crossref-context.json` with PR number + head SHA so
    the companion workflow doesn't have to rely on the unreliable
    `workflow_run.pull_requests[0].number`.
  - Upload both as a single `crossref-context` artifact.

* `.github/workflows/crossref_review.yml` — rewritten. Old trigger
  `pull_request_target` → new trigger `workflow_run`. Downloads the
  artifact, resolves the PR number through a three-stage fallback
  chain (artifact JSON → workflow_run payload → `gh pr list` head
  SHA match), calls the mathlib-ci orchestrator (which now consumes
  the TSV and fetches snippets in Python), posts/updates/deletes
  via `update_PR_comment.sh`, and fails the check on `missing` tags.

**Privilege model.** The lint step runs with the PR's permissions and
produces only machine-readable TSV. The `workflow_run` job runs in the
base ref's permission scope (with `pull-requests: write`) and never
executes anything from the artifact — it parses the TSV defensively,
fetches snippets server-side from three fixed upstream APIs, and
builds the Markdown comment in trusted code.

Companion mathlib-ci PR:
leanprover-community/mathlib-ci#39

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kim-em

kim-em commented May 27, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favour of #39877, which uses the existing privilege-escalation-bridge pattern (see olean_report.yaml + olean_report_wf_run.yaml) and delegates snippet-fetching / comment-rendering to https://github.com/leanprover-community/external-tags.

This drops the ~250 LOC of bespoke artifact / PR-resolution machinery and the entire extract / check subcommand from scripts/crossref.lean.

@kim-em kim-em closed this May 27, 2026
kim-em added a commit to kim-em/mathlib4 that referenced this pull request May 27, 2026
…un shim

Stack on top of the dump_crossref_tags.lean PR. Adds two pieces:

* .github/workflows/build_template.yml: three new steps in `post_steps`,
  after the import-graph upload. Run scripts/dump_crossref_tags.lean to
  produce crossref-tags.tsv. For pull_request_target builds, build a
  bridge payload (pr_number, head_sha) and emit it via
  leanprover-community/privilege-escalation-bridge/emit using artifact
  name `crossref-tags-bridge`. Non-PR builds skip the emit.

* .github/workflows/crossref_review.yml: workflow_run listener on
  the two `continuous integration` workflows. Consumes the bridge
  artifact via privilege-escalation-bridge/consume, checks out
  leanprover-community/mathlib-ci, and invokes its
  scripts/crossref_review/post-comment.sh orchestrator. The orchestrator
  clones leanprover-community/external-tags at a pinned SHA and uses it
  to fetch snippets and render the PR comment.

Replaces leanprover-community#39666. The original PR rolled its own artifact upload + PR
number resolution fallback chain (~158 LOC) and included the snippet
fetcher in scripts/crossref.lean (~760 LOC). This version delegates to
the existing privilege-escalation-bridge infrastructure (same pattern
as olean_report.yaml + olean_report_wf_run.yaml) and to external-tags,
cutting the mathlib4 surface to ~97 LOC.

Trust model: the privileged workflow_run job runs only code from
mathlib-ci@<pinned SHA> (which runs only code from external-tags at a
pinned SHA). Nothing from the build artifact is interpreted as code;
the TSV is parsed as untrusted data with user-controllable fields
escaped before interpolation into the comment.

Depends on leanprover-community#39876.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
kim-em added a commit to kim-em/mathlib4 that referenced this pull request May 27, 2026
…un shim

Stack on top of the dump_crossref_tags.lean PR. Adds two pieces:

* .github/workflows/build_template.yml: three new steps in `post_steps`,
  after the import-graph upload. Run scripts/dump_crossref_tags.lean to
  produce crossref-tags.tsv. For pull_request_target builds, build a
  bridge payload (pr_number, head_sha) and emit it via
  leanprover-community/privilege-escalation-bridge/emit using artifact
  name `crossref-tags-bridge`. Non-PR builds skip the emit.

* .github/workflows/crossref_review.yml: workflow_run listener on
  the two `continuous integration` workflows. Consumes the bridge
  artifact via privilege-escalation-bridge/consume, checks out
  leanprover-community/mathlib-ci, and invokes its
  scripts/crossref_review/post-comment.sh orchestrator. The orchestrator
  clones leanprover-community/external-tags at a pinned SHA and uses it
  to fetch snippets and render the PR comment.

Replaces leanprover-community#39666. The original PR rolled its own artifact upload + PR
number resolution fallback chain (~158 LOC) and included the snippet
fetcher in scripts/crossref.lean (~760 LOC). This version delegates to
the existing privilege-escalation-bridge infrastructure (same pattern
as olean_report.yaml + olean_report_wf_run.yaml) and to external-tags,
cutting the mathlib4 surface to ~97 LOC.

Trust model: the privileged workflow_run job runs only code from
mathlib-ci@<pinned SHA> (which runs only code from external-tags at a
pinned SHA). Nothing from the build artifact is interpreted as code;
the TSV is parsed as untrusted data with user-controllable fields
escaped before interpolation into the comment.

Depends on leanprover-community#39876.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
kim-em added a commit to kim-em/mathlib4 that referenced this pull request May 27, 2026
…un shim

Stack on top of the dump_crossref_tags.lean PR. Adds two pieces:

* .github/workflows/build_template.yml: three new steps in `post_steps`,
  after the import-graph upload. Run scripts/dump_crossref_tags.lean to
  produce crossref-tags.tsv. For pull_request_target builds, build a
  bridge payload (pr_number, head_sha) and emit it via
  leanprover-community/privilege-escalation-bridge/emit using artifact
  name `crossref-tags-bridge`. Non-PR builds skip the emit.

* .github/workflows/crossref_review.yml: workflow_run listener on
  the two `continuous integration` workflows. Consumes the bridge
  artifact via privilege-escalation-bridge/consume, checks out
  leanprover-community/mathlib-ci, and invokes its
  scripts/crossref_review/post-comment.sh orchestrator. The orchestrator
  clones leanprover-community/external-tags at a pinned SHA and uses it
  to fetch snippets and render the PR comment.

Replaces leanprover-community#39666. The original PR rolled its own artifact upload + PR
number resolution fallback chain (~158 LOC) and included the snippet
fetcher in scripts/crossref.lean (~760 LOC). This version delegates to
the existing privilege-escalation-bridge infrastructure (same pattern
as olean_report.yaml + olean_report_wf_run.yaml) and to external-tags,
cutting the mathlib4 surface to ~97 LOC.

Trust model: the privileged workflow_run job runs only code from
mathlib-ci@<pinned SHA> (which runs only code from external-tags at a
pinned SHA). Nothing from the build artifact is interpreted as code;
the TSV is parsed as untrusted data with user-controllable fields
escaped before interpolation into the comment.

Depends on leanprover-community#39876.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked-by-other-PR This PR depends on another PR (this label is automatically managed by a bot)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant