odb: add LEFDEF grammar railroad diagram documentation by sparsh-karna · Pull Request #9866 · The-OpenROAD-Project/OpenROAD

sparsh-karna · 2026-03-21T10:16:43Z

Summary

Closes #2488.

Adds auto-generated SVG railroad diagrams for all LEF and DEF grammar rules
in OpenROAD's ODB parser, alongside Markdown documentation pages that embed
them. Railroad diagrams visually show every valid syntax path through the
LEF/DEF grammars, making them significantly easier to read than raw Bison
grammar files.

This is a documentation-only change. No production source code is modified.

What's Included

src/odb/doc/generate_railroad_diagrams.py: Python script that
converts lef.y / def.y (Bison grammars) → W3C EBNF → SVG railroad
diagrams via two vendored Java tools.
src/odb/doc/tools/ebnf-convert.war: Vendored
Gunther Rademacher's ebnf-convert
tool (Bison → EBNF conversion). Vendored directly to avoid requiring
contributors to build or download it separately.
src/odb/doc/tools/rr.war: Vendored
Railroad Diagram Generator rr lib
(de.bottlecaps.rr:rr-lib:2.6 from Maven Central). Vendored for the
same reason.
src/odb/doc/images/lef/*.svg: 246 SVG railroad diagrams covering
every LEF grammar rule.
src/odb/doc/images/def/*.svg: 276 SVG railroad diagrams covering
every DEF grammar rule.
src/odb/doc/LEFGrammar.md: Markdown doc embedding all LEF diagrams,
grouped by category.
src/odb/doc/DEFGrammar.md: Markdown doc embedding all DEF diagrams,
grouped by category.
.github/workflows/github-actions-update-grammar-railroad-diagrams.yml:
CI workflow that auto-regenerates diagrams and opens a PR whenever
lef.y or def.y changes on master.
docs/toc.yml: Adds LEFGrammar and DEFGrammar entries to the
OpenROAD documentation navigation.
src/odb/doc/.gitignore: Ignores intermediate .ebnf and .zip
artifacts produced during generation.

How to Regenerate Diagrams

Requires Java 11+ on PATH.

# Both grammars
python3 src/odb/doc/generate_railroad_diagrams.py all

# LEF only
python3 src/odb/doc/generate_railroad_diagrams.py lef

# DEF only
python3 src/odb/doc/generate_railroad_diagrams.py def

Example Diagram:

gemini-code-assist

Code Review

This pull request introduces auto-generated SVG railroad diagrams for LEF and DEF grammar rules in OpenROAD's ODB parser, along with Markdown documentation pages embedding these diagrams. The changes are documentation-only, with no modifications to production source code. The pull request includes a Python script for diagram generation, vendored Java tools, SVG diagrams for LEF and DEF grammars, Markdown documentation, and a CI workflow for automatic regeneration. No specific style guides were provided, so the review focuses on general code quality and best practices.

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

Add SVG railroad diagrams generated directly from lef.y and def.y, a Python script to regenerate them, Markdown documentation that embeds the diagrams, and a GitHub Actions workflow that opens a PR whenever a grammar file changes. Files added: - src/odb/doc/generate_railroad_diagrams.py — converts .y → EBNF → individual SVG files using ebnf-convert and rr (downloaded from Maven Central on first use; not committed). - src/odb/doc/images/lef/ — 246 SVGs generated from lef.y - src/odb/doc/images/def/ — 276 SVGs generated from def.y - src/odb/doc/LEF_Grammar.md — documentation with embedded diagrams - src/odb/doc/DEF_Grammar.md — documentation with embedded diagrams - src/odb/doc/.gitignore — excludes tools/ and intermediate files - .github/workflows/github-actions-update-grammar-railroad-diagrams.yml — auto-regenerates and opens a PR on grammar changes Resolves The-OpenROAD-Project#2488 Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

Place ebnf-convert.war (v0.73) and rr.war (v2.6) in src/odb/doc/tools/ so the script works offline without a Maven Central fetch. Update ensure_tools() to fail with actionable instructions rather than silently downloading, and remove tools/ from .gitignore now that the files are committed. Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

LEF_Grammar.md now covers 243 rules across 18 sections: File structure, Units, Layers, Vias, Via Rules, Spacing, Sites, Macros, Macro Pins, Non-default Rules, Properties, IRDrop, Noise & Correction Tables, Timing, Arrays (Floorplan), Antenna Rules, Dielectric & Minfeature, Primitives. DEF_Grammar.md now covers 274 rules across 28 sections: File structure, Design Attributes, Properties, Components, Nets, Net Routing, Net Subnets, Special Nets, Pins, Virtual Pins, Pin Properties, Blockages, Regions, Groups, Via Definitions, Non-default Rules, Fills, Slots, Styles, IO Timing, Scan Chains, Timing Disables, Partitions, Constraints, Assertions, Floorplan Constraints, Extension, Primitives. Each section has a linked TOC at the top of the page. Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

Inject a CSS custom property and inline style into each generated SVG so the diagrams render with a solid warm-white background (#FFFCF0) instead of being transparent. This keeps them legible in both light and dark documentation themes. Also clean up generate_railroad_diagrams.py: remove dead Maven Central download code (WARs are now vendored), update the module docstring, and wire in the new fix_svg_background() helper that is called after each SVG is extracted from the rr output ZIP. Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

sparsh-karna · 2026-03-21T11:00:44Z

@maliberty both CI failures are unrelated to the C++ changes.

clang-tidy: the clang-tidy-review action uses the GitHub API to fetch changed files, which hard-caps at 300. this PR has ~500 new SVGs so it hits a 406 before it even looks at any C++ code. no actual tidy warnings.

scan-code: flagging src/odb/doc/tools/ebnf-convert.war as a binary with blocked content. the WAR itself is a standard open-source Java tool (GuntherRademacher/ebnf-convert)

the routing code itself is clean.

luarss

Very nice first cut. Just some feedback to make it more future-proof

- Use **/*.y glob in CI workflow trigger instead of hardcoded lef.y/def.y paths so any future grammar additions are covered automatically - Refactor argument parsing in generate_railroad_diagrams.py to use argparse instead of manual sys.argv handling Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

sparsh-karna · 2026-03-21T14:48:37Z

hey @luarss could you check the changes that i made

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

sparsh-karna · 2026-03-21T19:37:54Z

hey @luarss, i have updated the ci for this could you check and give feedback

luarss · 2026-03-22T03:18:15Z

+      - name: Regenerate railroad diagram SVGs
+        run: python3 src/odb/doc/generate_railroad_diagrams.py
+
+      - name: Open PR if diagrams changed


What happens in the case of successive updates? Is there a flag in this action that you can use for pruning old branches/rebasing?

peter-evans/create-pull-request@v7 uses a fixed branch name (auto/grammar-railroad-diagrams), so on successive runs it force-pushes to that same branch and updates the existing open PR instead of creating a new one. delete-branch: true is set so the branch is automatically pruned after merge. I also verified that update-branch is not a valid input in v7's action.yml, the fixed-branch behavior handles successive updates natively.

luarss · 2026-03-22T03:23:12Z

+        id: cache-tools
+        with:
+          path: src/odb/doc/tools
+          key: railroad-tools-ebnf-v0.73-rr-2.6


Is it possible that this cache might cache invalid tools? Can you validate the cache before using/storing it?

Added a Validate cached railroad tools step that runs after cache restore. It checks both WARs are non-empty and uses jar tf to verify internal contents, WEB-INF/lib/Saxon-HE-12.9.jar in ebnf-convert and de/bottlecaps/railroad/Railroad.class in rr.war. If either fails, the files are deleted and the build step is forced to run regardless of cache hit status.

luarss · 2026-03-22T03:25:25Z

+    """Inject a solid background colour into an SVG file."""
+    content = svg_path.read_text(encoding="utf-8")
+    style_block = "<style>\n" f"  :root {{ --diagram-bg: {DIAGRAM_BG}; }}\n" "</style>"
+    content = re.sub(r"(<svg[^>]*>)", r"\1" + style_block, content, count=1)


This seems fragile. Is there a python library we can use, maybe xmltree?

Replaced the regex approach with xml.etree.ElementTree. The implementation pre-registers all namespace prefixes via ET.iterparse (so xlink is preserved), inserts <style> as the first child of , and writes back with xml_declaration=True. This is in the next commit, being pushed shortly.

luarss · 2026-03-22T03:26:14Z

+        classpath = sep.join([str(RR_WAR)] + dep_jars)
+        subprocess.run(
+            [
+                "java",


add java shutil checks in ensure_tools()

Added in the next commit, ensure_tools() now calls shutil.which("java") and additionally checks the version is ≥ 11, reporting a clear error message if Java is missing or too old.

maliberty · 2026-03-22T15:34:36Z

+      - name: Open PR if diagrams changed
+        uses: peter-evans/create-pull-request@v7


Why not add a commit to the current PR rather than make a new one?

The workflow triggers on pushes to master after a grammar PR is already merged, so there is no open PR to commit back to at that point. Opening a dedicated PR for the SVG updates keeps diagram regeneration separate from grammar changes and gives maintainers a focused diff to review.

maliberty · 2026-03-22T15:36:54Z

+      - name: Regenerate railroad diagram SVGs
+        run: python3 src/odb/doc/generate_railroad_diagrams.py
+
+      - name: Open PR if diagrams changed


What makes this conditional on changes? Is that somehow within peter-evans/create-pull-request?

I added an explicit Detect diagram changes step that runs git diff --quiet -- src/odb/doc/images/ and sets a changed output. The Open PR step is gated with if: steps.diagram-changes.outputs.changed == 'true', so no PR is opened if the diagrams are unchanged. This is explicit rather than relying on action internals.

maliberty · 2026-03-22T15:39:55Z

It is a unfortunate that this has no semantic information (i.e., number, string, string). Is there a way to add annotations?

The diagram reflects the grammar tokens as defined in def.y. The terminal names (NUMBER, STRING) come directly from the bison grammar, rr.war has no mechanism to inject external annotations into an already-generated SVG. To add semantic labels, the grammar itself would need named sub-rules (e.g., halo_dist, layer_name). That would be a change to def.y which is out of scope for this PR, but could be a follow-up. Happy to open an issue for it.

… check Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

luarss · 2026-03-22T23:31:54Z

+          valid=true
+          if [[ ! -s "$ebnf_war" ]] || [[ ! -s "$rr_war" ]]; then
+            valid=false
+          elif ! jar tf "$ebnf_war" | grep -q "WEB-INF/lib/Saxon-HE-12.9.jar"; then


Why is named a war but the file extension is really a jar?

ebnf-convert.war is a proper WAR, the .jar in the grep is Saxon-HE-12.9.jar, a dependency bundled inside WEB-INF/lib/ per standard WAR layout. rr.war is actually rr-lib-2.6.jar from Maven Central renamed to .war (verified by MD5), so it has no WEB-INF structure, its check looks for a class at the root level instead.

luarss · 2026-03-22T23:33:40Z

+      - name: Detect diagram changes
+        id: diagram-changes
+        run: |
+          if git diff --quiet -- src/odb/doc/images/; then


Will this catch untracked files?

git diff misses untracked files (new SVGs being generated for the first time). Switching to git status --porcelain src/odb/doc/images/ which reports both modified and untracked files in one check.

Signed-off-by: Sparsh Karna <78959094+sparsh-karna@users.noreply.github.com>

sparsh-karna · 2026-03-23T19:20:00Z

@maliberty @luarss I’ve updated the PR based on your suggestions. Let me know if anything still needs work, happy to iterate further if needed.

maliberty · 2026-03-24T13:37:20Z

+
+          echo "valid=$valid" >> "$GITHUB_OUTPUT"
+
+      - name: Build ebnf-convert and fetch rr.war


Do we really have to build from source on every PR?

No, the Gradle build is gated behind if: steps.cache-tools.outputs.cache-hit != 'true' || steps.validate-tools.outputs.valid != 'true'. The cache key railroad-tools-ebnf-v0.73-rr-2.6 is pinned to exact tool versions, so normal PRs skip the build entirely and just restore from cache.

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

…arsh-karna/OpenROAD into odb/grammar-railroad-diagrams

maliberty · 2026-03-25T18:53:14Z

@luarss are your concerns satisfied?

sparsh-karna · 2026-03-30T09:43:14Z

Hey @maliberty @luarss, I am sorry I wasn't available over the weekend. Is there any feedback or changes I have to do? I am happy to mend it according to your instructions.

luarss

LGTM

sparsh-karna force-pushed the odb/grammar-railroad-diagrams branch from 54ed0c5 to aeac43c Compare March 21, 2026 10:20

gemini-code-assist Bot reviewed Mar 21, 2026

View reviewed changes

Comment thread src/odb/doc/generate_railroad_diagrams.py Outdated

Comment thread src/odb/doc/generate_railroad_diagrams.py

Comment thread src/odb/doc/LEF_Grammar.md

Comment thread src/odb/doc/DEF_Grammar.md

sparsh-karna force-pushed the odb/grammar-railroad-diagrams branch from aeac43c to 20aaaf4 Compare March 21, 2026 10:44

sparsh-karna added 7 commits March 21, 2026 16:16

sync with main

e174368

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

added modified images

b790ae2

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

odb: fix Black formatting in generate_railroad_diagrams.py

e73ad78

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

sparsh-karna force-pushed the odb/grammar-railroad-diagrams branch from 20aaaf4 to e73ad78 Compare March 21, 2026 10:47

luarss suggested changes Mar 21, 2026

View reviewed changes

sparsh-karna requested a review from luarss March 21, 2026 13:52

luarss suggested changes Mar 21, 2026

View reviewed changes

Comment thread src/odb/doc/generate_railroad_diagrams.py Outdated

Comment thread src/odb/doc/generate_railroad_diagrams.py Outdated

Comment thread src/odb/doc/generate_railroad_diagrams.py Outdated

sparsh-karna added 2 commits March 22, 2026 00:41

odb: make 'all' the default target in generate_railroad_diagrams.py

22925eb

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

odb: remove vendored WARs, build ebnf-convert from source in CI

06aa958

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

sparsh-karna requested a review from luarss March 21, 2026 19:38

luarss suggested changes Mar 22, 2026

View reviewed changes

maliberty reviewed Mar 22, 2026

View reviewed changes

Comment thread src/odb/doc/generate_railroad_diagrams.py Outdated

maliberty reviewed Mar 22, 2026

View reviewed changes

odb: address review comments — ET xml parsing, cache validation, java…

020b626

… check Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

sparsh-karna requested review from luarss and maliberty March 22, 2026 17:19

odb: black lint formatting

76be7c1

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

luarss suggested changes Mar 22, 2026

View reviewed changes

odb: Update diagram change detection logic in workflow

6639525

Signed-off-by: Sparsh Karna <78959094+sparsh-karna@users.noreply.github.com>

sparsh-karna requested a review from luarss March 23, 2026 09:57

sparsh-karna marked this pull request as ready for review March 23, 2026 19:17

maliberty reviewed Mar 24, 2026

View reviewed changes

Comment thread .github/workflows/github-actions-update-grammar-railroad-diagrams.yml

maliberty reviewed Mar 24, 2026

View reviewed changes

sparsh-karna added 2 commits March 25, 2026 01:30

odb: ci: Skip grammar diagram update in private fork

685eb47

Signed-off-by: Sparsh Karna <sparsh2005karna@gmail.com>

Merge branch 'odb/grammar-railroad-diagrams' of https://github.com/sp…

b934754

…arsh-karna/OpenROAD into odb/grammar-railroad-diagrams

sparsh-karna requested a review from maliberty March 25, 2026 10:23

luarss approved these changes Mar 30, 2026

View reviewed changes

maliberty approved these changes Mar 30, 2026

View reviewed changes

maliberty merged commit 08138fb into The-OpenROAD-Project:master Mar 30, 2026
14 of 15 checks passed

sparsh-karna deleted the odb/grammar-railroad-diagrams branch March 30, 2026 22:42

		- name: Open PR if diagrams changed
		uses: peter-evans/create-pull-request@v7


		echo "valid=$valid" >> "$GITHUB_OUTPUT"

		- name: Build ebnf-convert and fetch rr.war

Uh oh!

Conversation

sparsh-karna commented Mar 21, 2026

Summary

What's Included

How to Regenerate Diagrams

Example Diagram:

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sparsh-karna commented Mar 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

luarss left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sparsh-karna commented Mar 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sparsh-karna commented Mar 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sparsh-karna commented Mar 23, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maliberty commented Mar 25, 2026

Uh oh!

sparsh-karna commented Mar 21, 2026 •

edited

Loading