Skip to content

[WIP] Render API doc markdown as native RST in docstrings#1474

Open
tanmay-db wants to merge 1 commit into
databricks:mainfrom
tanmay-db:docfmt-native-docstrings
Open

[WIP] Render API doc markdown as native RST in docstrings#1474
tanmay-db wants to merge 1 commit into
databricks:mainfrom
tanmay-db:docfmt-native-docstrings

Conversation

@tanmay-db

@tanmay-db tanmay-db commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Regenerates service docstrings with the SDK generator's new doc renderer, which parses OpenAPI spec descriptions as CommonMark and emits Sphinx-native RST instead of copying raw markdown through. Docstring-only change: code is AST-identical to main in all 33 files.

Why

Spec descriptions are written in markdown, but Python docstrings are rendered by Sphinx as RST. The generator previously copied the markdown through verbatim (with word-wrapping), so the published SDK docs show literal backticks instead of code formatting, bullet lists broken by wrapping (* markers dangling mid-line), multi-line code examples collapsed onto one line, and unresolved [text]: url reference links. This PR applies the new renderer's output for the spec revision the SDK is already on, so the diff contains only documentation fixes — no API changes.

Before → After examples

1. Inline code and bullet listsbefore: jobs.py L282–L284 on main (live rendering today: note the italics instead of code formatting and the dangling * at the end of the first line) → after: jobs.py L288–L290

Before (word-wrap left the second bullet's * dangling at the end of the first line; single backticks render as italics in RST, not code):

* `STANDARD`: Enables cost-efficient execution of serverless workloads. *
`PERFORMANCE_OPTIMIZED`: Prioritizes fast startup and execution times through rapid scaling and
optimized cluster performance.

After (proper RST list with hanging indent; double-backtick literals):

- ``STANDARD``: Enables cost-efficient execution of serverless workloads.
- ``PERFORMANCE_OPTIMIZED``: Prioritizes fast startup and execution times through rapid scaling
  and optimized cluster performance.

2. Code examplesbefore: sql.py L9533–L9535 on main (live rendering today: under the format parameter, the CSV example is a single garbled line) → after: sql.py L9542–L9553

Before (the fenced CSV example was word-wrapped onto a single line, destroying it):

For example, the output of `SELECT
concat('id-', id) AS strCol, id AS intCol, null as nullCol FROM range(3)` would look like this:

``` strCol,intCol,nullCol id-1,1,null id-2,2,null id-3,3,null ```

After (a real .. code-block:: directive Sphinx renders as a code box):

For example, the output of ``SELECT concat('id-', id) AS strCol, id AS intCol, null as
nullCol FROM range(3)`` would look like this:

.. code-block::

   strCol,intCol,nullCol
   id-1,1,null
   id-2,2,null
   id-3,3,null

3. Linksbefore: sql.py L9527 + orphan definition at L9537 on main (live rendering today: same page, [Apache Arrow streaming format]. appears as literal bracketed text with the bare URL definition below) → after: sql.py L9537–L9539

Before (markdown reference link never resolves in RST; the URL definition dangles at the bottom of the docstring):

will be formatted as Apache Arrow Stream. See the [Apache Arrow streaming format].
...
[Apache Arrow streaming format]: https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format

After (RST anonymous hyperlink, rendered as a real link):

will be formatted as Apache Arrow Stream. See the `Apache Arrow streaming format
<https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format>`__.

What changed

Interface changes

None.

Behavioral changes

None — docstring text only. Every changed file is AST-identical to main once docstrings (including PEP 257 attribute docstrings) are stripped.

Internal changes

  • Regenerated docstrings in 33 modules under databricks/sdk/service/ (plus AccountClient/WorkspaceClient method docs) from the same OpenAPI spec revision the SDK is currently on.
  • bundledeployments.py, database.py, and postgres.py are intentionally excluded: main lags the spec slightly there, and including them would pull API changes into this docs-only diff. They will pick up the new doc rendering with the next regular spec bump.
  • Output formatted with the repo's pinned ruff==0.5.6 (make fmt equivalent).

How is this tested?

  • AST equivalence check: each changed file on main and on this branch was parsed, all docstrings stripped, and the remaining code compared — identical for all 33 files, confirming a docs-only diff.
  • The renderer itself is covered by table-driven unit tests in the generator (markdown → RST cases for inline code, fenced blocks, links, lists, paragraphs) plus the generator's full test suite.
  • Rendering spot-checked across the diff: double-backtick literals, .. code-block:: directives, anonymous hyperlinks, and list formatting (examples linked above).

Note for sequencing: this should merge after the generator-side change lands, so the next automated regeneration preserves (rather than reverts) these docstrings.

This pull request and its description were written by Isaac.

@tanmay-db tanmay-db temporarily deployed to test-trigger-is June 12, 2026 14:02 — with GitHub Actions Inactive
@tanmay-db tanmay-db changed the title Render API doc markdown as native RST in docstrings [WIP] Render API doc markdown as native RST in docstrings Jun 12, 2026
Regenerated service docstrings with genkit's new docfmt renderer, which
parses spec descriptions as CommonMark and emits Sphinx-native RST
instead of leaking raw markdown:

- inline backticks -> ``double-backtick`` literals
- fenced code examples -> .. code-block:: directives
- markdown links -> anonymous RST hyperlinks
- bullet/numbered lists -> proper RST lists (previously broken by
  word-wrap, leaving list markers dangling mid-line)
- stray lone list markers (proto-comment artifacts) are dropped instead
  of rendering an empty list

Generated from the same OpenAPI spec the SDK is current with, so the
diff is docstring-only: code is AST-identical to main for all 33 files.
Formatted with the repo's pinned ruff 0.5.6 (make fmt equivalent).

Co-authored-by: Isaac
@tanmay-db tanmay-db force-pushed the docfmt-native-docstrings branch from cc67e34 to 7f90f2d Compare June 12, 2026 14:48
@tanmay-db tanmay-db temporarily deployed to test-trigger-is June 12, 2026 14:48 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-py

Inputs:

  • PR number: 1474
  • Commit SHA: 7f90f2de5408a016e935c188805d4258aec802c8

Checks will be approved automatically on success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant