Skip to content

feat: vgplot Python API#1007

Closed
karthik-anand wants to merge 310 commits into
uwdata:mainfrom
cmudig:pythonAPI_improvements
Closed

feat: vgplot Python API#1007
karthik-anand wants to merge 310 commits into
uwdata:mainfrom
cmudig:pythonAPI_improvements

Conversation

@karthik-anand

@karthik-anand karthik-anand commented Apr 5, 2026

Copy link
Copy Markdown

This PR introduces a Python API for vgplot, letting users build Mosaic visualizations programmatically in Python instead of writing raw JSON/YAML specs by hand. It adds a new packages/vgplot/vgplot-python package (published to PyPI as vgplot) and wires Python spec output into the docs and examples.

Key additions

  • vgplot Python package (packages/vgplot/vgplot-python) — a fluent builder API mirroring the vgplot JavaScript API, using Python-idiomatic snake_case naming (e.g. line_y, x_domain instead of lineY, xDomain). Python keywords are escaped with a trailing underscore (from_, for_).
  • Composable View objectsvg.plot(), vg.vconcat(), and vg.hconcat() return View objects that render directly in Jupyter/marimo and export to JSON.
  • Automatic data/param discovery — data sources (vg.parquet(), vg.csv(), vg.table(), vg.spatial()) and params/selections declared as local variables are discovered from the caller's frame at render/export time; no explicit registration or passing required.
  • AST-to-Python code generator (packages/vgplot/spec/src/ast-to-python.js) — a JavaScript-side utility that translates parsed Mosaic spec ASTs into equivalent Python, used to regenerate the specs/python/ and docs/public/specs/python/ examples. Regenerate with npm run docs:examples.
  • Python specs for all examples — every example now ships a generated Python version alongside the existing JSON/YAML/ESM, surfaced as a Python tab in the docs code groups.
  • Round-trip parity tests (test_full_round_trip.py) — verify that running each generated Python spec produces JSON identical to its specs/json/ reference fixture, and that the Python/JSON/ESM example sets stay in sync.

How it works

The package is hand-written fluent helpers over Mosaic's spec format (not codegen from a schema):

  1. Builder functions (plot.py, data.py, params.py, encodings.py) construct marks, directives, data sources, params/selections, and input widgets. Common marks and attributes have explicit helpers; anything else is reachable by its snake_case name via a dynamic __getattr__ fallback (e.g. vg.regression_y(...), vg.x_tick_rotate(45)) that maps snake_case → camelCase.

  2. View serialization (spec.py) — a View holds the composed layout, and at to_dict() / to_json() / display time it inspects the caller's local variables to collect the DataDef and param objects it references, assembling a complete Mosaic spec. The camelCase ↔ snake_case mapping is applied during serialization.

Option names match the vgplot API reference but in snake_case (xDomainx_domain, colorSchemecolor_scheme, filterByfilter_by).

How to use

Install the package (add mosaic-widget to render in notebooks):

pip install vgplot mosaic-widget

Build a scatter plot and render it in Jupyter or marimo:

import vgplot as vg

athletes = vg.parquet("data/athletes.parquet")

view = vg.plot(
    vg.dot(athletes, x="weight", y="height", fill="steelblue", opacity=0.5),
    vg.width(600),
    vg.height(400),
)

view  # renders the widget (or call view.show())

Add an interactive crossfilter selection:

import vgplot as vg

flights = vg.parquet("data/flights-200k.parquet")
brush = vg.selection.crossfilter()

view = vg.plot(
    vg.rect_y(flights, x=vg.bin("delay"), y=vg.count(), filter_by=brush, fill="steelblue"),
    vg.interval_x(bind=brush),
    vg.width(600),
)

view

Bind a scalar param to an input widget:

import vgplot as vg

walk = vg.parquet("data/random-walk.parquet")
bias = vg.param(0)

view = vg.vconcat(
    vg.slider(label="Bias", bind=bias, min=0, max=1000, step=1),
    vg.plot(
        vg.area_y(walk, x="t", y=vg.sql("v + $bias")),
        vg.width(680),
        vg.height(200),
    ),
)

view

Data sources and params are plain local variables — view discovers them automatically by name at render time, so no vg.data() wrapper or explicit passing is required. Export with view.to_dict() (a plain dict) or view.to_json(indent=2) (a JSON string).

@domoritz domoritz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noticed that we still use some string references in the specs and that the widget needs to be updated. Hopefully those are small changes.

Comment thread bin/prepare-examples.js Outdated
Comment thread docs/public/specs/python/weather.py Outdated
Comment thread dev/notebooks/weather.py
Comment thread dev/notebooks/weather.py
@domoritz

Copy link
Copy Markdown
Member

🎉

Screenshot 2026-06-30 at 09 51 22

@domoritz domoritz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @karthik-anand @SayeVikram @sittingthyme @Hamzu24 @Xinyue-Yang for all these changes!

@domoritz

domoritz commented Jul 6, 2026

Copy link
Copy Markdown
Member

@karthik-anand @SayeVikram, why did we remove the API generation from the schema in cmudig#24? I'm worried about maintainability of the vgplot python api if we don't automatically update when the js api changes.

@domoritz

domoritz commented Jul 9, 2026

Copy link
Copy Markdown
Member

Closed in favor of #1067

@domoritz domoritz closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants