Merge pull request #3143 from perspective-dev/engine-inner-joins

`Client::join` API

Author: texodus

docs/md/SUMMARY.md

@@ -22,6 +22,10 @@
         - [Selection and Ordering](./explanation/view/config/selection_and_ordering.md)
         - [`expressions`](./explanation/view/config/expressions.md)
     - [Advanced View Operations](./explanation/view/advanced.md)
+- [`Join`](./explanation/join.md)
+    - [Join Types](./explanation/join/join_types.md)
+    - [Join Options](./explanation/join/options.md)
+    - [Reactivity and Constraints](./explanation/join/reactivity.md)
 
 # JavaScript
 
@@ -32,6 +36,7 @@
     - [Cleaning up resources](./how_to/javascript/deleting.md)
     - [Hosting a `WebSocketServer` in Node.js](./how_to/javascript/nodejs_server.md)
     - [Customizing `perspective.worker()`](./how_to/javascript/custom_worker.md)
+    - [Joining Tables](./how_to/javascript/join.md)
 - [`perspective-viewer` Custom Element library](./how_to/javascript/viewer.md)
     - [Loading data](./how_to/javascript/loading_data.md)
     - [Theming](./how_to/javascript/theming.md)
@@ -52,6 +57,7 @@
 - [Callbacks and events](./how_to/python/callbacks.md)
 - [Multithreading](./how_to/python/multithreading.md)
 - [Hosting a WebSocket server](./how_to/python/websocket.md)
+- [Joining Tables](./how_to/python/join.md)
 - [`PerspectiveWidget` for JupyterLab](./how_to/python/jupyterlab.md)
 - [Virtual Servers](./how_to/python/virtual_server.md)
     - [DuckDB](./how_to/python/virtual_server/duckdb.md)

docs/md/explanation/join.md

@@ -0,0 +1,12 @@
+# Join
+
+`Client::join` creates a read-only `Table` by joining two source tables on a
+shared key column. The `left` and `right` arguments can be `Table` objects or
+string table names (as returned by `get_hosted_table_names()`). The resulting
+table is _reactive_: whenever either source table is updated, the join is
+automatically recomputed and any `View` derived from the joined table will
+update accordingly.
+
+Joined tables support the full `View` API — you can apply `group_by`,
+`split_by`, `sort`, `filter`, `expressions`, and all other `View` operations on
+the result, just as you would with any other `Table`.

docs/md/explanation/join/join_types.md

@@ -0,0 +1,25 @@
+# Join Types
+
+`Client::join` supports three join types, specified via the `join_type` option.
+The default is `"inner"`.
+
+## Inner Join (default)
+
+An inner join includes only rows where the key column exists in _both_ source
+tables. Rows from either table that have no match in the other are excluded.
+
+## Left Join
+
+A left join includes all rows from the left table. For left rows that have no
+match in the right table, right-side columns are filled with `null`.
+
+## Outer Join
+
+An outer join includes all rows from both tables. Unmatched rows on either side
+have their missing columns filled with `null`.
+
+| `join_type` | Left-only rows | Right-only rows |
+| ----------- | -------------- | --------------- |
+| `"inner"`   | excluded       | excluded        |
+| `"left"`    | included       | excluded        |
+| `"outer"`   | included       | included        |

docs/md/explanation/join/options.md

@@ -0,0 +1,35 @@
+# Join Options
+
+## `on` — Join Key Column
+
+The `on` parameter specifies the column name used to match rows between the left
+and right tables. This column must exist in the left table and, by default, must
+also exist in the right table with the same name and compatible type.
+
+The join key column becomes the index of the resulting table.
+
+## `right_on` — Different Right Key Column
+
+When the join key has a different name in the right table, use `right_on` to
+specify the right table's column name. The left table's column name (`on`) is
+used in the output schema; the right key column is excluded from the result.
+
+The `on` and `right_on` columns must have compatible types. An error is thrown
+if the types do not match.
+
+## `join_type` — Join Type
+
+Controls which rows are included in the result. See
+[Join Types](./join_types.md) for details.
+
+| Value       | Behavior                                              |
+| ----------- | ----------------------------------------------------- |
+| `"inner"`   | Only rows with matching keys in both tables (default) |
+| `"left"`    | All left rows; unmatched right columns are `null`     |
+| `"outer"`   | All rows from both tables; unmatched columns are `null` |
+
+## `name` — Table Name
+
+An optional name for the resulting joined table. If omitted, a random name is
+generated. This name is used to identify the table in the server's hosted table
+registry.

docs/md/explanation/join/reactivity.md

@@ -0,0 +1,46 @@
+# Reactivity and Constraints
+
+## Reactive Updates
+
+Joined tables are fully reactive. When either source table receives an
+`update()`, the join is automatically recomputed and any `View` created from the
+joined table will reflect the new data. This includes:
+
+- Updates that modify existing rows in either source table.
+- New rows added to either source table that create new matches.
+- Chained joins — if a joined table is itself used as input to another join,
+  updates propagate through the entire chain.
+
+## Duplicate Keys
+
+Like SQL, `join()` produces a cross-product for each matching key value. When
+multiple rows in the left table share the same key, each is paired with every
+matching row in the right table (and vice versa). The number of output rows for
+a given key is `left_count × right_count`.
+
+This behavior depends on whether the source tables are _indexed_:
+
+- **Unindexed tables** (no `index` option) — rows are appended, so duplicate
+  keys accumulate naturally. Each `update()` appends new rows, which may
+  introduce additional duplicates.
+- **Indexed tables** (`index` set to the join key) — each key appears at most
+  once per table, so the join produces at most one row per key. Updates replace
+  existing rows in-place rather than appending.
+
+## Read-Only
+
+Joined tables are read-only. Calling `update()`, `remove()`, `clear()`, or
+`replace()` on a joined table will throw an error. Data can only change
+indirectly, by updating the source tables.
+
+## Column Name Conflicts
+
+The left and right tables must not have overlapping column names (other than the
+join key). If a non-key column name appears in both tables, `join()` throws an
+error. Rename columns in your source data or use `View` expressions to avoid
+conflicts.
+
+## Source Table Deletion
+
+A source table cannot be deleted while a joined table depends on it. You must
+delete the joined table first, then delete the source tables.

docs/md/how_to/javascript/join.md

@@ -0,0 +1,65 @@
+# Joining Tables
+
+`perspective.join()` creates a read-only `Table` by joining two source tables on
+a shared key column. The result is reactive — it updates automatically when
+either source table changes. See [`Join`](../../explanation/join.md) for
+conceptual details.
+
+## Basic Inner Join
+
+```javascript
+const orders = await perspective.table([
+    { id: 1, product_id: 101, qty: 5 },
+    { id: 2, product_id: 102, qty: 3 },
+    { id: 3, product_id: 101, qty: 7 },
+]);
+
+const products = await perspective.table([
+    { product_id: 101, name: "Widget" },
+    { product_id: 102, name: "Gadget" },
+]);
+
+const joined = await perspective.join(orders, products, "product_id");
+const view = await joined.view();
+const json = await view.to_json();
+// [
+//   { product_id: 101, id: 1, qty: 5, name: "Widget" },
+//   { product_id: 101, id: 3, qty: 7, name: "Widget" },
+//   { product_id: 102, id: 2, qty: 3, name: "Gadget" },
+// ]
+```
+
+## Join Types
+
+Pass `join_type` in the options to select inner, left, or outer join behavior:
+
+```javascript
+// Left join: all left rows, nulls for unmatched right columns
+const left_joined = await perspective.join(left, right, "id", {
+    join_type: "left",
+});
+
+// Outer join: all rows from both tables
+const outer_joined = await perspective.join(left, right, "id", {
+    join_type: "outer",
+});
+```
+
+## Reactive Updates
+
+The joined table recomputes automatically when either source table is updated:
+
+```javascript
+const left = await perspective.table([{ id: 1, x: 10 }]);
+const right = await perspective.table([{ id: 2, y: "b" }]);
+
+const joined = await perspective.join(left, right, "id");
+const view = await joined.view();
+
+let json = await view.to_json();
+// [] — no matching keys yet
+
+await right.update([{ id: 1, y: "a" }]);
+json = await view.to_json();
+// [{ id: 1, x: 10, y: "a" }] — new match detected
+```

docs/md/how_to/python/join.md

@@ -0,0 +1,64 @@
+# Joining Tables
+
+`perspective.join()` creates a read-only `Table` by joining two source tables on
+a shared key column. The result is reactive — it updates automatically when
+either source table changes. See [`Join`](../../explanation/join.md) for
+conceptual details.
+
+## Basic Inner Join
+
+```python
+orders = perspective.table([
+    {"id": 1, "product_id": 101, "qty": 5},
+    {"id": 2, "product_id": 102, "qty": 3},
+    {"id": 3, "product_id": 101, "qty": 7},
+])
+
+products = perspective.table([
+    {"product_id": 101, "name": "Widget"},
+    {"product_id": 102, "name": "Gadget"},
+])
+
+joined = perspective.join(orders, products, "product_id")
+view = joined.view()
+json = view.to_json()
+```
+
+## Join Types
+
+Pass `join_type` to select inner, left, or outer join behavior:
+
+```python
+# Left join: all left rows, nulls for unmatched right columns
+left_joined = perspective.join(left, right, "id", join_type="left")
+
+# Outer join: all rows from both tables
+outer_joined = perspective.join(left, right, "id", join_type="outer")
+```
+
+## Reactive Updates
+
+The joined table recomputes automatically when either source table is updated:
+
+```python
+left = perspective.table([{"id": 1, "x": 10}])
+right = perspective.table([{"id": 2, "y": "b"}])
+
+joined = perspective.join(left, right, "id")
+view = joined.view()
+
+json = view.to_json()
+# [] — no matching keys yet
+
+right.update([{"id": 1, "y": "a"}])
+json = view.to_json()
+# [{"id": 1, "x": 10, "y": "a"}] — new match detected
+```
+
+## Async Client
+
+The async client has the same API:
+
+```python
+joined = await client.join(orders, products, "product_id", join_type="left")
+```

docs/md/how_to/rust.md

@@ -26,3 +26,64 @@ let mut options = TableInitOptions::default();
 options.set_name("my_data_source");
 client.table(data.into(), options).await?;
 ```
+
+# Joining Tables
+
+`Client::join` creates a read-only `Table` by joining two source tables on a
+shared key column. The result is reactive — it updates automatically when
+either source table changes. See [`Join`](../explanation/join.md) for
+conceptual details.
+
+```rust
+let orders = client.table(
+    TableData::Update(UpdateData::JsonRows(
+        "[{\"id\":1,\"product_id\":101,\"qty\":5},{\"id\":2,\"product_id\":102,\"qty\":3}]".into(),
+    )),
+    TableInitOptions::default(),
+).await?;
+
+let products = client.table(
+    TableData::Update(UpdateData::JsonRows(
+        "[{\"product_id\":101,\"name\":\"Widget\"},{\"product_id\":102,\"name\":\"Gadget\"}]".into(),
+    )),
+    TableInitOptions::default(),
+).await?;
+
+let joined = client.join(
+    (&orders).into(),
+    (&products).into(),
+    "product_id",
+    JoinOptions::default(),
+).await?;
+
+let view = joined.view(None).await?;
+let json = view.to_json().await?;
+```
+
+Use `JoinOptions` to configure the join type, table name, or `right_on` column:
+
+```rust
+let options = JoinOptions {
+    join_type: Some(JoinType::Left),
+    name: Some("orders_with_products".into()),
+    right_on: None,
+};
+
+let joined = client.join(
+    (&orders).into(),
+    (&products).into(),
+    "product_id",
+    options,
+).await?;
+```
+
+You can also join by table name strings instead of `Table` references:
+
+```rust
+let joined = client.join(
+    "orders".into(),
+    "products".into(),
+    "product_id",
+    JoinOptions::default(),
+).await?;
+```

rust/metadata/main.rs

@@ -32,8 +32,8 @@ use std::fs;
 
 use perspective_client::config::*;
 use perspective_client::{
-    ColumnWindow, DeleteOptions, OnUpdateData, OnUpdateOptions, SystemInfo, TableInitOptions,
-    UpdateOptions, ViewWindow,
+    ColumnWindow, DeleteOptions, JoinOptions, OnUpdateData, OnUpdateOptions, SystemInfo,
+    TableInitOptions, UpdateOptions, ViewWindow,
 };
 use perspective_viewer::config::ViewerConfigUpdate;
 use ts_rs::TS;
@@ -71,6 +71,7 @@ pub fn generate_type_bindings_js() -> Result<(), Box<dyn Error>> {
     ViewConfigUpdate::export_all_to(&path)?;
     OnUpdateData::export_all_to(&path)?;
     OnUpdateOptions::export_all_to(&path)?;
+    JoinOptions::export_all_to(&path)?;
     UpdateOptions::export_all_to(&path)?;
     DeleteOptions::export_all_to(&path)?;
     ViewWindow::export_all_to(&path)?;

rust/perspective-client/build.rs

@@ -57,6 +57,10 @@ fn prost_build() -> Result<()> {
             .field_attribute("ViewOnUpdateResp.delta", "#[ts(as = \"Vec::<u8>\")]")
             .field_attribute("ViewOnUpdateResp.delta", "#[serde(with = \"serde_bytes\")]")
             .type_attribute("ColumnType", "#[derive(ts_rs::TS)]")
+            .type_attribute(
+                "JoinType",
+                "#[derive(serde::Deserialize, ts_rs::TS)] #[serde(rename_all = \"snake_case\")]",
+            )
             .field_attribute("ViewToArrowResp.arrow", "#[serde(skip)]")
             .field_attribute("from_arrow", "#[serde(skip)]")
             .type_attribute(".", "#[derive(serde::Serialize)]")