docs: update design docs to reflect actual implementation

- Add prune() method to both spec and design docs
- Rename _propagate_to_children → _propagate_restrictions + _apply_propagation_rule
- Fix delete() part_integrity: post-check with rollback, not pre-check
- Add _part_integrity instance attribute
- Update files affected, verification, and implementation phases
- Mark open questions as resolved with actual decisions
- Mark export/restore as future work

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: dimitri-yatsenko

docs/design/restricted-diagram-spec.md

@@ -34,6 +34,7 @@ self._connection        # Connection — stored during __init__
 self._cascade_restrictions   # dict[str, list] — per-node OR restrictions
 self._restrict_conditions    # dict[str, AndList] — per-node AND restrictions
 self._restriction_attrs      # dict[str, set] — restriction attribute names per node
+self._part_integrity         # str — "enforce", "ignore", or "cascade"
 ```
 
 Initialized empty in `__init__`. Copied in the copy constructor (`Diagram(other_diagram)`).
@@ -105,29 +106,34 @@ rd = dj.Diagram(schema).restrict(Session & cond).restrict(Stimulus & cond2)
 1. Verify no existing cascade restrictions (raise if present)
 2. Same algorithm as `cascade` but accumulates into `_restrict_conditions` using `AndList`
 
-### `_propagate_to_children(self, parent_node, mode)`
+### `_propagate_restrictions(self, start_node, mode, part_integrity="enforce")`
 
-Internal. Propagates restriction from one node to its children.
+Internal. Propagates restrictions from `start_node` to all its descendants in topological order. Only processes descendants of `start_node` to avoid duplicate propagation when chaining `restrict()`.
 
-For each `out_edge(parent_node)`:
+Uses multiple passes (up to 10) to handle `part_integrity="cascade"` upward propagation, which can add new restricted nodes that need further propagation.
 
-1. Get `child_name, edge_props` from edge
-2. If child is an alias node (`.isdigit()`), follow through to the real child
-3. Get `attr_map`, `aliased` from `edge_props`
-4. Build parent `FreeTable` with current restriction
-5. Compute child restriction using propagation rules:
+For each restricted node, iterates over `out_edges(node)`:
+
+1. If target is an alias node (`.isdigit()`), follow through to real child via `out_edges(alias_node)`
+2. Delegate to `_apply_propagation_rule()` for the actual restriction computation
+3. Track propagated edges to avoid duplicate work
+4. Handle `part_integrity="cascade"`: if child is a part table and its master is not already restricted, propagate upward from part to master using `make_condition(master, (master.proj() & part.proj()).to_arrays(), ...)`, expand the allowed node set, and continue to next pass
+
+### `_apply_propagation_rule(self, parent_ft, parent_attrs, child_node, attr_map, aliased, mode, restrictions)`
+
+Internal. Applies one of three propagation rules to a parent→child edge:
 
 | Condition | Child restriction |
 |-----------|-------------------|
 | Non-aliased AND `parent_restriction_attrs ⊆ child.primary_key` | Copy parent restriction directly |
 | Aliased FK (`attr_map` renames columns) | `parent_ft.proj(**{fk: pk for fk, pk in attr_map.items()})` |
 | Non-aliased AND `parent_restriction_attrs ⊄ child.primary_key` | `parent_ft.proj()` |
 
-6. Accumulate on child:
-   - `cascade` mode: `_cascade_restrictions[child].extend(child_restr)` — list = OR
-   - `restrict` mode: `_restrict_conditions[child].extend(child_restr)` — AndList = AND
+Accumulates on child:
+- `cascade` mode: `restrictions.setdefault(child, []).extend(...)` — list = OR
+- `restrict` mode: `restrictions.setdefault(child, AndList()).extend(...)` — AndList = AND
 
-7. Handle `part_integrity="cascade"`: if child is a part table and its master is not already restricted, propagate upward from part to master using `make_condition(master, (master.proj() & part.proj()).to_arrays(), ...)`, then re-propagate from the master.
+Updates `_restriction_attrs` for the child with the relevant attribute names.
 
 ### `delete(self, transaction=True, prompt=None) -> int`
 
@@ -140,15 +146,16 @@ rd.delete()
 
 **Algorithm:**
 
-1. Pre-check `part_integrity="enforce"`: for each node in `_cascade_restrictions`, if it's a part table and its master is not restricted, raise `DataJointError`
-2. Get nodes with restrictions in topological order
-3. If `prompt`: show preview (table name + row count for each)
-4. Start transaction (if `transaction=True`)
-5. Iterate in **reverse** topological order (leaves first):
+1. Get non-alias nodes with restrictions in topological order
+2. If `prompt`: show preview (table name + row count for each)
+3. Start transaction (if `transaction=True`)
+4. Iterate in **reverse** topological order (leaves first):
    - `ft = FreeTable(conn, table_name)`
-   - `ft._restriction = self._cascade_restrictions[table_name]`
+   - `ft.restrict_in_place(self._cascade_restrictions[table_name])`
    - `ft.delete_quick(get_count=True)`
-6. On `IntegrityError`: diagnostic fallback — parse FK error for actionable message about unloaded schemas
+   - Track which tables had rows deleted
+5. On `IntegrityError`: cancel transaction, diagnostic fallback — parse FK error for actionable message about unloaded schemas
+6. Post-check `part_integrity="enforce"`: if any part table had rows deleted but its master did not, cancel transaction and raise `DataJointError`
 7. Confirm/commit transaction (same logic as current `Table.delete`)
 8. Return count from the root table
 
@@ -180,6 +187,34 @@ rd.preview()  # logs and returns {table_name: count}
 
 Returns dict of `{full_table_name: row_count}` for each node that has a cascade or restrict restriction.
 
+### `prune(self) -> Diagram`
+
+Removes tables with zero matching rows from the diagram. Returns a new `Diagram`.
+
+```python
+# Unrestricted: remove physically empty tables
+dj.Diagram(schema).prune()
+
+# After restrict: remove tables with zero matching rows
+dj.Diagram(schema).restrict(Session & cond).prune()
+```
+
+**Algorithm:**
+
+1. `result = Diagram(self)` — copy
+2. If restrictions exist (`_cascade_restrictions` or `_restrict_conditions`):
+   - For each restricted node, build `FreeTable` with restriction applied
+   - If `len(ft) == 0`: remove node from restrictions dict, `_restriction_attrs`, and `nodes_to_show`
+3. If no restrictions (unrestricted diagram):
+   - For each node in `nodes_to_show`, check `len(FreeTable(conn, node))`
+   - If 0: remove from `nodes_to_show`
+4. Return `result`
+
+**Properties:**
+- Idempotent — pruning twice yields the same result
+- Chainable — `restrict()` can be called after `prune()`
+- Skips alias nodes (`.isdigit()`)
+
 ### Visualization methods (gated)
 
 All existing visualization methods (`draw`, `make_dot`, `make_svg`, `make_png`, `make_image`, `make_mermaid`, `save`, `_repr_svg_`) raise `DataJointError("Install matplotlib and pygraphviz...")` when `diagram_active is False`. When active, they work as before.
@@ -307,23 +342,24 @@ For `_restrict_conditions`: values are `AndList` (AND). Each `.restrict()` call
 
 | File | Change |
 |------|--------|
-| `src/datajoint/diagram.py` | Restructure: single `Diagram(nx.DiGraph)` class, gate only visualization. Add `_connection`, restriction dicts, `cascade()`, `restrict()`, `_propagate_to_children()`, `delete()`, `drop()`, `preview()`, `_from_table()` |
+| `src/datajoint/diagram.py` | Restructure: single `Diagram(nx.DiGraph)` class, gate only visualization. Add `_connection`, restriction dicts, `_part_integrity`, `cascade()`, `restrict()`, `_propagate_restrictions()`, `_apply_propagation_rule()`, `delete()`, `drop()`, `preview()`, `prune()`, `_from_table()` |
 | `src/datajoint/table.py` | Rewrite `Table.delete()` (~200 → ~10 lines), `Table.drop()` (~35 → ~10 lines). Remove error-driven cascade code |
 | `src/datajoint/user_tables.py` | `Part.drop()`: pass `part_integrity` through to `super().drop()` |
-| `tests/integration/test_diagram_operations.py` | **New** — tests for `cascade`, `delete`, `drop`, `preview` |
+| `tests/integration/test_erd.py` | Add 5 `prune()` integration tests: unrestricted, after restrict, after cascade, idempotency, prune-then-restrict chaining |
 
 ## Verification
 
+All phases complete. Tests passing:
+
 1. All existing tests pass unchanged:
-   - `pytest tests/integration/test_cascading_delete.py -v`
-   - `pytest tests/integration/test_cascade_delete.py -v`
-   - `pytest tests/integration/test_erd.py -v`
-2. New tests pass: `pytest tests/integration/test_diagram_operations.py -v`
-3. Manual: `(Session & 'subject_id=1').delete()` behaves identically
-4. Manual: `dj.Diagram(schema).cascade(Session & cond).preview()` shows correct counts
-5. `dj.Diagram` works without matplotlib/pygraphviz for operational methods
+   - `pytest tests/integration/test_cascading_delete.py -v` — 12 tests
+   - `pytest tests/integration/test_cascade_delete.py -v` — 6 tests (3 MySQL + 3 PostgreSQL)
+   - `pytest tests/integration/test_erd.py -v` — 7 existing + 5 new prune tests
+2. Manual: `(Session & 'subject_id=1').delete()` behaves identically
+3. Manual: `dj.Diagram(schema).cascade(Session & cond).preview()` shows correct counts
+4. `dj.Diagram` works without matplotlib/pygraphviz for operational methods
 
-## Open questions resolved
+## Resolved design decisions
 
 | Question | Resolution |
 |----------|------------|
@@ -334,23 +370,23 @@ For `_restrict_conditions`: values are `AndList` (AND). Each `.restrict()` call
 | Upward cascade scope? | Master's restriction propagates to all its descendants (natural from re-running propagation) |
 | Can cascade and restrict be mixed? | No. Mutually exclusive modes. `cascade` applied once; `restrict` chainable |
 
-## Implementation phases
+## Implementation phases (all complete)
 
-### Phase 1: Restructure `Diagram` class
-Remove the `if/else` gate. Single class. Gate only visualization methods.
-Store `_connection` and restriction dicts. Adjust copy constructor.
+### Phase 1: Restructure `Diagram` class ✓
+Single class. Gate only visualization methods.
+Store `_connection`, restriction dicts, `_part_integrity`. Copy constructor copies all.
 
-### Phase 2: Restriction propagation
-`cascade()`, `restrict()`, `_propagate_to_children()`.
+### Phase 2: Restriction propagation ✓
+`cascade()`, `restrict()`, `_propagate_restrictions()`, `_apply_propagation_rule()`.
 Propagation rules, alias node handling, `part_integrity="cascade"` upward propagation.
 
-### Phase 3: Diagram operations
-`delete()`, `drop()`, `preview()`, `_from_table()`.
+### Phase 3: Diagram operations ✓
+`delete()`, `drop()`, `preview()`, `prune()`, `_from_table()`.
 Diagnostic fallback for unloaded schemas. Transaction handling.
 
-### Phase 4: Migrate `Table.delete()` and `Table.drop()`
-Rewrite to delegate to `Diagram`. Update `Part.drop()`.
-Remove dead cascade code from `table.py`.
+### Phase 4: Migrate `Table.delete()` and `Table.drop()` ✓
+Rewritten to delegate to `Diagram`. Updated `Part.drop()`.
+Dead cascade code removed from `table.py`.
 
-### Phase 5: Tests
-Run existing tests. Add `test_diagram_operations.py`.
+### Phase 5: Tests ✓
+Existing tests pass. 5 prune integration tests added to `test_erd.py`.

docs/design/restricted-diagram.md

@@ -276,7 +276,17 @@ rd = (dj.Diagram(schema)
       .restrict(Session & 'subject_id=1')
       .restrict(Stimulus & 'type="visual"'))
 rd.preview()      # show selected tables and row counts
-rd.export(path)   # includes upstream context, AND at convergence
+
+# prune: remove tables with zero matching rows
+rd = (dj.Diagram(schema)
+      .restrict(Subject & {'species': 'mouse'})
+      .restrict(Session & 'session_date > "2024-01-01"')
+      .prune())
+rd.preview()   # only tables with matching rows
+rd             # visualize the export subgraph
+
+# unrestricted prune: remove physically empty tables
+dj.Diagram(schema).prune()
 
 # drop: no restriction, drops entire tables
 rd = dj.Diagram(Session)   # Session + all descendants
@@ -296,9 +306,6 @@ rd.delete()
 Session.drop()
 # equivalent to:
 # dj.Diagram(Session).drop()
-
-# Preview and visualization
-rd.draw()      # visualize with restricted/affected nodes highlighted
 ```
 
 ## Advantages over current approach
@@ -313,64 +320,49 @@ rd.draw()      # visualize with restricted/affected nodes highlighted
 | Reusability | Delete-only | Delete, export, backup, sharing |
 | Inspectability | Opaque recursive cascade | Preview affected data before executing |
 
-## Implementation plan
+## Implementation status
 
-### Phase 1: RestrictedDiagram core
+### Phase 1: Diagram restructure and restriction propagation ✓
 
-1. Add `_cascade_restrictions` and `_restrict_conditions` to `Diagram` — per-node restriction storage
-2. Implement `_propagate_downstream(mode)` — walk edges in topo order, compute child restrictions via `attr_map`, accumulate as OR (cascade) or AND (restrict)
-3. Implement `cascade(table_expr)` — OR propagation entry point
-4. Implement `restrict(table_expr)` — AND propagation entry point
-5. Handle alias nodes during propagation (always OR for multiple FK paths from same parent)
-6. Handle `part_integrity` during cascade propagation (upward cascade from part to master)
+Single `Diagram(nx.DiGraph)` class with `_cascade_restrictions`, `_restrict_conditions`, `_restriction_attrs`, `_part_integrity`. `cascade()`, `restrict()`, `_propagate_restrictions()`, `_apply_propagation_rule()`. Alias node handling, `part_integrity="cascade"` upward propagation.
 
-### Phase 2: Graph-driven delete and drop
+### Phase 2: Graph-driven operations ✓
 
-1. Implement `Diagram.delete()` — reverse topo order, `delete_quick()` at each cascade-restricted node
-2. Implement `Diagram.drop()` — reverse topo order, `drop_quick()` at each node (no restrictions)
-3. Shared: unloaded-schema fallback error handling, `part_integrity` pre-checks
-4. Migrate `Table.delete()` to construct a diagram + `cascade()` internally
-5. Migrate `Table.drop()` to construct a diagram + `drop()` internally
-6. Preserve `Part.delete()` and `Part.drop()` behavior with diagram-based `part_integrity`
-7. Remove error-message parsing from the delete critical path (retain as diagnostic fallback)
+`delete()`, `drop()`, `preview()`, `prune()`, `_from_table()`. Unloaded-schema fallback error handling. `Table.delete()` and `Table.drop()` rewritten to delegate to `Diagram`. Dead cascade code removed.
 
-### Phase 3: Preview and visualization
+### Phase 3: Tests ✓
 
-1. `Diagram.preview()` — show restricted nodes with row counts
-2. `Diagram.draw()` — highlight restricted nodes, show restriction labels
+All existing tests pass. 5 prune integration tests added to `test_erd.py`.
 
 ### Phase 4: Export and backup (future, #864/#560)
 
-1. `Diagram.export(path)` — forward topo order, fetch + write at each restrict-restricted node
-2. Upward pass to include referenced parent rows (referential context)
-3. `Diagram.restore(path)` — forward topo order, insert at each node
+Not yet implemented. See "Future work" above.
 
-## Files affected
+## Files changed
 
 | File | Change |
 |------|--------|
-| `src/datajoint/diagram.py` | Add `cascade()`, `restrict()`, `_propagate_downstream()`, `delete()`, `drop()`, `preview()` |
-| `src/datajoint/table.py` | Rewrite `Table.delete()` and `Table.drop()` to use diagram internally |
-| `src/datajoint/user_tables.py` | Update `Part.delete()` and `Part.drop()` to use diagram-based part_integrity |
-| `src/datajoint/dependencies.py` | Possibly add helper methods for edge traversal with attr_map |
-| `tests/integration/test_cascading_delete.py` | Update tests, add graph-driven cascade tests |
-| `tests/integration/test_diagram.py` | New tests for restricted diagram |
+| `src/datajoint/diagram.py` | Single `Diagram(nx.DiGraph)` class with `cascade()`, `restrict()`, `_propagate_restrictions()`, `_apply_propagation_rule()`, `delete()`, `drop()`, `preview()`, `prune()`, `_from_table()` |
+| `src/datajoint/table.py` | `Table.delete()` (~200 → ~10 lines) and `Table.drop()` (~35 → ~10 lines) rewritten to delegate to `Diagram`. Dead cascade code removed |
+| `src/datajoint/user_tables.py` | `Part.drop()`: pass `part_integrity` through to `super().drop()` |
+| `tests/integration/test_erd.py` | 5 prune integration tests added |
 
-## Open questions
+## Resolved design decisions
 
-1. **Should `cascade()`/`restrict()` return a new object or mutate in place?**
-   Returning a new object enables chaining (`diagram.restrict(A).restrict(B)`) and keeps the original diagram reusable. Mutating in place is simpler but prevents reuse.
+| Question | Resolution |
+|----------|------------|
+| Return new or mutate? | Return new `Diagram` — enables chaining and keeps original reusable |
+| Upward propagation scope? | Master's restriction propagates to all its descendants (natural from re-running `_propagate_restrictions`) |
+| Transaction boundaries? | Build diagram (read-only), preview, confirm, execute all deletes in one transaction |
+| Lazy vs eager propagation? | Eager — propagate when `cascade()`/`restrict()` is called. Restrictions are `QueryExpression` objects, not executed until `preview()`/`delete()` |
+| Export upward context? | Deferred to future work (Phase 4) |
 
-2. **Upward propagation scope for `part_integrity="cascade"`:**
-   When a restriction propagates up from part to master, should the master's restriction then propagate to the master's *other* parts and descendants? The current implementation does this (lines 1098–1108 of `table.py`). The diagram approach would naturally do the same — restricting the master triggers downstream propagation to all its children.
+## Future work
 
-3. **Transaction boundaries:**
-   The current `Table.delete()` wraps everything in a single transaction with user confirmation. The diagram-based delete should preserve this: build the restricted diagram (read-only), show preview, get confirmation, then execute all deletes in one transaction.
+### Export and backup (#864/#560)
 
-4. **Lazy vs eager restriction propagation:**
-   Eager: propagate all restrictions when `restrict()` is called (computes row counts immediately).
-   Lazy: store parent restrictions and propagate during `delete()`/`export()` (defers queries).
-   Eager is better for preview but may issue many queries upfront. Lazy is more efficient when the user just wants to delete without preview.
+Not yet implemented. Planned:
 
-5. **Export: upward context scope.**
-   When exporting, non-downstream tables should be included for referential integrity. How far upstream? Options: (a) all ancestors of restricted nodes, (b) only directly referenced parents, (c) full referential closure. Full closure is safest but may pull in large amounts of unrestricted data.
+- `Diagram.export(path)` — forward topo order, fetch + write at each restrict-restricted node
+- Upward pass to include referenced parent rows (referential context)
+- `Diagram.restore(path)` — forward topo order, insert at each node