Commit 98c3ab0
## Pair `exclude_types` as a canonical NeighborGraph transform
> **Stacked on #5715 (NeighborGraph PR-D).** Only the commits after
`6fc45bd26` belong to this PR; will rebase onto master once #5715
merges.
Makes pair-type exclusion a single canonical transform applied once at
the neighbor-list/graph build seam, and uses it to add descriptor-level
`exclude_types` support to the dpa1 graph path (removing that
eligibility gate), consistently across dpmodel, pt_expt, jax, and the
C++ inference path.
### What changed
- **Canonical graph transform** `apply_pair_exclusion(graph, atype,
pair_excl, *, compact=False)` in `deepmd/dpmodel/utils/neighbor_graph/`:
ANDs `PairExcludeMask.build_edge_exclude_mask` into `graph.edge_mask`.
`compact=False` is mask-only (shape-static, export/AOTI-safe);
`compact=True` drops masked edges (eager-only; raises on angle-carrying
graphs). Idempotent.
- **Atomic-model seam** (`base_atomic_model`): model-level
`pair_exclude_types` is applied at BUILD time, so
`forward_common_atomic{,_graph}` no longer re-apply it — they consume a
pre-excluded nlist/graph (single-owner, **not** a backstop). To keep
that from being fail-open, an eager (numpy) fail-safe assertion
(`_assert_nlist_pair_excluded` / `_assert_graph_pair_excluded`) rejects
a non-excluded input at the seam with a clear error; it is a no-op under
`torch.export` / jax `jit` and in compiled production (the
exported-`.pt2` / C++ paths are covered by ingestion-site tests). See
the ingestion-path inventory below.
- **dpa1 graph path supports descriptor-level `exclude_types`**: the
`NotImplementedError` and the `uses_graph_lower()` exclude condition are
removed; exclusion applied inside `DescrptBlockSeAtten.call_graph`
before the segment sums. Graph-vs-dense parity at non-binding sel is
exact (rtol=atol=1e-12, attn_layer 0 and 2, type_one_side both).
- **Build-time exclusion (dispatcher)**: `build_neighbor_graph` and the
pt_expt graph builders (dense/ase/vesin/nv) gain optional
`pair_excl`/`compact` with default post-search application;
`_call_common_graph` passes model-level excludes at build time; oracle
set-equality tests per available builder.
- **Dense-nlist port**: `apply_pair_exclusion_nlist(nlist, atype_ext,
pair_excl)` extracted from the inline seam code; `build_neighbor_list` +
Vesin/Nv/Default strategies gain `pair_excl`; `return_mode='edges'` +
`pair_excl` fails fast.
- **C++ twin**: `buildPairExcludeTable` / `applyPairExclusion` /
`applyPairExclusionNlist` in `source/api_cc/include/commonPT.h`,
mirroring the Python transforms (same arg order/variable names,
cross-referenced docs); `pair_exclude_types` serialized into `.pt2`
`metadata.json` and rebuilt once in `DeepPotPTExpt::init`
(device-resident table, uploaded once — no per-step H2D). Exclusion is
applied at the C++ ingestion seam — the **single owner** on every run
path; the exported lower consumes a pre-excluded input and never
re-applies it. New gtest (8 tests) vs Python DeepEval reference at
1e-10, plus multi-rank LAMMPS exclusion tests (below).
- **Fix**: `apply_pair_exclusion` uses `logical_and` + bool cast
(array_api_strict rejected `bool*bool`), caught by the jax/strict
consistency rows now traversing the graph path.
### Ingestion-path inventory (exclusion coverage)
Because the seam is now the single owner (fail-open if skipped), here is
every entry point that builds a nlist/graph reaching the
exclusion-owning lower, and how each applies exclusion:
| Entry point | Applies exclusion at | Coverage |
|---|---|---|
| dpmodel `_call_common` (dense) | `build_neighbor_list(pair_excl=)` |
eager guard + consistency rows |
| dpmodel `_call_common_graph` | `build_neighbor_graph(pair_excl=)` |
eager guard + graph/dense parity |
| dpmodel descriptor-level `exclude_types` | `apply_pair_exclusion` in
`call_graph` | dpa1 graph parity |
| pt_expt DeepEval — nlist | `apply_pair_exclusion_nlist` / vesin
`pair_excl` | parity vs dpmodel |
| pt_expt DeepEval — graph | `_build_eval_graph(pair_excl=)`
(dense/ase/vesin/nv) | parity vs dpmodel |
| pt_expt/jax/pd training | `build_neighbor_graph(pair_excl=)` in
stat/forward | training e2e |
| input statistics | `build_neighbor_list(pair_excl=)` in
`EnvMatStatSe.iter` | stat tests (hash key = follow-up, pre-existing) |
| **jax-md `call_lower`** | `apply_pair_exclusion_nlist` at the seam
(**fixed here**) | `test_dense_neighbor_applies_model_pair_exclusion` |
| C++ SP dense | `applyPairExclusionNlist` | gtest |
| C++ MP dense (with-comm) | `applyPairExclusionNlist` (**fixed here**)
| DPA3 MP≡SP + active-vs-baseline |
| C++ SP graph | `applyPairExclusion` | gtest |
| C++ MP graph (non-MP, extended) | `applyPairExclusion` | dpa1 MP≡SP +
active-vs-baseline |
| C++ MP graph (message-passing) | — | **fail-fast** (PR-G) |
| C++ / DeepEval edge lower (SeZM/DPA4) | baked into the exported graph
by the **pt backend** (unchanged) | out of scope — pt-backend export |
Guard: `forward_common_atomic{,_graph}` additionally carry an eager
fail-safe assertion (numpy only) that rejects a non-excluded input, so
any *future* dpmodel/jax ingestion miss fails loudly instead of silently
including excluded pairs.
### Known limitations
- nv builder's `pair_excl` path has no local oracle test (CUDA-only); to
be validated on a GPU box.
- Input statistics remain on the dense path (graph-native stats is a
separate follow-on); the stat-cache **hash key** does not yet include
`pair_exclude_types` — pre-existing (predates this PR) and tracked as a
separate fix.
- smooth_type_embedding + exclude parity untestable at 1e-12
(pre-existing dense sel-padding divergence, #5715).
- `build_edge_exclude_mask` still returns int32 (bool cast at call
sites; follow-up).
### Spin routing
Spin models auto-inject `exclude_types` (virtual/placeholder types) into
their backbone descriptor; before this PR that condition *accidentally*
kept spin on the dense path. With exclude_types now graph-eligible, spin
backbones flipped onto the carry-all graph route, which (a) diverges
from the sel-capped reference on sel-binding spin systems and (b) trips
a torch-inductor scatter codegen assertion during spin `.pt2` export.
Fixed explicitly: `DescrptDPA1.disable_graph_lower()` (not serialized;
re-derived structurally) is set in `SpinModel.__init__` — the single
choke point covering `get_spin_model`, `SpinModel.deserialize`, and the
pt_expt spin classes — plus a belt-and-braces
`neighbor_graph_method="legacy"` at `SpinModel.call_common`. Regression
tests pin the routing and its serialize→deserialize survival; the full
spin export suite (23) and spin checkpoint-interop suite (12) are green.
### Verification
Full pt_expt suite: 1196 passed / 39 skipped / 3 failed — the 3 failures
(`test_dpa4_freeze_to_pt2`, `test_dpa4_deep_eval_*`) are byte-identical
on the base commit (pre-existing torch-inductor dpa4 export issue on
this box, unrelated). dpmodel exclusion suites 69 passed; consistency
dpa1 99 passed/63 skipped (incl. jax + array_api_strict exclude rows);
C++ `Dpa1PairExcl` gtest 8/8.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added model-level pair-type exclusion across neighbor lists and
neighbor graphs, including propagation into exported `.pt2` metadata and
enforcement at inference ingestion.
* Expanded DPA1 graph-native attention support (including higher
attention layers) and introduced stable segmented reductions for graph
attention.
* Added export-time guards to block unsupported graph tracing on older
torch versions.
* **Bug Fixes**
* Improved dense vs graph consistency for exclusion/masking behavior.
* Preserved spin model legacy neighbor routing for sel-binding cases.
* **Documentation**
* Refreshed graph-export eligibility and backend behavior notes for
attention/smooth differences.
* **Tests**
* Added/expanded parity, exclusion, compaction, and tracing regression
coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 5cd05f4 commit 98c3ab0
66 files changed
Lines changed: 4070 additions & 205 deletions
File tree
- deepmd
- dpmodel
- atomic_model
- descriptor
- model
- utils
- neighbor_graph
- jax
- jax2tf
- jax_md
- model
- train
- pt_expt
- entrypoints
- infer
- model
- train
- utils
- pt/utils
- tf2
- model
- train
- utils
- doc/model
- source
- api_cc
- include
- src
- tests
- install
- lmp/tests
- tests
- common/dpmodel
- consistent
- io
- model
- infer
- jax
- pt_expt
- descriptor
- model
- utils
- tf2
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | 2 | | |
4 | 3 | | |
5 | 4 | | |
| |||
270 | 269 | | |
271 | 270 | | |
272 | 271 | | |
273 | | - | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
274 | 277 | | |
275 | 278 | | |
276 | 279 | | |
| |||
294 | 297 | | |
295 | 298 | | |
296 | 299 | | |
297 | | - | |
298 | | - | |
299 | | - | |
300 | | - | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
301 | 308 | | |
302 | 309 | | |
303 | 310 | | |
| |||
326 | 333 | | |
327 | 334 | | |
328 | 335 | | |
329 | | - | |
330 | | - | |
331 | | - | |
332 | | - | |
333 | | - | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
334 | 341 | | |
335 | 342 | | |
336 | 343 | | |
| |||
356 | 363 | | |
357 | 364 | | |
358 | 365 | | |
359 | | - | |
360 | | - | |
361 | | - | |
362 | | - | |
363 | | - | |
364 | | - | |
365 | | - | |
366 | | - | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
367 | 373 | | |
368 | 374 | | |
369 | 375 | | |
| |||
373 | 379 | | |
374 | 380 | | |
375 | 381 | | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
376 | 468 | | |
377 | 469 | | |
378 | 470 | | |
| |||
673 | 765 | | |
674 | 766 | | |
675 | 767 | | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
676 | 771 | | |
677 | 772 | | |
678 | 773 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
356 | 356 | | |
357 | 357 | | |
358 | 358 | | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
359 | 362 | | |
360 | 363 | | |
361 | 364 | | |
| |||
432 | 435 | | |
433 | 436 | | |
434 | 437 | | |
435 | | - | |
436 | | - | |
437 | | - | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
438 | 443 | | |
439 | 444 | | |
440 | 445 | | |
441 | 446 | | |
442 | 447 | | |
443 | 448 | | |
444 | 449 | | |
| 450 | + | |
| 451 | + | |
445 | 452 | | |
446 | 453 | | |
447 | 454 | | |
448 | 455 | | |
449 | | - | |
450 | | - | |
451 | | - | |
452 | | - | |
453 | | - | |
454 | | - | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
455 | 474 | | |
456 | 475 | | |
457 | 476 | | |
| |||
581 | 600 | | |
582 | 601 | | |
583 | 602 | | |
584 | | - | |
585 | | - | |
586 | | - | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
587 | 606 | | |
588 | 607 | | |
589 | 608 | | |
| |||
632 | 651 | | |
633 | 652 | | |
634 | 653 | | |
635 | | - | |
| 654 | + | |
636 | 655 | | |
637 | 656 | | |
638 | 657 | | |
639 | | - | |
640 | 658 | | |
641 | | - | |
642 | | - | |
643 | | - | |
644 | | - | |
645 | | - | |
646 | | - | |
647 | | - | |
648 | | - | |
649 | | - | |
650 | | - | |
651 | | - | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
652 | 663 | | |
653 | | - | |
654 | | - | |
655 | 664 | | |
656 | 665 | | |
657 | 666 | | |
| |||
665 | 674 | | |
666 | 675 | | |
667 | 676 | | |
668 | | - | |
669 | | - | |
670 | | - | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
671 | 681 | | |
672 | 682 | | |
673 | 683 | | |
| |||
677 | 687 | | |
678 | 688 | | |
679 | 689 | | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
680 | 696 | | |
681 | 697 | | |
682 | 698 | | |
| |||
687 | 703 | | |
688 | 704 | | |
689 | 705 | | |
690 | | - | |
| 706 | + | |
691 | 707 | | |
692 | 708 | | |
693 | 709 | | |
| |||
1383 | 1399 | | |
1384 | 1400 | | |
1385 | 1401 | | |
1386 | | - | |
| 1402 | + | |
| 1403 | + | |
| 1404 | + | |
| 1405 | + | |
| 1406 | + | |
1387 | 1407 | | |
1388 | 1408 | | |
1389 | 1409 | | |
| |||
1756 | 1776 | | |
1757 | 1777 | | |
1758 | 1778 | | |
1759 | | - | |
| 1779 | + | |
1760 | 1780 | | |
1761 | 1781 | | |
| 1782 | + | |
1762 | 1783 | | |
1763 | 1784 | | |
1764 | 1785 | | |
| |||
1767 | 1788 | | |
1768 | 1789 | | |
1769 | 1790 | | |
1770 | | - | |
1771 | | - | |
1772 | | - | |
1773 | | - | |
1774 | | - | |
1775 | 1791 | | |
1776 | 1792 | | |
1777 | 1793 | | |
1778 | 1794 | | |
1779 | 1795 | | |
1780 | 1796 | | |
1781 | 1797 | | |
| 1798 | + | |
| 1799 | + | |
| 1800 | + | |
| 1801 | + | |
| 1802 | + | |
| 1803 | + | |
| 1804 | + | |
1782 | 1805 | | |
1783 | 1806 | | |
1784 | | - | |
1785 | 1807 | | |
1786 | 1808 | | |
1787 | 1809 | | |
| |||
0 commit comments