Skip to content

Commit 0b8b4fb

Browse files
wanghan-iapcmHan Wang
andauthored
feat(dpmodel): NeighborGraph support for dpa1 tebd_input_mode="strip" (#5747)
## Summary Adds NeighborGraph (graph-native lower) support for the dpa1 descriptor with `tebd_input_mode="strip"`, closing the last descriptor-level gap that forced strip-mode models (and `se_atten_v2`, which is strip-by-construction) onto the legacy dense path. The dense strip branch factorizes the per-neighbor feature as `gg = gg_s*gg_t + gg_s` — a radial-only geometric embedding times a type-pair strip embedding (optionally switch-smoothed). Because this has **no neighbor-axis coupling**, it maps to the graph path edge-for-edge. The change is: - **Kernel** (`DescrptBlockSeAtten`): a new per-edge helper `_graph_edge_gg_strip` (op-for-op mirror of the dense strip branch, including the `center*ntypes_pad + nei` nei-fastest two-side table layout and `int64` gather indices), selected by a `concat`/`strip` branch in `call_graph`. - **Routing** (`DescrptDPA1.uses_graph_lower`): admits `strip`, while keeping compressed descriptors and `exclude_types` on the dense path (they have no graph kernel here). `se_atten_v2` inherits this and becomes graph-eligible for free. - **pt_expt**: the two graph `make_fx` export tests are parametrized over `tebd_input_mode` to prove the strip kernel is fx-traceable. No new op, no attention/`segment_sum` change, no C++/serialization change. ## Scope This PR is deliberately **independent of #5733** (graph `exclude_types`). It does **not** change `exclude_types` eligibility — the `uses_graph_lower` `and not exclude_types` gate and the `call_graph` `exclude_types` raise are both kept. When both land, whichever merges second resolves a small (2–3 line) mechanical conflict at `uses_graph_lower` / the `call_graph` guard. ## Test plan - Block-level graph-vs-dense strip parity at `rtol=atol=1e-12` over `type_one_side × smooth` (attn=0) and `type_one_side` (attn=2, non-smooth). - Descriptor-level routed-`call` vs `_call_dense` parity over `type_one_side × smooth × attn_layer` (incl. attn=2 + smooth=True, bit-exact via the `static_nnei` adapter), plus a negative-contract gate test (compressed → dense, strip+`exclude_types` → dense). - `se_atten_v2` eligibility + graph-vs-dense parity (replaces the obsolete "strip stays dense" test). - pt_expt strip `make_fx` export; cross-backend consistency strip cases now route pt_expt through the graph adapter. Validated on **CPU** and on **GPU (Tesla T4, cuda:0)**: pt_expt dpa1 50 passed, consistency strip 22 passed + `se_atten_v2` 110 passed, dpmodel strip suites 46 passed. No tolerances relaxed, no tests skipped. ## Known limitations - **Compression** stays on the dense path by design (strip-only tabulation has no graph kernel); the gate excludes `self.compress`. - **`exclude_types`** stays dense (out of scope — owned by #5733). - **jax** graph lower remains energy-only (analytical force on the graph route is a separate follow-up). - The graph path's `segment_sum`→`index_add` is atomic/non-deterministic on CUDA (1–2 fp64 ULP), inherent to atomic scatter; GPU parity validated within tolerance. - Pre-existing (not introduced here): a softmax `RuntimeWarning` on the shared attention path (max over fully-masked segments), also present on the concat path. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary * **New Features** * Added graph-based execution support for the additional stripped type-embedding mode. * **Bug Fixes** * Updated graph routing eligibility: compressed descriptors and excluded-type configurations now reliably fall back to dense execution; graph routing is disabled when compression is enabled. * **Tests** * Added bit-exact parity tests between graph and dense paths for the new stripped mode (including routing eligibility checks). * Expanded FX graph export/trace coverage for both embedding modes. * Adjusted neighbor-list fallback validation with model-specific tolerance handling and added a new smooth variant. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>
1 parent d8892c7 commit 0b8b4fb

6 files changed

Lines changed: 397 additions & 37 deletions

File tree

deepmd/dpmodel/descriptor/dpa1.py

Lines changed: 107 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -432,18 +432,24 @@ def uses_graph_lower(self) -> bool:
432432
433433
The graph-native lower (``call_graph``) covers the factorizable path
434434
AND transformer attention (``attn_layer >= 0``, NeighborGraph PR-D)
435-
with concat type-embedding and no type exclusion. Remaining ineligible
436-
configs (``tebd_input_mode == "strip"``, ``exclude_types``) fall back
437-
to the legacy dense path, so those models keep working unchanged.
435+
with concat OR strip type-embedding. Remaining ineligible configs
436+
(``exclude_types``, and compressed descriptors) fall back to the legacy
437+
dense path, so those models keep working unchanged.
438438
439439
Eligibility does NOT imply numerical interchangeability with the
440440
dense route for every config: with ``smooth_type_embedding=True``
441441
the carry-all graph attention is sel-independent by design and
442442
differs from the dense lower by up to ~1e-4 (see the Notes of
443443
:meth:`call_graph`).
444444
"""
445+
# compressed descriptors have no graph kernel (geo/tebd tabulation is
446+
# dense-only); keep them on the legacy dense path.
447+
if self.compress:
448+
return False
449+
# exclude_types stays dense (graph exclusion is owned elsewhere); strip is
450+
# now graph-eligible (per-edge factorized embedding, no neighbor coupling).
445451
return (
446-
self.se_atten.tebd_input_mode == "concat"
452+
self.se_atten.tebd_input_mode in ("concat", "strip")
447453
and not self.se_atten.exclude_types
448454
)
449455

@@ -575,7 +581,7 @@ def call(
575581
nall = xp.reshape(coord_ext, (nlist.shape[0], -1)).shape[1] // 3
576582
# graph-eligible configs route through the graph-native adapter (decision
577583
# #14: graph = single math source, dense call = thin adapter). Ineligible
578-
# configs (attention, strip tebd, exclude_types) and the ghost case with
584+
# configs (exclude_types, compressed descriptors) and the ghost case with
579585
# no mapping fall back to the legacy dense body. The graph needs `mapping`
580586
# to fold ghosts to local owners; without it only nall == nloc is valid.
581587
if self.uses_graph_lower() and (mapping is not None or nall == nloc):
@@ -680,8 +686,8 @@ def _call_dense(
680686
atype_ext: Array,
681687
nlist: Array,
682688
) -> Array:
683-
"""Legacy dense descriptor body (the ineligible ``call`` path: attention,
684-
strip tebd, exclude_types, or the no-mapping ghost case).
689+
"""Legacy dense descriptor body (the ineligible ``call`` path:
690+
compressed descriptors, exclude_types, or the no-mapping ghost case).
685691
686692
Parameters
687693
----------
@@ -1749,17 +1755,17 @@ def call_graph(
17491755
Notes
17501756
-----
17511757
Known limitations:
1752-
- ``tebd_input_mode == "concat"`` only (strip mode lands later);
1758+
- ``tebd_input_mode`` in {"concat", "strip"}; compressed descriptors stay dense;
17531759
- ``exclude_types`` is not yet supported and raises (lands in a later PR).
17541760
"""
17551761
from deepmd.dpmodel.utils.neighbor_graph import (
17561762
edge_env_mat,
17571763
segment_sum,
17581764
)
17591765

1760-
if self.tebd_input_mode not in ["concat"]:
1766+
if self.tebd_input_mode not in ["concat", "strip"]:
17611767
raise NotImplementedError(
1762-
"graph path supports tebd_input_mode='concat' only (NeighborGraph PR-A)"
1768+
f"graph path does not support tebd_input_mode={self.tebd_input_mode!r}"
17631769
)
17641770
if self.exclude_types:
17651771
raise NotImplementedError(
@@ -1794,20 +1800,25 @@ def call_graph(
17941800
) # (E, 4), (E, 1) sw zeroed on padding
17951801
# radial channel
17961802
ss = rr[:, 0:1] # (E, 1)
1797-
# neighbor / center type embeddings (concat mode); ghost type == owner type
1798-
# so gathering by the LOCAL owner (src) reproduces the dense neighbor tebd.
1799-
# NB: do NOT wrap in ``xp.asarray(..., device=dev)`` -- that DETACHES under
1800-
# torch and severs the type-embedding weight gradient (the tebd net would
1801-
# never train); type_embedding already lives on the model device.
1802-
tebd = type_embedding
1803-
atype_embd_nlist = xp.take(tebd, nei_type, axis=0) # (E, tebd_dim)
1804-
if not self.type_one_side:
1805-
atype_embd_nnei = xp.take(tebd, center_type, axis=0) # (E, tebd_dim)
1806-
ss = xp.concat([ss, atype_embd_nlist, atype_embd_nnei], axis=-1)
1807-
else:
1808-
ss = xp.concat([ss, atype_embd_nlist], axis=-1)
1809-
# embedding net (same weights as the dense path); applies on the last axis
1810-
gg = self.embeddings[0].call(ss) # (E, ng)
1803+
if self.tebd_input_mode == "concat":
1804+
# neighbor / center type embeddings; ghost type == owner type so
1805+
# gathering by the LOCAL owner (src) reproduces the dense neighbor tebd.
1806+
# NB: do NOT wrap in ``xp.asarray(..., device=dev)`` -- that DETACHES
1807+
# under torch and severs the type-embedding weight gradient (the tebd
1808+
# net would never train); type_embedding already lives on the device.
1809+
tebd = type_embedding
1810+
atype_embd_nlist = xp.take(tebd, nei_type, axis=0) # (E, tebd_dim)
1811+
if not self.type_one_side:
1812+
atype_embd_nnei = xp.take(tebd, center_type, axis=0) # (E, tebd_dim)
1813+
ss = xp.concat([ss, atype_embd_nlist, atype_embd_nnei], axis=-1)
1814+
else:
1815+
ss = xp.concat([ss, atype_embd_nlist], axis=-1)
1816+
# embedding net (same weights as the dense path); applies on last axis
1817+
gg = self.embeddings[0].call(ss) # (E, ng)
1818+
else: # strip: factorized gg_s*gg_t + gg_s (per-edge; no neighbor coupling)
1819+
gg = self._graph_edge_gg_strip(
1820+
ss, center_type, nei_type, type_embedding, sw_e
1821+
)
18111822
# transformer attention over each center's edges — mirrors the dense
18121823
# self.dpa1_attention(gg, nlist_mask, input_r, sw), which also runs on
18131824
# the UNMASKED gg (padding rows are neutralized afterwards).
@@ -1835,6 +1846,78 @@ def call_graph(
18351846
rot_mat = gr[:, :, 1:]
18361847
return grrg, rot_mat
18371848

1849+
def _graph_edge_gg_strip(
1850+
self,
1851+
ss: Array,
1852+
center_type: Array,
1853+
nei_type: Array,
1854+
type_embedding: Array,
1855+
sw_e: Array,
1856+
) -> Array:
1857+
"""Per-edge stripped-tebd embedding, op-for-op vs the dense strip branch.
1858+
1859+
Mirrors the ``tebd_input_mode == "strip"`` block of :meth:`call`: the
1860+
geometric net runs on the radial channel only (``gg_s``), the stripped
1861+
type-embedding net produces a per-type(-pair) factor (``gg_t``,
1862+
optionally switch-smoothed), and the two combine as
1863+
``gg_s * gg_t + gg_s``. The compression branches (geo/tebd) are NOT
1864+
reached on the graph route: :meth:`DescrptDPA1.uses_graph_lower`
1865+
excludes compressed descriptors, so this kernel assumes no compression.
1866+
1867+
Parameters
1868+
----------
1869+
ss
1870+
(E, 1) per-edge radial channel (``rr[:, 0:1]``).
1871+
center_type
1872+
(E,) center (dst) LOCAL atom type of each edge.
1873+
nei_type
1874+
(E,) neighbor (src) LOCAL atom type of each edge.
1875+
type_embedding
1876+
(ntypes_with_padding, tebd_dim) type-embedding table.
1877+
sw_e
1878+
(E, 1) smooth switch, zeroed on padding edges.
1879+
1880+
Returns
1881+
-------
1882+
gg
1883+
(E, ng) per-edge embedding feeding the attention / segment_sum.
1884+
"""
1885+
assert self.embeddings_strip is not None
1886+
xp = array_api_compat.array_namespace(ss)
1887+
nt = self.tebd_dim
1888+
ntypes_with_padding = type_embedding.shape[0]
1889+
# geometric net on the radial channel only (dense: gg_s = cal_g(ss_scalar))
1890+
gg_s = self.embeddings[0].call(ss) # (E, ng)
1891+
if self.type_one_side:
1892+
# one-side strip table indexed by NEIGHBOR type only
1893+
tt_full = self.cal_g_strip(type_embedding, 0) # (ntypes_pad, ng)
1894+
gg_t = xp.take(tt_full, nei_type, axis=0) # (E, ng)
1895+
else:
1896+
# two-side type-pair table; row = center * ntypes_pad + nei
1897+
# (dense builds the same (ntypes_pad**2, 2*nt) table, nei-fastest).
1898+
type_embedding_nei = xp.tile(
1899+
xp.reshape(type_embedding, (1, ntypes_with_padding, nt)),
1900+
(ntypes_with_padding, 1, 1),
1901+
)
1902+
type_embedding_center = xp.tile(
1903+
xp.reshape(type_embedding, (ntypes_with_padding, 1, nt)),
1904+
(1, ntypes_with_padding, 1),
1905+
)
1906+
two_side_type_embedding = xp.reshape(
1907+
xp.concat([type_embedding_nei, type_embedding_center], axis=-1),
1908+
(-1, nt * 2),
1909+
)
1910+
tt_full = self.cal_g_strip(
1911+
two_side_type_embedding, 0
1912+
) # (ntypes_pad**2, ng)
1913+
# int64 for torch take (take_along/take requires Long indices)
1914+
idx = xp.astype(center_type * ntypes_with_padding + nei_type, xp.int64)
1915+
gg_t = xp.take(tt_full, idx, axis=0) # (E, ng)
1916+
if self.smooth:
1917+
# dense: gg_t = gg_t * sw (per-neighbor); sw_e is (E, 1), zeroed on padding
1918+
gg_t = gg_t * sw_e
1919+
return gg_s * gg_t + gg_s
1920+
18381921
def _graph_attention(
18391922
self,
18401923
gg: Array,

source/tests/common/dpmodel/test_dpa1_call_graph_block.py

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,3 +114,93 @@ def test_exclude_types_raises(self) -> None:
114114
dd.se_atten.call_graph(
115115
ng, self.atype.reshape(-1), type_embedding=dd.type_embedding.call()
116116
)
117+
118+
119+
class TestDpa1BlockCallGraphStrip:
120+
"""Bit-exact parity between the graph-native ``call_graph`` and the dense
121+
``call`` for ``tebd_input_mode='strip'``.
122+
123+
Strip mode factorizes the per-neighbor feature as ``gg = gg_s*gg_t + gg_s``
124+
(radial embedding times type-pair strip embedding); it has no neighbor-axis
125+
coupling, so the graph translation is edge-for-edge and must be bit-exact
126+
with the dense block on the SAME neighbor list.
127+
"""
128+
129+
def setup_method(self) -> None:
130+
rng = np.random.default_rng(3)
131+
self.nloc = 4
132+
self.coord = rng.normal(size=(1, self.nloc, 3)) * 1.5
133+
self.atype = np.array([[0, 1, 0, 1]], dtype=np.int64)
134+
135+
def _make(self, type_one_side: bool, smooth: bool, attn_layer: int) -> DescrptDPA1:
136+
return DescrptDPA1(
137+
rcut=4.0,
138+
rcut_smth=0.5,
139+
sel=[20], # non-binding sel: carry-all graph == dense on real neighbors
140+
ntypes=2,
141+
attn_layer=attn_layer,
142+
axis_neuron=2,
143+
neuron=[6, 12],
144+
tebd_input_mode="strip",
145+
type_one_side=type_one_side,
146+
smooth_type_embedding=smooth,
147+
)
148+
149+
def _assert_parity(self, dd: DescrptDPA1, compact: bool) -> None:
150+
(
151+
ext_coord,
152+
ext_atype,
153+
mapping,
154+
nlist,
155+
) = extend_input_and_build_neighbor_list(
156+
self.coord,
157+
self.atype,
158+
dd.get_rcut(),
159+
dd.get_sel(),
160+
mixed_types=dd.mixed_types(),
161+
box=None,
162+
)
163+
tebd = dd.type_embedding.call()
164+
nf, nall = ext_atype.shape
165+
atype_embd_ext = np.reshape(
166+
np.take(tebd, np.reshape(ext_atype, (-1,)), axis=0),
167+
(nf, nall, dd.tebd_dim),
168+
)
169+
dense_g, *_ = dd.se_atten.call(
170+
nlist,
171+
ext_coord,
172+
ext_atype,
173+
atype_embd_ext=atype_embd_ext,
174+
mapping=None,
175+
type_embedding=tebd,
176+
)
177+
ng = from_dense_quartet(ext_coord, nlist, mapping, compact=compact)
178+
graph_g, _rot_mat = dd.se_atten.call_graph(
179+
ng,
180+
np.reshape(ext_atype, (-1,)),
181+
type_embedding=tebd,
182+
)
183+
assert not np.any(np.isnan(graph_g))
184+
np.testing.assert_allclose(
185+
graph_g.reshape(dense_g.shape),
186+
dense_g,
187+
rtol=1e-12,
188+
atol=1e-12,
189+
)
190+
191+
@pytest.mark.parametrize(
192+
"type_one_side", [False, True]
193+
) # two-side vs one-side strip table
194+
@pytest.mark.parametrize("smooth", [False, True]) # gg_t switch-smoothing branch
195+
def test_strip_attn0_equals_dense(self, type_one_side, smooth) -> None:
196+
"""attn_layer=0: no attention, so strip parity is bit-exact for both smooth values."""
197+
dd = self._make(type_one_side, smooth, attn_layer=0)
198+
self._assert_parity(dd, compact=True)
199+
200+
@pytest.mark.parametrize(
201+
"type_one_side", [False, True]
202+
) # two-side vs one-side strip table
203+
def test_strip_attn2_equals_dense(self, type_one_side) -> None:
204+
"""attn_layer=2, smooth=False: bit-exact (avoids by-design smooth softmax divergence)."""
205+
dd = self._make(type_one_side, smooth=False, attn_layer=2)
206+
self._assert_parity(dd, compact=False)

source/tests/common/dpmodel/test_dpa1_call_graph_descriptor.py

Lines changed: 89 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -99,14 +99,13 @@ def test_descriptor_graph_equals_dense_full_tuple(self, sel) -> None:
9999
@pytest.mark.parametrize(
100100
"kwargs",
101101
[
102-
{"tebd_input_mode": "strip"}, # strip tebd: graph unsupported -> dense
103102
{"exclude_types": [(0, 1)]}, # type exclusion: graph unsupported -> dense
104103
],
105104
)
106105
def test_ineligible_config_falls_back_to_dense(self, kwargs) -> None:
107-
"""attn_layer=0 configs the graph can't handle (strip tebd, exclude_types)
108-
must report uses_graph_lower()=False and run the dense body without
109-
raising (regression: Task-3 routing previously raised NotImplementedError).
106+
"""attn_layer=0 configs the graph can't handle (exclude_types) must report
107+
uses_graph_lower()=False and run the dense body without raising (strip is
108+
now graph-eligible; its routing + parity is covered by TestDpa1StripRouting).
110109
"""
111110
dd = DescrptDPA1(
112111
rcut=4.0, rcut_smth=0.5, sel=[30], ntypes=2, attn_layer=0, **kwargs
@@ -200,3 +199,89 @@ def test_call_graph_returns_flat_node_axis(self) -> None:
200199
n = atype_local.shape[0]
201200
assert grrg.shape[0] == n and grrg.ndim == 2
202201
assert rot_mat.shape[0] == n and rot_mat.ndim == 3
202+
203+
204+
class TestDpa1StripRouting:
205+
"""Strip is now graph-eligible: ``uses_graph_lower()`` admits it, ``call``
206+
routes through the graph adapter, and the adapter (``static_nnei``) is
207+
bit-exact with the legacy ``_call_dense`` for EVERY strip config -- including
208+
``smooth_type_embedding=True`` at ``attn_layer>0`` (the adapter reproduces
209+
the dense phantom-neighbor terms, unlike the direct carry-all ``call_graph``).
210+
"""
211+
212+
def setup_method(self) -> None:
213+
rng = np.random.default_rng(7)
214+
self.nloc = 4
215+
self.coord = rng.normal(size=(1, self.nloc, 3)) * 1.5
216+
self.atype = np.array([[0, 1, 0, 1]], dtype=np.int64)
217+
218+
def _make(self, type_one_side: bool, smooth: bool, attn_layer: int) -> DescrptDPA1:
219+
return DescrptDPA1(
220+
rcut=4.0,
221+
rcut_smth=0.5,
222+
sel=[20], # non-binding
223+
ntypes=2,
224+
attn_layer=attn_layer,
225+
axis_neuron=2,
226+
neuron=[6, 12],
227+
tebd_input_mode="strip",
228+
type_one_side=type_one_side,
229+
smooth_type_embedding=smooth,
230+
resnet_dt=False,
231+
)
232+
233+
def test_uses_graph_lower_strip_gate(self) -> None:
234+
"""The gate admits non-compressed strip; excludes compressed and exclude_types."""
235+
dd = self._make(type_one_side=False, smooth=True, attn_layer=2)
236+
assert dd.uses_graph_lower() is True # strip is now graph-eligible
237+
# negative contract: compression keeps the descriptor on the dense path
238+
dd.compress = True
239+
assert dd.uses_graph_lower() is False
240+
dd.compress = False
241+
# negative contract: exclude_types still forces dense (separate feature,
242+
# owned by the pair-exclude PR; strip does NOT change its eligibility)
243+
dd_excl = DescrptDPA1(
244+
rcut=4.0,
245+
rcut_smth=0.5,
246+
sel=[20],
247+
ntypes=2,
248+
attn_layer=2,
249+
tebd_input_mode="strip",
250+
exclude_types=[(0, 1)],
251+
resnet_dt=False,
252+
)
253+
assert dd_excl.uses_graph_lower() is False
254+
255+
@pytest.mark.parametrize("type_one_side", [False, True]) # strip table branch
256+
@pytest.mark.parametrize(
257+
"smooth", [False, True]
258+
) # switch-smoothing + smooth attention
259+
@pytest.mark.parametrize("attn_layer", [0, 2]) # no-attn and multi-layer attention
260+
def test_call_strip_graph_equals_dense(
261+
self, type_one_side, smooth, attn_layer
262+
) -> None:
263+
"""The routed ``call`` (graph adapter) is bit-exact with ``_call_dense``."""
264+
dd = self._make(type_one_side, smooth, attn_layer)
265+
assert dd.uses_graph_lower() is True # precondition: call routes to graph
266+
(
267+
ext_coord,
268+
ext_atype,
269+
mapping,
270+
nlist,
271+
) = extend_input_and_build_neighbor_list(
272+
self.coord,
273+
self.atype,
274+
dd.get_rcut(),
275+
dd.get_sel(),
276+
mixed_types=dd.mixed_types(),
277+
box=None,
278+
)
279+
routed = dd.call(ext_coord, ext_atype, nlist, mapping=mapping)
280+
dense = dd._call_dense(ext_coord, ext_atype, nlist)
281+
assert len(routed) == len(dense)
282+
for r, d in zip(routed, dense, strict=True):
283+
if r is None:
284+
assert d is None
285+
continue
286+
assert not np.any(np.isnan(r))
287+
np.testing.assert_allclose(r, d, rtol=1e-12, atol=1e-12)

0 commit comments

Comments
 (0)