merge from origin by jameswind · Pull Request #2 · jameswind/deepmd-kit

jameswind · 2020-06-27T14:49:14Z

No description provided.

github-advanced-security

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

updates: - [github.com/astral-sh/ruff-pre-commit: v0.15.4 → v0.15.5](astral-sh/ruff-pre-commit@v0.15.4...v0.15.5)  Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

#5277) ## Summary This PR integrates pretrained model support directly into `deepmd-kit` under `deepmd/pretrained`, while keeping `DeepPot` usage unchanged. ### Added - New command: - `dp pretrained download <MODEL>` - New module folder: - `deepmd/pretrained/` - includes `registry.py`, `download.py`, `backend.py`, `entrypoints.py` - Built-in model registry (currently): - `DPA-3.2-5M` - `DPA-3.1-3M` - Multi-source download strategy: - parallel probe over candidate sources - rank by response latency - fastest-first with automatic fallback on timeout/failure/checksum mismatch - SHA256 verification + atomic `.part` writes - `.pretrained` backend alias support via `deepmd/backend/pretrained.py` - allows `DeepPot("DPA-3.2-5M.pretrained")` while keeping existing DeepPot API unchanged - deep-eval adapter is lazy-loaded to avoid circular import issues ### CLI wiring - Added `pretrained` parser/subparser in `deepmd/main.py` - Added dispatch in `deepmd/entrypoints/main.py` ### Tests - `source/tests/common/test_pretrained_parser.py` - `source/tests/common/test_pretrained_download.py` - `source/tests/common/test_pretrained_backend.py` ### Formatting / lint - Ran `uvx prek run --all-files` and committed auto-format updates. Authored by OpenClaw (model: custom-chat-jinzhezeng-group/gpt-5.3-codex)  ## Summary by CodeRabbit * **New Features** * New "pretrained" CLI group with `pretrained download` (optional --cache-dir) to fetch pretrained models. * Built-in pretrained model registry (includes DPA-3.2-5M and DPA-3.1-3M). * Added a pretrained backend scaffold and a lazy adapter so `.pretrained` aliases work transparently with existing evaluation flow (some backend hooks intentionally unsupported). * **Downloads & Caching** * HTTPS-only downloads with SHA256 verification, multi-source fallbacks, parallel probing, atomic writes, and centralized caching (~/.cache/deepmd/pretrained/models). * **Tests** * Added tests for backend detection, alias parsing, download/resolution behavior, URL ranking, and CLI parsing.  --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Problem - Paddle nightly wheels are fetched from https://www.paddlepaddle.org.cn/... and the site’s TLS certificate outage breaks CI installs. Change - Add `--trusted-host www.paddlepaddle.org.cn` and `--trusted-host paddlepaddle.org.cn` to the Paddle install commands in GitHub Actions workflows. Notes - This is a temporary workaround to keep CI green until Paddle fixes their cert. Authored by OpenClaw (model: gpt-5.2)  ## Summary by CodeRabbit * **Chores** * Updated build workflow configurations to include additional package host verification options during dependency installation.

Bumps [tensorflow-cpu](https://github.com/tensorflow/tensorflow) from 2.20.0 to 2.21.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tensorflow/tensorflow/releases">tensorflow-cpu's releases</a>.</em></p> <blockquote> <h2>TensorFlow 2.21.0</h2> <h1>Release 2.21.0</h1> <h2>TensorFlow</h2> <h3>Breaking Changes</h3> <ul> <li>Support for Python 3.9 has been removed starting with TF 2.21.</li> <li>The TensorBoard (TB) dependency has been removed starting with TF 2.21.</li> </ul> <h3>Major Features and Improvements</h3> <ul> <li> <p><code>tf.lite</code></p> <ul> <li>Adds int8 and int16x8 support for SQRT operator.</li> <li>Adds int16x8 support for EQUAL and NOT_EQUAL operators.</li> <li>Adds support for int2 type.</li> <li>Adds support for int2/int4 in tfl.cast .</li> <li>Adds support for SRQ int2 in tfl.fully_connected.</li> <li>Adds support for int4 in tfl.slice.</li> <li>Adds support for uint4 type.</li> </ul> </li> <li> <p><code>tf.image</code></p> <ul> <li>Adds JPEG XL support in decode_image.</li> </ul> </li> </ul> <h3>Bug Fixes and Other Changes</h3> <ul> <li><code>tf.data</code> <ul> <li>Adds <code>NoneTensorSpec</code> to the public API so that <code>None</code>s in <code>element_spec</code> can be identified via <code>isinstance(..., tf.NoneTensorSpec)</code>.</li> </ul> </li> </ul> <h2>Thanks to our Contributors</h2> <p>This release contains contributions from many people at Google, as well as:</p> <p>Aaraviitkgp, Abhijeet, Abhinav Gunjal, Abhishek, Adam Paszke, Aditya Gupta, Aditya Jha, Aditya Sharma, Adrian Kuegel, Aiden Grossman, Akarsh, Akhil Goel, Alan Kelly, Aleksa Arsic, Aleksei, Aleksei Nurmukhametov, Alex, Alexander Belyaev, Alexander Grund, Alexander Lyashuk, Alexander Shaposhnikov, Alex Pivovarov, Aliia Khasanova, Alina Sbirlea, Allan Renucci, Amelia Thurdekoos, Amit Sabne, Andrei Ivanov, Andrew Dame, Andrey Portnoy, Anish Nair, Anlun Xu, Antonio Sanchez, anuj chincholikar, Anuj Chincholikar, Aravindh Balaji, aravindhbalaji1985, Arian Arfaian, Armin Felder, Artem Belevich, Ashish Rao, Ashitesh Singh, A. Unique TensorFlower, Bart Chrzaszcz, benediktjohannes, Benjamin Chetioui, Benjamin Kramer, Berkin Ilbeyi, Bhatu, Bhavani Subramanian, Bhupendra Dubey, Bill Varcho, Bixia Zheng, Blake Hechtman, Bodhi Silberling, BruceXinXin, Bryan Massoth, Buddh Prakash, Byungchul Kim, Ce Zheng, Changhui Lin, Chao, Charles Alaras, Chase Riley Roberts, Chenhao Jiang, Chris Ashton, Chris Jones, Chris Kennelly, Christian Sigg, Chuan He, Chunlei Niu, Chun-nien Chan, Chunyu Jin, Clive Verghese, Cong Liu, Corentin Kerisit, Daniel Chen, Daniel Kuts, Daniel Ng, Daniel Sosa, Daniel Suo, Danila Malyutin, David Dunleavy, David Majnemer, David Pizzuto, Deepika Rajani, deeptanshusekhri, dependabot[bot], Deqiang Chen, Derek Murray, Dillon Sharlet, Dimitar (Mitko) Asenov, Dimitris Vardoulakis, Dirk Hornung, DottsGit, Dragan Mladjenovic, Eetu Sjöblom, Elen Kalda, Emilio Cota, Emily Fertig, Eugene Zhulenev, Eusebio Durán Montaña, Evan Brown, Ezekiel Calubaquib, Faijul Amin, Felix Wang, Fengwu Yao, Fergus Henderson, Frederic Rechtenstein, Frederik Gossen, Gabriel Gerlero, Gagan Nagaraj, gaikwadrahul8, garry00107, gaurides, George Pawelczak, Georg Stefan Schmid, gns, Goran Flegar, Graham, Grant Jensen, Greg Olechwierowicz, Gregory Pataky, Grzegorz Gawryał, Gunhyun Park, guozhong.zhuang, Haibo Huang, Hana Joo, Hariprasad Ravishankar, Harsha H S, Harshit Monish, Henning Becker, Hittanshu, Hoeseong (Hayden) Kim, Hugo Mano, Hyeontaek Lim, Ibrahim Umit Akgun, ILCSFNO, Ilia Sergachev, Ilya Tikhonovskiy, Iman Hosseini, Ionel Gog, Isha Arkatkar, isharif168, Ivo Ristovski List, Jacques Pienaar, Jae H. Yoo, Jaeyoon Jung, Jake Harmon, James Hilliard, jameslovespancakes, James Spooner, Jane Liu, Jaroslav Sevcik, Jeff Parker, Jeffrey A. Dean, Jeremy Meredith, Jialei Chen, Jian Cai, Jian Li, Jie Luo, Jim Lin, Jing Pu, Jinliang Wei, Jiya Zhang, Joel Wee, Johannes Buchner, Johannes Reifferscheid, Johnny, Jorge Gorbe Moya, Joshua Lang, Joshua Wang, Joss Briody, jparkerh, Juanli Shen, Juhyun Lee, Jun Jiang, Junwhan Ahn, Kadir Barut, Kanglan Tang, Kanish Anand, Kanvi Khanna, Karlo Basioli, Ken Franko, Kevin Chen, Kevin Gleason, Kingston Mandisodza, Koki Ibukuro, Kostiantyn Liepieshov, Krishna Haridasan, Krishna Somani, Krzysztof Kosiński, Kuy Mainwaring, lambert, Larry Lansing, Lin Chai, Lord ε Rebel, Luke Baumann, Luke Hutton, madhavmadupu, Majid Dadashi, Mani Ananth, Manjunath Gaonkar, Marcello Maggioni, Marcin Radomski, Maria Lyubimtseva, Marissa Ikonomidis, Mark Daoust, Mason Chang, Matej Aleksandrov, Mateusz Sokół, Matthias Guenther, Matthias Kramm, Matt Hurd, Matt Kreileder, Maxime France-Pillois, Maxim Ermilov, Mehrdad Khani, Melissa Weber Mendonça, MERT-CKR, Michael Goldfarb, Michael Green, Michael Kuperstein, Michael Voznesensky, Michael Whittaker, Mihai Maruseac, Mikhail Goncharov, Ming-Xu Huang, Mircea Trofin, Misha Gutman, misterBart, mmakevic-amd, Mohamed AbdElmoneim, Mohamed Amine Zghal, Mohammadreza Heydary, Mohammed Anany, mraunak, Mudit Gokhale, Nayana Thorat, Nevi, nhatle, Nhat Le, Nihar0071, Nikhil, Nikita Putikhin, Niklas Vangerow, Nitin Srinivasan, Oleg Shyshkov, Olli Lupton, Om Thakkar, Pankaj Kanwar, Parker Schuh, Paul Ganssle, Pauline Sho, Pavithra Eswaramoorthy, Pedro Gonnet, pemeliya, Penporn Koanantakool, Perry Gibson, Peter Buchlovsky, Peter Gavin, Peter Hawkins, Pham Binh, Phani Paladugula, Philipp Hack, Praneeth Mandala, Praveen Batra, psinfinity, Qingwei Zhang, Quentin Khan, Quoc Truong, QZero, Rachel Han, Raffi Khatchadourian, Ram Rachum, RasheedAli-Shaik, Raviteja Gorijala, Reed Wanderman-Milne, Reilly Grant, Renjie Wu, Richard Levasseur, Robert David, Ryan M. Lefever, Sachin M, Sagun Bajra, Sai Ganesh Muthuraman, Saksham Singh Rathore, Sannidhya Chauhan, Sayan Saha, Sean Talts, Seher Ellis, Sergei Lebedev, Sergey Kozub, Sevin Fide Varoglu, Shahriar Rouf, Shanbin Ke, Shaogang Wang, Sharad Vikram, Shawn Lu, Siddhartha Menon, Siqiao Wu, skill, Smit Hinsu, snadampal, Sohaib Iftikhar, Soowon Jeong, spiao, Srijan Upadhyay, stevemcgregory, Subham Soni, Subhankar Shah, Swachhand Lokhande, Tai Ly, TensorFlower Gardener, Terry Heo, Terry Sun, Terry Tao, Theotime Combes, Thomas Joerg, Thomas Köppe, Tiago Quelhas, TJ Xu, Toli Yevtushenko, Tomás Longeri, Tom Hennigan, Tommy Chiang, Tom Natan, Tongfei Guo, Tori Baker, Uwe L. Korn, Vadym Matsishevskyi, Vamsi Manchala, Venkat6871, Victor Stone, Ville Vesilehto, Vitalii Dziuba, Vladimir Belitskiy, Vlad Sytchenko, Volodymyr Kysenko, Wai Hon Law, wan3x, Weiyi Wang, Will Froom, William S. Moses, wondertx, Xuefei Jiang, Yang Chen, Yash Katariya, Yasir Ashfaq, yasiribmcon, Yeou Chiou, Yicheng Luo, Yi Kong, Yimei Sun, Yin Zhang, Yuchen Yao, Yue Sheng, Yulia Baturina, Yunjie Xu, Yunlong Liu, Yun Peng, Yurii Topin, Zac Cranko, Zac Mustin, Zenong Zhang, Zeyu Wang, Zhanyong Wan, Zixuan Jiang, Ziyin Huang, Zviki Nozadze</p> <h2>TensorFlow 2.21.0-rc1</h2> <h1>Release 2.21.0</h1> <h2>TensorFlow</h2> <h3>Breaking Changes</h3> <ul> <li>Support for Python 3.9 has been removed starting with TF 2.21.</li> </ul> <h3>Major Features and Improvements</h3> <ul> <li><code>tf.lite</code> <ul> <li>Adds int8 and int16x8 support for SQRT operator.</li> <li>Adds int16x8 support for EQUAL and NOT_EQUAL operators.</li> </ul> </li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md">tensorflow-cpu's changelog</a>.</em></p> <blockquote> <h1>Release 2.21.0</h1> <h2>TensorFlow</h2> <h3>Breaking Changes</h3> <ul> <li>Support for Python 3.9 has been removed starting with TF 2.21.</li> <li>The TensorBoard (TB) dependency has been removed starting with TF 2.21.</li> </ul> <h3>Major Features and Improvements</h3> <ul> <li> <p><code>tf.lite</code></p> <ul> <li>Adds int8 and int16x8 support for SQRT operator.</li> <li>Adds int16x8 support for EQUAL and NOT_EQUAL operators.</li> <li>Adds support for int2 type.</li> <li>Adds support for int2/int4 in tfl.cast .</li> <li>Adds support for SRQ int2 in tfl.fully_connected.</li> <li>Adds support for int4 in tfl.slice.</li> <li>Adds support for uint4 type.</li> </ul> </li> <li> <p><code>tf.image</code></p> <ul> <li>Adds JPEG XL support in decode_image.</li> </ul> </li> </ul> <h3>Bug Fixes and Other Changes</h3> <ul> <li><code>tf.data</code> <ul> <li>Adds <code>NoneTensorSpec</code> to the public API so that <code>None</code>s in <code>element_spec</code> can be identified via <code>isinstance(..., tf.NoneTensorSpec)</code>.</li> </ul> </li> </ul> <h2>Thanks to our Contributors</h2> <p>This release contains contributions from many people at Google, as well as:</p> <p>Aaraviitkgp, Abhijeet, Abhinav Gunjal, Abhishek, Adam Paszke, Aditya Gupta, Aditya Jha, Aditya Sharma, Adrian Kuegel, Aiden Grossman, Akarsh, Akhil Goel, Alan Kelly, Aleksa Arsic, Aleksei, Aleksei Nurmukhametov, Alex, Alexander Belyaev, Alexander Grund, Alexander Lyashuk, Alexander Shaposhnikov, Alex Pivovarov, Aliia Khasanova, Alina Sbirlea, Allan Renucci, Amelia Thurdekoos, Amit Sabne, Andrei Ivanov, Andrew Dame, Andrey Portnoy, Anish Nair, Anlun Xu, Antonio Sanchez, anuj chincholikar, Anuj Chincholikar, Aravindh Balaji, aravindhbalaji1985, Arian Arfaian, Armin Felder, Artem Belevich, Ashish Rao, Ashitesh Singh, A. Unique TensorFlower, Bart Chrzaszcz, benediktjohannes, Benjamin Chetioui, Benjamin Kramer, Berkin Ilbeyi, Bhatu, Bhavani Subramanian, Bhupendra Dubey, Bill Varcho, Bixia Zheng, Blake Hechtman, Bodhi Silberling, BruceXinXin, Bryan Massoth, Buddh Prakash, Byungchul Kim, Ce Zheng, Changhui Lin, Chao, Charles Alaras, Chase Riley Roberts, Chenhao Jiang, Chris Ashton, Chris Jones, Chris Kennelly, Christian Sigg, Chuan He, Chunlei Niu, Chun-nien Chan, Chunyu Jin, Clive Verghese, Cong Liu, Corentin Kerisit, Daniel Chen, Daniel Kuts, Daniel Ng, Daniel Sosa, Daniel Suo, Danila Malyutin, David Dunleavy, David Majnemer, David Pizzuto, Deepika Rajani, deeptanshusekhri, dependabot[bot], Deqiang Chen, Derek Murray, Dillon Sharlet, Dimitar (Mitko) Asenov, Dimitris Vardoulakis, Dirk Hornung, DottsGit, Dragan Mladjenovic, Eetu Sjöblom, Elen Kalda, Emilio Cota, Emily Fertig, Eugene Zhulenev, Eusebio Durán Montaña, Evan Brown, Ezekiel Calubaquib, Faijul Amin, Felix Wang, Fengwu Yao, Fergus Henderson, Frederic Rechtenstein, Frederik Gossen, Gabriel Gerlero, Gagan Nagaraj, gaikwadrahul8, garry00107, gaurides, George Pawelczak, Georg Stefan Schmid, gns, Goran Flegar, Graham, Grant Jensen, Greg Olechwierowicz, Gregory Pataky, Grzegorz Gawryał, Gunhyun Park, guozhong.zhuang, Haibo Huang, Hana Joo, Hariprasad Ravishankar, Harsha H S, Harshit Monish, Henning Becker, Hittanshu, Hoeseong (Hayden) Kim, Hugo Mano, Hyeontaek Lim, Ibrahim Umit Akgun, ILCSFNO, Ilia Sergachev, Ilya Tikhonovskiy, Iman Hosseini, Ionel Gog, Isha Arkatkar, isharif168, Ivo Ristovski List, Jacques Pienaar, Jae H. Yoo, Jaeyoon Jung, Jake Harmon, James Hilliard, jameslovespancakes, James Spooner, Jane Liu, Jaroslav Sevcik, Jeff Parker, Jeffrey A. Dean, Jeremy Meredith, Jialei Chen, Jian Cai, Jian Li, Jie Luo, Jim Lin, Jing Pu, Jinliang Wei, Jiya Zhang, Joel Wee, Johannes Buchner, Johannes Reifferscheid, Johnny, Jorge Gorbe Moya, Joshua Lang, Joshua Wang, Joss Briody, jparkerh, Juanli Shen, Juhyun Lee, Jun Jiang, Junwhan Ahn, Kadir Barut, Kanglan Tang, Kanish Anand, Kanvi Khanna, Karlo Basioli, Ken Franko, Kevin Chen, Kevin Gleason, Kingston Mandisodza, Koki Ibukuro, Kostiantyn Liepieshov, Krishna Haridasan, Krishna Somani, Krzysztof Kosiński, Kuy Mainwaring, lambert, Larry Lansing, Lin Chai, Lord ε Rebel, Luke Baumann, Luke Hutton, madhavmadupu, Majid Dadashi, Mani Ananth, Manjunath Gaonkar, Marcello Maggioni, Marcin Radomski, Maria Lyubimtseva, Marissa Ikonomidis, Mark Daoust, Mason Chang, Matej Aleksandrov, Mateusz Sokół, Matthias Guenther, Matthias Kramm, Matt Hurd, Matt Kreileder, Maxime France-Pillois, Maxim Ermilov, Mehrdad Khani, Melissa Weber Mendonça, MERT-CKR, Michael Goldfarb, Michael Green, Michael Kuperstein, Michael Voznesensky, Michael Whittaker, Mihai Maruseac, Mikhail Goncharov, Ming-Xu Huang, Mircea Trofin, Misha Gutman, misterBart, mmakevic-amd, Mohamed AbdElmoneim, Mohamed Amine Zghal, Mohammadreza Heydary, Mohammed Anany, mraunak, Mudit Gokhale, Nayana Thorat, Nevi, nhatle, Nhat Le, Nihar0071, Nikhil, Nikita Putikhin, Niklas Vangerow, Nitin Srinivasan, Oleg Shyshkov, Olli Lupton, Om Thakkar, Pankaj Kanwar, Parker Schuh, Paul Ganssle, Pauline Sho, Pavithra Eswaramoorthy, Pedro Gonnet, pemeliya, Penporn Koanantakool, Perry Gibson, Peter Buchlovsky, Peter Gavin, Peter Hawkins, Pham Binh, Phani Paladugula, Philipp Hack, Praneeth Mandala, Praveen Batra, psinfinity, Qingwei Zhang, Quentin Khan, Quoc Truong, QZero, Rachel Han, Raffi Khatchadourian, Ram Rachum, RasheedAli-Shaik, Raviteja Gorijala, Reed Wanderman-Milne, Reilly Grant, Renjie Wu, Richard Levasseur, Robert David, Ryan M. Lefever, Sachin M, Sagun Bajra, Sai Ganesh Muthuraman, Saksham Singh Rathore, Sannidhya Chauhan, Sayan Saha, Sean Talts, Seher Ellis, Sergei Lebedev, Sergey Kozub, Sevin Fide Varoglu, Shahriar Rouf, Shanbin Ke, Shaogang Wang, Sharad Vikram, Shawn Lu, Siddhartha Menon, Siqiao Wu, skill, Smit Hinsu, snadampal, Sohaib Iftikhar, Soowon Jeong, spiao, Srijan Upadhyay, stevemcgregory, Subham Soni, Subhankar Shah, Swachhand Lokhande, Tai Ly, TensorFlower Gardener, Terry Heo, Terry Sun, Terry Tao, Theotime Combes, Thomas Joerg, Thomas Köppe, Tiago Quelhas, TJ Xu, Toli Yevtushenko, Tomás Longeri, Tom Hennigan, Tommy Chiang, Tom Natan, Tongfei Guo, Tori Baker, Uwe L. Korn, Vadym Matsishevskyi, Vamsi Manchala, Venkat6871, Victor Stone, Ville Vesilehto, Vitalii Dziuba, Vladimir Belitskiy, Vlad Sytchenko, Volodymyr Kysenko, Wai Hon Law, wan3x, Weiyi Wang, Will Froom, William S. Moses, wondertx, Xuefei Jiang, Yang Chen, Yash Katariya, Yasir Ashfaq, yasiribmcon, Yeou Chiou, Yicheng Luo, Yi Kong, Yimei Sun, Yin Zhang, Yuchen Yao, Yue Sheng, Yulia Baturina, Yunjie Xu, Yunlong Liu, Yun Peng, Yurii Topin, Zac Cranko, Zac Mustin, Zenong Zhang, Zeyu Wang, Zhanyong Wan, Zixuan Jiang, Ziyin Huang, Zviki Nozadze</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/tensorflow/tensorflow/commit/a481b10260dfdf833a1b16007eead49c1d7febf3"><code>a481b10</code></a> Merge pull request <a href="https://redirect.github.com/tensorflow/tensorflow/issues/111627">#111627</a> from tensorflow-jenkins/version-numbers-2.21.0-25481</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/a8f642e88e001734075133f9576ffd253b9eacc3"><code>a8f642e</code></a> Update version numbers to 2.21.0</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/3c51664da546f2e7741ad27e98f8dd22dbbf86f9"><code>3c51664</code></a> Merge pull request <a href="https://redirect.github.com/tensorflow/tensorflow/issues/111517">#111517</a> from tejaswin432/r2.21</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/460d178dda3124806d534ec0fb5e3d0830319628"><code>460d178</code></a> Update RELEASE.md with removal of TB dependency.</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/9e2628c11188f4b1f4361ff08e1de198d13cadb1"><code>9e2628c</code></a> Update RELEASE.md with removal of TB dependency.</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/00a1ba7b4fd31bf1c75482bfff620a1cf21c5815"><code>00a1ba7</code></a> Merge pull request <a href="https://redirect.github.com/tensorflow/tensorflow/issues/111234">#111234</a> from psamanoelton/remove_tb_nigthly</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/41beecf3ee272e928c59e53edfcd5d8b40bf7f2b"><code>41beecf</code></a> Remove tb-nigthly and replace it with protobuf.</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/01dec748b0d0daa3eb60a386b857519644fe15e2"><code>01dec74</code></a> Merge pull request <a href="https://redirect.github.com/tensorflow/tensorflow/issues/111216">#111216</a> from psamanoelton/remove_tb_dependency_partial_rol...</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/9657881d9689a4c06d334645932abff9d3481698"><code>9657881</code></a> Partial rollback to resolve breakage.</li> <li><a href="https://github.com/tensorflow/tensorflow/commit/78d130aec00f8b4dbee85d3757ce32032ec7cbdd"><code>78d130a</code></a> Remove TensorBoard dependency from TensorFlow build</li> <li>Additional commits viewable in <a href="https://github.com/tensorflow/tensorflow/compare/v2.20.0...v2.21.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=tensorflow-cpu&package-manager=pip&previous-version=2.20.0&new-version=2.21.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Jinzhe Zeng <jinzhe.zeng@ustc.edu.cn> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jinzhe Zeng <jinzhe.zeng@ustc.edu.cn>

fix issue introduced by this pr: #5157 This pull request updates the PaddlePaddle dependency versions across the codebase to ensure compatibility with the latest releases. It also improves the installation instructions in the documentation to reflect these updates and to recommend best practices for installing nightly builds. Dependency version updates: * Updated the `paddlepaddle-gpu` dependency in the CUDA test workflow to version `3.4.0.dev20260310` (`.github/workflows/test_cuda.yml`). * Updated the `paddlepaddle` dependency in the Python test workflow to version `3.4.0.dev20260310` (`.github/workflows/test_python.yml`). Documentation improvements: * Updated installation instructions in `doc/install/easy-install.md` and `doc/install/install-from-source.md` to reference `paddlepaddle-gpu==3.3.0` and `paddlepaddle==3.3.0` as the latest stable release versions [[1]](diffhunk://#diff-8072dac581dd568fe718ff7204aae121465797d5d2890c4bc4d3ee8d978951e7L172-R174) [[2]](diffhunk://#diff-8072dac581dd568fe718ff7204aae121465797d5d2890c4bc4d3ee8d978951e7L183-R185) [[3]](diffhunk://#diff-865b1d35cff7d06cf73ebb95b0f2d93c5cc77b19c64305753008c90dfe9370c8L100-R108). * Modified nightly build installation commands in the documentation to recommend using the `-U` (upgrade) flag for both GPU and CPU versions, ensuring users get the latest nightly build [[1]](diffhunk://#diff-8072dac581dd568fe718ff7204aae121465797d5d2890c4bc4d3ee8d978951e7L172-R174) [[2]](diffhunk://#diff-8072dac581dd568fe718ff7204aae121465797d5d2890c4bc4d3ee8d978951e7L183-R185) [[3]](diffhunk://#diff-865b1d35cff7d06cf73ebb95b0f2d93c5cc77b19c64305753008c90dfe9370c8L100-R108).  ## Summary by CodeRabbit * **New Features** * Training now supports the AdamW optimizer alongside Adam. * **Documentation** * Updated installation guides with newer PaddlePaddle stable and nightly package versions and improved nightly/pre-release pip syntax. * **Chores** * Bumped PaddlePaddle dependency versions used in CI workflows (Python and CUDA test pipelines).

## Summary by CodeRabbit * **New Features** * Optional charge/spin electronic embedding for DPA3 (toggleable). * Descriptor and model APIs accept optional per-frame parameters (fparam); when enabled, default per-frame params are forwarded into descriptor embeddings. * **Tests** * Expanded test coverage for charge/spin embedding, fparam construction/propagation, backend paths, and serialization. * **Documentation** * Added help/config entry for the charge/spin embedding option.  --------- Signed-off-by: Duo <50307526+iProzd@users.noreply.github.com>

Problem - DPA-2.4-7M is available on HuggingFace and has mirrors (hf-mirror + ModelScope), but is not listed in the built-in pretrained registry. Change - Add DPA-2.4-7M to deepmd/pretrained/registry.py with HuggingFace URL, hf-mirror URL, and ModelScope mirror URL. - Record the model file SHA256 for integrity checking. Notes - ModelScope mirror: https://modelscope.cn/models/DeepModelingCommunity/DPA-2.4-7M Authored by OpenClaw (model: custom-chat-jinzhezeng-group/gpt-5.2)  ## Summary by CodeRabbit * **New Features** * The DPA-2.4-7M pretrained model is now available for download and use. * Additional download sources/URLs added for DPA-3.2-5M and DPA-3.1-3M to improve availability.

This pull request updates the optimizer handling in the `deepmd/pd/train/training.py` file, adding support for selecting between `Adam` and `AdamW` optimizers based on configuration, and improves profiling clarity. **Optimizer selection improvements:** * Refactored optimizer initialization to dynamically select between `paddle.optimizer.Adam` and `paddle.optimizer.AdamW` depending on the value of `self.opt_type`. This allows for more flexibility in choosing the optimizer. **Profiling and logging enhancements:** * Updated the profiling context label from `"Adam update"` to `"Optimizer update"` to accurately reflect the optimizer in use, improving the clarity of profiling output.  ## Summary by CodeRabbit * **New Features** * Added support for selecting between Adam and AdamW optimizers during training.

## Summary by CodeRabbit * **New Features** * Public C API bumped to v26 and adds queries to check whether models use default frame parameters across model types and backends. * Backends and model interfaces now expose a consistent "has default fparam" status and fall back safely when backend hooks are absent. * **Tests** * Added extensive unit, integration, and LAMMPS tests plus a model fixture to validate default-frame-parameter behavior.

This PR fixes a typo pattern.  ## Summary by CodeRabbit * **Documentation** * Fixed multiple typos and wording in method docstrings and parameter descriptions (e.g., corrected "statisitcs" → "statistics", "Reveive" → "Receive", and similar inconsistencies). These edits improve clarity and consistency of technical documentation without changing behavior, interfaces, or functionality.

Problem - flake8 is redundant with ruff in pre-commit and adds extra dependency/latency. Change - Remove the flake8 repo hook from .pre-commit-config.yaml. Authored by OpenClaw (model: gpt-5.2)  ## Summary by CodeRabbit * **Chores** * Removed Flake8 linting tool from the development pre-commit pipeline. This change reduces external dependencies and maintenance overhead for developers. The pre-commit pipeline continues to function normally with other configured quality checks remaining active.

updates: - [github.com/astral-sh/ruff-pre-commit: v0.15.5 → v0.15.6](astral-sh/ruff-pre-commit@v0.15.5...v0.15.6) - [github.com/pre-commit/mirrors-clang-format: v22.1.0 → v22.1.1](pre-commit/mirrors-clang-format@v22.1.0...v22.1.1)  Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

#5302) ## Summary - Add `dp freeze` support for the pt_expt backend, enabling checkpoint `.pt` → exported `.pte` conversion - Add end-to-end tests for both `dp freeze` and `dp test` with `.pte` models ## Background The pt_expt backend can export models to `.pte` via `deserialize_to_file()`, and `dp test` can already load `.pte` models through the registered `DeepEval`. However, `dp freeze` was not wired up — calling `dp freeze -b pt-expt` hit `RuntimeError: Unsupported command 'freeze'`. ## Changes **`deepmd/pt_expt/entrypoints/main.py`** - Add `freeze()` function: loads `.pt` checkpoint → reconstructs model via `get_model` + `ModelWrapper` → serializes → exports to `.pte` via `deserialize_to_file` - Wire `freeze` command in `main()` dispatcher with checkpoint directory resolution and `.pte` default suffix **`source/tests/pt_expt/test_dp_freeze.py`** (new) - `test_freeze_pte` — verify `.pte` file is created from checkpoint - `test_freeze_main_dispatcher` — test `main()` CLI dispatcher with freeze command - `test_freeze_default_suffix` — verify non-`.pte` output suffix is corrected to `.pte` **`source/tests/pt_expt/test_dp_test.py`** (new) - `test_dp_test_system` — test `dp test` with `-s` system path, verify `.e.out`, `.f.out`, `.v.out` outputs - `test_dp_test_input_json` — test `dp test` with `--valid-data` JSON input ## Test plan - [x] `python -m pytest source/tests/pt_expt/test_dp_freeze.py -v` (3 passed) - [x] `python -m pytest source/tests/pt_expt/test_dp_test.py -v` (2 passed)  ## Summary by CodeRabbit * **New Features** * Added a "freeze" CLI command to convert PyTorch checkpoints into portable .pte model files, with output filename normalization and sensible default naming; multi-task head usage now emits a clear unsupported message. * **Tests** * Added unit tests for the freeze command and CLI dispatch behavior. * Added integration tests validating end-to-end dp_test workflows using frozen models.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

For standard deviation of `fparam/aparam`, $\sigma = \sqrt{\frac{1}{N} \sum_{i=1}^{N} (x_i - \bar{x})^2}=\sqrt{\frac{\sum x_i^2}{N} - \left( \frac{\sum x_i}{N} \right)^2}$. When all `fparam`/`aparam` have equal values in one dimension, $\frac{\sum x_i^2}{N} - \left( \frac{\sum x_i}{N} \right)^2$ equals zero. However, it sometimes becomes a very small negative number(for example, 1e-18) due to numerical instability, so $\sqrt{\frac{\sum x_i^2}{N} - \left( \frac{\sum x_i}{N} \right)^2}$ becomes `nan`.  ## Summary by CodeRabbit * **Bug Fixes** * Improved numerical stability in variance/std calculations by ensuring intermediate variance values are non-negative before taking the square root. This prevents occasional floating-point underflow from producing invalid results and yields more reliable statistical outputs across edge-case inputs.  --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

#5325) - Fix natoms[0] -> natoms in generalized force branch (natoms is int) - Replace xp.einsum with array-API-compatible xp.sum + broadcasting - Fix return type annotation of Loss.call and EnergyLoss.call from dict[str, Array] to tuple[Array, dict[str, Array]] - Add TestEnerGF consistency test for generalized force code path - Add dpmodel-level unit tests for EnergyLoss (basic, aecoeff, generalized force, huber, serialize round-trip)  ## Summary by CodeRabbit * **Bug Fixes** * Enhanced numerical accuracy in energy loss force calculations through optimized computation methods. * **Tests** * Added comprehensive test coverage for energy loss calculations, including generalized coordinate scenarios. * Expanded multi-backend compatibility validation across TensorFlow, PyTorch, JAX, Array API, and Paddle.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Summary - Add `FrozenModel` to pt_expt backend for loading pre-frozen model files (`.pte`, `.pth`, `.dp`) - Create dpmodel-level `FrozenModel` (`NativeOP` + `BaseModel`) with all delegation methods, so pt_expt wraps it via `@torch_module` instead of duplicating code - pt_expt `FrozenModel` handles `.pte` natively via `serialize_from_file`, falls back to generic backend detection for other formats - Add pt_expt support to frozen model consistency test ## Test plan - [x] Cross-backend consistency test (`source/tests/consistent/model/test_frozen.py`) — pt_expt consistent_with_ref and self_consistent pass - [x] Existing pt/tf frozen model tests unaffected  ## Summary by CodeRabbit * **New Features** * Added support for loading and using frozen model files across workflows and exposed a FrozenModel in the Python API. * Broadened backend compatibility to include an additional experiment backend for frozen models. * **Tests** * Added/updated tests to validate frozen-model loading and evaluation across supported backends, including the new experiment backend.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

1. refactor name-based routing 2. add slice mode for HybridMuon opt 3. add Magma-lite damping for Muon path  ## Summary by CodeRabbit * **New Features** * HybridMuon gains routing modes (slice, 2d, flat), name-aware routing for biases/Adam variants, and a magma_muon option for Magma-lite damping. Optimizer now accepts named parameters; deprecated 2D-only options removed. * **Documentation** * Updated optimizer docs to describe new routing modes, magma_muon and flash_muon options, and adjusted lr_adjust default. * **Tests** * Expanded tests for routing modes, Magma damping, and state compatibility; some legacy tests consolidated.

Problem - DPA3-Omol-Large is already published on model hubs but is not exposed through `dp pretrained`. - Users currently cannot download or resolve it via a built-in model alias. Change - add `DPA3-Omol-Large` to the built-in pretrained model registry - include Hugging Face / hf-mirror / ModelScope download URLs and the model sha256 - update the pretrained-model docs and add alias/backend coverage in tests Notes - The SHA256 (`dc4d252b31450b41eb3546cc48f640ad0831c0b5d069ce27d996e0ff58fc037a`) was taken from the Hugging Face LFS object for `DPA3-Omol-Large.pt`. - In this environment I only ran lightweight local validation (`py_compile` + an AST-based registry check). I did not run the full project test suite because the repo test environment was not fully provisioned here. Authored by OpenClaw (model: gpt-5.4)  ## Summary by CodeRabbit * **New Features** * Added "DPA3-Omol-Large" as a new pretrained model, available from multiple download mirrors for improved accessibility and reliability. * **Documentation** * Updated pretrained model examples to include "DPA3-Omol-Large". * **Tests** * Added tests to validate recognition and alias normalization for the new model name.

## Summary by CodeRabbit * **New Features** * Introduced loss_func ("mse" or "mae") to select MSE vs MAE for energy/force/virial/atom losses. * Added f_use_norm to enable vector‑norm MAE behavior when allowed. * **Validation** * Enforced that f_use_norm is only valid when use_huber is enabled or loss_func="mae"; invalid combos are rejected. * **Tests** * Extended loss tests and skipping logic to cover loss_func and f_use_norm combinations. * **Documentation** * Updated docs to describe loss_func and resulting metric names (rmse_* vs mae_*). * **Chores** * New options are persisted in serialized configurations. * **Notes** * Some backends currently only support "mse" (MAE not yet available everywhere).

This is a **breaking change**: bump the model data versio of `LinearEnergyAtomicModel` from 2 to 3 due to the bug fixing. ## Summary - Add `LinearEnergyModel` to pt_expt backend, enabling linear combination of multiple sub-models - Add `get_linear_model` factory in pt_expt for constructing from config dicts - Fix bugs in dpmodel/pt shared code: - `get_linear_model` (pt) not propagating `type_map` to sub-models - `LinearEnergyAtomicModel` (dpmodel) missing `weights` parameter, causing deserialization failure - `_compute_weight` calling `array_api_compat.array_namespace()` with Python list and using numpy dtype with torch ## Test plan - [x] Cross-backend consistency test (`source/tests/consistent/model/test_linear_ener.py`) — pt vs pt_expt, with parameterized exclude types - [x] Existing dpmodel/pt linear model tests still pass  ## Summary by CodeRabbit * **New Features** * Added LinearEnergyModel and a "linear_ener" fitting path to combine multiple sub-models. * Configurable weighting when combining sub-model energies: "mean", "sum", or a custom vector; weights are validated and stored. * **Behavior** * Sub-model type mappings are propagated when omitted. * Model serialization now persists weight settings and advances the model version for compatibility. * **Tests** * Added cross-backend and unit tests validating weighting behaviors, outputs, and selector updates.  --------- Signed-off-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com> Co-authored-by: Han Wang <wang_han@iapcm.ac.cn> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

## Summary - Implement `SpinModel` and `SpinEnergyModel` in the pt_expt backend, supporting spin degrees of freedom for magnetic systems - Make dpmodel `SpinModel` array-API compatible so the same code works across numpy/torch/jax backends - Add spin virial correction (`coord_corr_for_virial`) to dpmodel and pt_expt, matching the pt backend - Fix `get_spin_model` in dpmodel to not mutate the caller's input data dict (pt backend already used `deepcopy`) ## Changes ### dpmodel (`deepmd/dpmodel/model/`) - `spin_model.py`: Replace all `np.*` operations with `array_api_compat` equivalents (`xp.concat`, `xp.where`, `xp.zeros` with `device=`, slicing instead of `xp.split`). Add `compute_or_load_stat` and virial correction support via `coord_corr_for_virial` / `extended_coord_corr`. - `make_model.py`: Thread `coord_corr_for_virial` through `call_common` → `model_call_from_call_lower` (extends to ghost atoms via mapping) → `call_common_lower` → `forward_common_atomic`. - `model.py`: Add `copy.deepcopy(data)` in `get_spin_model` to prevent in-place mutation of input dict. ### pt_expt (`deepmd/pt_expt/model/`) - `spin_model.py` (new): `@torch_module` wrapper inheriting from dpmodel `SpinModel`. - `spin_ener_model.py` (new): `SpinEnergyModel` with `forward()` / `forward_lower()` / `forward_lower_exportable()` providing user-facing output translation. - `make_model.py`, `transform_output.py`: Accept `extended_coord_corr` for virial correction. ### Tests - `test_spin_ener_model.py` (new): Unit tests for output keys/shapes, serialize/deserialize round-trip, dpmodel consistency, force finite-difference, virial finite-difference, and `torch.export` exportability. - `test_spin_ener.py`: Cross-backend consistency tests for `call`/`call_lower`, `compute_or_load_stat`, and load-from-file. Virial output now compared across pt and pt_expt. ## Test plan - [x] `python -m pytest source/tests/pt_expt/model/ -v` — all 28 tests pass - [x] `python -m pytest source/tests/consistent/model/test_spin_ener.py -v` — all 12 tests pass (18 skipped for uninstalled backends) - [x] Force and virial verified by finite-difference tests - [x] `torch.export.export` verified on `forward_lower_exportable` - [x] `compute_or_load_stat` load-from-file verified across dp/pt/pt_expt  ## Summary by CodeRabbit * **New Features** * Added SpinEnergyModel with exportable lower-level forward, energy/force/virial outputs, and compute_or_load_stat preprocessing. * Optional virial coordinate-correction can be supplied and is propagated through forward paths. * **Bug Fixes** * Prevented in-place mutation of input data during model preparation. * **Tests** * Expanded tests for exportable workflows, force/virial validation, multi-backend (including PT_EXPT) and array‑API strict modes.  --------- Signed-off-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com> Co-authored-by: Han Wang <wang_han@iapcm.ac.cn> Co-authored-by: Duo <50307526+iProzd@users.noreply.github.com>

## Summary This PR completely restructures the `learning-rate.md` documentation to improve clarity, organization, and accuracy. The previous version had content scattered across sections with significant repetition. The new structure follows a user-centric approach: quick start → configuration reference → mathematical theory. ## Key Changes ### Structural Improvements - **Reorganized section order**: Quick Start → Parameters → Schedule Types → Warmup → Mathematical Theory → Migration Guide - **Eliminated content duplication**: Removed redundant formulas between Theory and Instructions sections  ## Summary by CodeRabbit * **Documentation** * Reworked learning-rate guide into a structured, example-driven reference with Quick Start, explicit exponential and cosine schedules, and JSON configuration examples. * Added a Notation/Theory section, clear warmup formulas and mutual‑exclusivity rules, unified parameter descriptions (start_lr/stop_lr/stop_lr_ratio/decay_steps) and smooth vs stepped behavior. * Expanded migration guidance for versions prior to 3.1.3 and refreshed references.

Make long-running training progress easier to read by keeping the relative ETA and appending a concise absolute finish time across the pt, pd, tf, and pt_expt backends.  ## Summary by CodeRabbit * **New Features** * Training logs now include both remaining ETA and an estimated local finish time (YYYY-MM-DD HH:MM). * Timezone-aware local timestamps are shown across training frameworks for clearer cross-region monitoring and more consistent periodic timing output.

## Summary - Add model compression (embedding net tabulation) for the pt_expt backend, matching the existing pt backend capability - Compressed models replace embedding net forward passes with polynomial lookup tables via C++ custom ops (`tabulate_fusion_se_*`), significantly speeding up inference - Support all compressible descriptors: `se_e2_a`, `se_r`, `se_t`, `se_t_tebd`, `dpa1`, `se_atten_v2`, `dpa2` (hybrid delegates automatically) ### Key changes **Infrastructure:** - `deepmd/pt_expt/utils/tabulate_ops.py` — Register `torch.library.register_fake` for all 5 custom ops to enable `torch.export`/`make_fx` tracing through compressed forward paths - `deepmd/pt_expt/utils/tabulate.py` — `DPTabulate` subclass that detects descriptor type via serialized data (avoids `isinstance` checks against pt-specific classes) - `deepmd/pt_expt/entrypoints/compress.py` — Entry point: load `.pte` → deserialize → `enable_compression()` → re-export `.pte` **Descriptors:** Each gets `enable_compression()` + `@cast_precision` `call()` override with compressed branch using the appropriate custom op. **dpmodel — compression state serialization (breaking version bumps):** The pt_expt backend persists models via `serialize()` → `model.json` → `deserialize()` (the `.pte` format), unlike pt/tf which use native framework save mechanisms (torch.jit.save / tf.saved_model) that capture the full runtime state. This means compression state (tabulated polynomial coefficients, precomputed type embeddings) must survive the serialize/deserialize round-trip for compressed `.pte` models to work. Each compressible descriptor's serialization version is bumped when the model is compressed. **Uncompressed models continue to use the old version**, so there is no breakage for existing uncompressed model files. All backends (pt, pd, tf) accept the new version in `deserialize()` and simply ignore the `"compress"` key. | Descriptor | Version bump | Added fields | |---|---|---| | `se_e2_a` | 2 → 3 | `compress_data`, `compress_info` | | `se_r` | 2 → 3 | `compress_data`, `compress_info` | | `se_t` | 2 → 3 | `compress_data`, `compress_info` | | `se_t_tebd` | 1 → 2 | `compress_data`, `compress_info`, `type_embd_data` | | `dpa1` | 2 → 3 | `type_embd_data`, `geo_compress`, `compress_data`/`info` (if geo) | | `se_atten_v2` | 2 → 3 | `type_embd_data`, `geo_compress`, `compress_data`/`info` (if geo) | | `dpa2` | 3 → 4 | compress dict inside `repinit_variable` | **dpmodel:** Initialize `self.compress = False` in all descriptor `__init__` methods. ## Test plan - [x] `source/tests/pt_expt/model/test_model_compression.py` — end-to-end compress → serialize → deserialize → eval - [x] `source/tests/pt_expt/descriptor/` — compressed forward, consistency, exportable, make_fx tests for all descriptors - [x] `source/tests/consistent/descriptor/` — cross-backend consistency tests pass with bumped versions  ## Summary by CodeRabbit * **New Features** * Added descriptor compression functionality to reduce model size and optimize memory usage during inference. * Introduced `compress` CLI command to enable tabulated embedding optimization on frozen trained models. * Enhanced descriptor serialization with improved version compatibility across multiple backends. * **Tests** * Added comprehensive test coverage for compressed descriptor forward passes and model compression workflows.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Summary - Add `--finetune`, `--model-branch`, and `--use-pretrain-script` support to `dp --pt-expt train`, mirroring the pt backend's finetune flow (load pretrained checkpoint, change type map, selective weight copy, output bias adjustment) - Support finetuning from both `.pt` checkpoints and frozen `.pte` models (embed `model_params` in `.pte` during freeze for `--use-pretrain-script`) - Fix a bug in dpmodel's `base_atomic_model.change_type_map` where `out_bias`/`out_std` were not extended before remapping when the new type map introduces unseen types, causing `IndexError` with negative remap indices ## Usage examples ```bash # Finetune from a .pt checkpoint dp --pt-expt train input.json --finetune pretrained.pt # Finetune from a frozen .pte model dp --pt-expt train input.json --finetune pretrained.pte # Copy descriptor/fitting config from pretrained model dp --pt-expt train input.json --finetune pretrained.pt --use-pretrain-script # Finetune from a multi-task pretrained model (select a branch) dp --pt-expt train input.json --finetune pretrained.pt --model-branch Default # Re-initialize fitting net randomly (only keep descriptor weights) dp --pt-expt train input.json --finetune pretrained.pt --model-branch RANDOM ``` ## Files changed | File | Change | |------|--------| | `deepmd/pt_expt/utils/finetune.py` | **New** — `get_finetune_rules()` for pt_expt, supports `.pt` and `.pte` | | `deepmd/pt_expt/entrypoints/main.py` | Wire `--finetune`/`--model-branch`/`--use-pretrain-script` through `train()` → `get_trainer()` → `Trainer`; pass `model_params` to `.pte` during freeze | | `deepmd/pt_expt/train/training.py` | Finetune weight loading in `Trainer.__init__` (`.pt` and `.pte`); `model_change_out_bias()` | | `deepmd/pt_expt/utils/serialization.py` | Embed/extract `model_params.json` in `.pte` archive | | `deepmd/dpmodel/atomic_model/base_atomic_model.py` | Fix `change_type_map` to extend `out_bias`/`out_std` for new types (array-api compatible) | | `source/tests/pt_expt/test_finetune.py` | **New** — 9 tests covering bias adjustment, type map change, CLI dispatch, `.pte` finetune, `--use-pretrain-script`, `random_fitting`, inherited weight consistency | | `source/tests/consistent/model/test_ener.py` | Add `test_change_type_map_new_type` verifying `out_bias`/`out_std` extension across dp, pt, pt_expt | ## Test plan - [x] `python -m pytest source/tests/pt_expt/test_finetune.py -v` (9 passed) - [x] `python -m pytest source/tests/pt_expt/test_training.py -v` (11 passed, no regression) - [x] `python -m pytest source/tests/consistent/model/test_ener.py -k change_type_map -v` (3 passed) - [x] `python -m pytest source/tests/consistent/descriptor/test_se_e2_a.py -v` (351 passed, no regression)  ## Summary by CodeRabbit * **New Features** * Fine-tuning workflow: supply pretrained checkpoints, select branch, and toggle pretrain-script behavior * Automatic expansion of atom type maps (new types get zero bias and unit std) while preserving existing mappings * Improved finetune resume: selective merging of pretrained descriptor/fitting weights and bias-adjustment modes * Export/import embeds/restores model metadata to/from artifacts * **Tests** * Unit and end-to-end tests for finetuning, bias adjustment, type-map expansion, and frozen-artifact scenarios  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

…#5699) ## Problem The `examples/infer_water` README tells users to run `cmake` and `make`, but the resulting inference binaries (`infer_water_cc`, `infer_water_c`, `infer_water_hpp`, `infer_water_nlist`) load a frozen model `graph.pb` from the current directory. `graph.pb` is gitignored and never generated, and the `convert_model.c` helper that creates it is neither built by `CMakeLists.txt` nor mentioned in the README. Following the documented steps therefore produces binaries that fail at runtime. ## Fix Add `convert_model` as a CMake target (it uses only `DP_ConvertPbtxtToPb` from `deepmd/c_api.h`, so it links `DeePMD::deepmd_c` like the other C examples) and document the full sequence in the README: build, run `./convert_model` to generate `graph.pb` from the bundled test model `source/tests/infer/deeppot.pbtxt`, then run the inference examples. The README makes explicit that `make` only compiles the executables and does not create the model file, and that `convert_model` must be run from the example directory (so the relative path to the bundled model resolves) with a TensorFlow-enabled build (since `graph.pb` is a TensorFlow frozen model). ## Test This is a build/documentation example that is not part of the unit-test suite; a meaningful test would require building the example project against an installed TensorFlow-enabled DeePMD-kit and running the binaries, which is outside the unit scope. The fix is verified by inspection: `convert_model.c` includes only `deepmd/c_api.h`, `DP_ConvertPbtxtToPb` is a real C-API function, and the bundled `deeppot.pbtxt` fixture exists and resolves relative to the example directory. Fix #5693  ## Summary by CodeRabbit * **New Features** * Added a new helper build target for converting a model file in the water inference example. * Expanded the example documentation with step-by-step build and inference guidance. * **Documentation** * Clarified that building the example only compiles executables and does not generate the model file. * Added instructions for creating the required `graph.pb` file and running the available inference binaries.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Problem The TensorFlow `change-bias` dispatcher only accepts inputs ending in `.pb`, `.pbtxt`, `.ckpt`, `.meta`, `.data`, or `.index`, and rejects everything else with `RuntimeError("The model provided must be a checkpoint file or frozen model file (.pb)")`. However, the CLI examples and the frozen-model fallback message recommend passing a checkpoint directory, and real TensorFlow checkpoint prefixes commonly look like `model.ckpt-1000`. A checkpoint directory has no recognized suffix, and a `model.ckpt-1000` prefix ends in `-1000`, so both are rejected before `_change_bias_checkpoint_file()` — which already resolves the checkpoint via the `checkpoint` state file — can run. ## Fix Route suffix-less inputs to the checkpoint handler when a TensorFlow `checkpoint` state file is present in the effective directory (the input itself if it is a directory, otherwise its parent). Inputs without such a state file still raise the original error. `_change_bias_checkpoint_file()` now also resolves `checkpoint_dir` correctly when the input is a directory. ## Test `source/tests/tf/test_change_bias.py` previously exercised only `.pb`, `.ckpt`, and bad-suffix inputs, never a bare directory or a `ckpt-<step>` prefix. Two new tests mock `_change_bias_checkpoint_file` and assert it is invoked for a `model.ckpt-1000` prefix and for a checkpoint directory that contain a `checkpoint` state file. On the current dispatcher both raise the "checkpoint file or frozen model file" RuntimeError before reaching the handler; after the fix both route correctly. The existing rejection tests (e.g. a `model.xyz` file with no checkpoint state file) still raise. Fix #5683  ## Summary by CodeRabbit * **Bug Fixes** * Improved handling of checkpoint inputs so bias changes now work reliably for both checkpoint file prefixes and checkpoint directories. * Unrecognized input paths are now treated as checkpoint-based inputs only when a valid checkpoint state is present, reducing incorrect routing. * **Tests** * Added coverage for checkpoint prefix and directory cases to verify the correct bias-change path is used.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Problem `change-bias --numb-batch` is documented as the number of frames to use per data system, with `0` meaning all data. On the TensorFlow data-based path this value was ignored: `_change_bias_checkpoint_file` called `_apply_data_based_bias(...)` without `numb_batch`, and `_apply_data_based_bias` then hardcoded `ntest=1` when calling `change_energy_bias_lower`. As a result the bias was always estimated from a single frame per system regardless of `-n/--numb-batch`, and the documented `0 means all data` behavior had no effect. ## Fix Thread `numb_batch` into `_apply_data_based_bias` and forward it to `change_energy_bias_lower` as `ntest`. In `change_energy_bias_lower`, treat `ntest <= 0` as "use all frames in the system" (`numb_test = nframes if ntest <= 0 else min(nframes, ntest)`), implementing the documented `0 means all data`. This is backward compatible: the default `ntest` is 10 and no existing caller passed a non-positive value. ## Test There was no test covering `numb_batch` or `change_energy_bias_lower`. A new test builds a small multi-frame `DeepmdDataSystem` and a mock evaluator that records how many frames it is asked to predict, then asserts that `ntest=1` evaluates one frame per system and `ntest=0` evaluates all frames. On the current code `ntest=0` selects zero frames (raising downstream); after the fix it uses all frames. The `numb_batch -> ntest` parameter plumbing is simple and verified by inspection; the test targets the frame-selection semantics that the fix makes controllable. Fix #5684  ## Summary by CodeRabbit * **Bug Fixes** * Fixed bias recalculation so batch settings now use the expected number of frames. * When no batch limit is set, all available frames are now included instead of a smaller default subset. * **Tests** * Added coverage for batch selection behavior to ensure bias updates use the correct frame counts.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

…C++ inference single & multi-rank (NeighborGraph PR-B) (#5604) ## NeighborGraph PR-B — graph `.pt2` export, compiled training, and C++ inference (single & multi-rank) This PR spans the full PR-B: **B1** (Python: graph `.pt2` export + compiled training on the graph lower), **B2** (C++ single-rank inference of the graph `.pt2`, dynamic edge axis), and **B3** (C++/LAMMPS multi-rank). Built on the merged PR-A (#5583). Scope: dpa1, `attn_layer=0`, pt_expt. ### B1 — graph `.pt2` export + compiled training (Python) - `forward_common_lower_graph_exportable` trace target; `serialization.py` graph export branch (`lower_kind="graph"`, `lower_input_kind` metadata); `_eval_model_graph` DeepEval dispatch (parity vs eager dpa1 **1e-10 pbc+nopbc**). - **Compiled training retargeted to the graph lower so eager == compiled** (the MUST-FIX) → `force_legacy_descriptor` deleted. Root cause was a real dpa1 `call_graph` autograd **detach** bug (`xp.asarray(tebd, device=)` drops the tebd-net gradient under torch); fixed. ### B2 — C++ graph ingestion (dynamic edge axis, single-rank) - Graph `.pt2` uses a **dynamic edge axis** (`Dim("nedge", min=2)`) — one artifact evals any system size (proven across 56- and 380-edge systems at 1e-10), no C++ capacity ceiling. - C++ `DeepPotPTExpt`: `lower_input_is_graph_` + `run_model_graph` (NeighborGraph ABI: `atype, n_node, edge_index, edge_vec, edge_mask, …`) + `buildGraphTensors` (mirrors the #5562 edge path; node types from `atype_ext`); `remap_graph_outputs_to_dense_keys` (single-rank). - gtest: 5 cases × {double,float} = 10/10 (build-nlist parity, dynamic-E 2nd size, `ago>0`, tiny system, atomic-overload). The review process caught two bugs that would otherwise have shipped: an `ago>0` heap-OOB (by inspection) and a public-vs-internal output-key mismatch (at runtime). ### B3 — multi-rank C++ / LAMMPS (non-MP) - **dpa1 is non-message-passing ⇒ multi-rank needs NO `border_op`/with-comm artifact** (that is a message-passing concern, deferred to PR-G). Multi-rank reuses the **same single-rank graph `.pt2`**, fed an **extended-region graph** (`buildGraphTensors(fold_to_local=false)`, `N=nall`, ghost node types from `atype_ext` incl. halo), with owned energy = `sum(atom_energy[0:nloc])` and the extended force folded to owners through the **existing dense `select_map` reverse-comm**. The fail-fast for `graph && multi_rank && has_message_passing` is retained. - **Validated locally on multi-CPU** (no GPU needed for correctness): `test_lammps_dpa1_graph_pt2.py` — single-rank vs reference, `mpirun -n 2` ≡ single-rank (energy + per-atom force + virial, atol 1e-8), plus an empty-subdomain (`nloc=0`) corner. Single-rank gtests stay 10/10 (multi-rank is purely additive). Multi-rank matched single-rank on the first run. ### Tests / known limitations - Per-task + whole-phase reviews all Ready-to-merge. - **pt_expt-only; dpa1 (non-MP) only.** Follow-ons: **PR-C** vesin/nv O(N) builders (carry-all builders still use `nonzero`, eager-only), **PR-D** attention, **PR-E** angles, **PR-F** jax graph force, **PR-G** dpa2/3 message-passing (forward halo + with-comm). CUDA multi-rank unvalidated locally. Carried code-cleanup follow-ups: a ~60-line DRY duplication in `training.py`; the multi-rank *atomic* output branch has no direct gtest (covered indirectly by the mpirun per-atom-virial assertion, since a single-process gtest can't set `nprocs>1`).  ## Summary by CodeRabbit ## Summary * **New Features** * Added support for graph-schema (NeighborGraph) model archives with a selectable `lower_kind="graph"` export path, including CLI support and new graph-form inference handling. * Added static edge-capacity support during graph construction. * **Bug Fixes** * Improved gradient continuity for type embeddings in graph mode. * Enhanced trace/export stability by preventing out-of-range graph indices/frame IDs and making scatter/frame sizing more consistent. * **Tests** * Added/extended parity, export metadata, training, and LAMMPS single-/multi-rank validation for graph-form `.pt2`, plus metadata checks for `lower_input_kind`.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

## Problem `DeepmdData.get_single_frame()` runs the data modifier through a one-off `ThreadPoolExecutor` but never calls `future.result()`. The context manager waits for the submitted task to finish, but any exception raised inside `modifier.modify_data(...)` stays stored on the `Future` and is never surfaced. As a result, frame-level data loading silently ignores modifier failures: callers receive the unmodified frame, and when caching is enabled that bad frame can be cached for future use. Other code paths call the modifier directly and do propagate errors, so the behavior was inconsistent. ## Fix Call `future.result()` before leaving the modifier block so an exception raised inside the modifier propagates instead of being swallowed (and the unmodified frame cached). ## Test A new test constructs a `DeepmdData` with a modifier whose `modify_data` always raises, and asserts that `get_single_frame` re-raises the error and does not cache the failed frame. On the current code the error is swallowed (no exception, frame cached); after the fix the error propagates. Fix #5690  ## Summary by CodeRabbit * **Bug Fixes** * Improved error handling when preparing a single frame so failures from data modifiers are no longer hidden. * Prevented invalid frames from being saved and reused after a modifier error. * **Tests** * Added coverage to verify modifier failures are reported correctly and do not populate the frame cache.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Summary - Add backend-independent training abstractions under `deepmd.dpmodel.train` for task/rank normalization, display scheduling, learning-curve output, checkpoint cadence, lifecycle hooks, and shared train entrypoint orchestration. - Factor common training-data helpers so single-task training is handled as a one-task collection and multi-task data construction/summary/probability handling is shared where possible. - Add a backend-independent finetune rule builder in `deepmd.utils.finetune`, and reduce the PT, PT-exportable, Paddle, and JAX backend finetune modules to backend-specific checkpoint loading plus shared rule generation. - Migrate JAX train entrypoint/trainer onto the shared pipeline and add JAX finetune plus multi-task support on top of the new abstractions. - Migrate `pt_expt` train entrypoint/trainer behavior further onto the shared pipeline, including single-task-as-multi-task normalization, data summaries, checkpoint retention, stat-file parent creation, relative latest checkpoint symlinks, and checkpoint parent creation. - Address PR review comments around task-key validation, learning-curve metric ordering, lifecycle cleanup, `print_summary` fallback behavior, broken `__len__` handling, JAX finetune branch/alias validation, numeric-looking JAX task keys, HDF5 stat paths, and `pt_expt` checkpoint symlinks. - Move the new dpmodel trainer/entrypoint tests from `source/tests/test_dpmodel_*.py` into `source/tests/common/dpmodel/`. Refs #5229, #5230, #5231 ## Tests - `ruff format .` - `ruff check .` - `git diff --check` - `PYTHONPATH=/home/jzzeng/codes/deepmd-kit pytest source/tests/common/dpmodel/test_train_abstract_trainer.py source/tests/common/dpmodel/test_train_entrypoint.py source/tests/common/dpmodel/test_train_data.py source/tests/common/dpmodel/test_training_utils.py source/tests/common/test_finetune_utils.py source/tests/jax/test_training.py source/tests/pt_expt/test_entrypoint.py source/tests/pt_expt/test_multitask.py::TestMultiTaskSeA::test_multitask_finetune source/tests/pt_expt/test_multitask.py::TestMultiTaskSeA::test_multitask_finetune_from_single_task source/tests/pt_expt/test_multitask.py::TestMultiTaskSeA::test_multitask_finetune_no_change_model_params -q` (`53 passed, 2 subtests passed`) - `PYTHONPATH=/home/jzzeng/codes/deepmd-kit timeout 180 srun --gres=gpu:1 dp --jax train input.json --skip-neighbor-stat --finetune pretrain.jax --use-pretrain-script` on a temporary 1-step water finetune smoke; completed on NVIDIA GeForce RTX 5090 and saved `ft-model-1.jax`. - `PYTHONPATH=/home/jzzeng/codes/deepmd-kit timeout 180 srun --gres=gpu:1 dp --pt-expt train input.json --skip-neighbor-stat` on a temporary 2-step water smoke; completed on NVIDIA GeForce RTX 5090, saved `ckpts/pt-model-2.pt`, created `stats/stat.hdf5`, and verified `ckpts/pt-model.pt -> pt-model-2.pt` with old step checkpoint pruned by `max_ckpt_keep=1`. ## Notes - Paddle-specific runtime tests were not run locally because `paddle` is not installed in this environment. - Plain PyTorch backend test collection is blocked in this environment by external `deepmd_gnn`/CUDA initialization, not by the shared finetune rule builder changes.  ## Summary by CodeRabbit * **New Features** * Introduced a unified, backend-independent training framework with consistent single-task and multi-task handling, learning-curve output, and structured training/validation steps. * Added a common training entrypoint abstraction that standardizes config normalization, neighbor-stat updates, and lifecycle teardown. * Implemented full-validation with best-checkpoint tracking, top-K selection, and `val.log` reporting (including backend-specific checkpoint suffixes). * **Bug Fixes** * Improved checkpoint save/restore and retention (including “latest” link updates and older checkpoint cleanup). * Improved task-weighting logic to better handle datasets with/without sizing information. * Fixed multi-task neighbor-stat updates and JAX full-validation error propagation across ranks. * **Tests** * Expanded unit and smoke tests for training orchestration, finetuning, validation, and checkpoint reconciliation. * **Documentation** * Updated validation-configuration help text to reflect broader backend support.  --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: njzjz-bot (driven by OpenClaw (model: custom-chat-jinzhezeng-group/gpt-5.5))[bot] <48687836+njzjz-bot@users.noreply.github.com>

## Summary - Stop requesting the Read the Docs PDF output artifact. - Keep the existing Sphinx HTML documentation build unchanged. ## Motivation Read the Docs builds are currently timing out around the PDF generation path. Removing the extra `formats: pdf` request should reduce build time while preserving the main hosted documentation. ## Testing - `git diff --check` --- Submitted by OpenClaw 2026.6.8 (844f405) Authored by OpenClaw (model: custom-chat-jinzhezeng-group/gpt-5.5)  ## Summary by CodeRabbit * **Documentation** * Removed PDF output from the documentation build, so published docs will no longer include a downloadable PDF version.

## Problem Fixes #5677. TF2 standard-model construction resolves descriptor classes through `BaseDescriptor.get_class_by_type(<type>)`, and the TF2 descriptor registry is a distinct registry from the dpmodel and JAX ones (each backend builds its own via `make_base_descriptor`). The `se_t`, `se_t_tebd`, and `se_atten_v2` wrappers were defined without the `@BaseDescriptor.register(...)` decorators their JAX counterparts carry, so their config type names could not be resolved and TF2 model construction failed with an unknown-descriptor error even though the wrapper classes exist and are exported by `deepmd.tf2.descriptor`. The affected type names are `se_e3`, `se_at`, `se_a_3be` (all `se_t`), `se_e3_tebd` (`se_t_tebd`), and `se_atten_v2`. ## Fix Add the missing `@BaseDescriptor.register(...)` decorators, matching the JAX registrations for the same descriptor names. The `se_e3_tebd` registration goes on the outer `DescrptSeTTebd` only (not the block), mirroring JAX. ## Test Adds `source/tests/consistent/test_tf2_descriptor_registration.py`, which asserts that every TF2 descriptor config type name resolves via `get_class_by_type`. It fails on master for the five previously-unregistered names and passes with the fix. The test is gated on `INSTALLED_TF2` (run with `DEEPMD_TEST_TF2=1`). Existing TF2 consistency tests never caught this because they instantiate the wrapper classes directly rather than through the registry string lookup.  ## Summary by CodeRabbit * **New Features** * Expanded TF2 descriptor availability so more descriptor names are recognized and usable. * **Bug Fixes** * Improved descriptor lookup reliability, helping ensure the correct TF2 descriptor is resolved at runtime. * **Tests** * Added coverage to verify that all expected TF2 descriptor types are registered and callable when the TF2 backend is available.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

…5720) ## Problem Fixes #5675. The JAX standard-model factory `get_standard_model` did a bare `data["fitting_net"].pop("type")` and assumed the `fitting_net` block exists, so a configuration that omits `fitting_net` or leaves `type` unset raised `KeyError` on the JAX backend only. The dpmodel and TF2 standard factories both create an empty `fitting_net` when absent and default `type` to `"ener"`, so JAX rejected otherwise valid default-energy configurations that the neighboring backends accept, and backend switching could fail for configs relying on the default energy fitting type. This is normally masked because argcheck normalization fills the default before the factory runs, but the factory is reachable with a raw, un-normalized dict (direct API use, deserialized configs, backend switching), where the missing key surfaces. ## Fix Mirror the TF2 factory: default the block and the type in place — `data["fitting_net"] = data.get("fitting_net", {})` then `fitting_type = data["fitting_net"].pop("type", "ener")`. ## Test Adds `source/tests/jax/test_model_factory.py`, which calls `get_model` with a `fitting_net` that lacks `type` and with `fitting_net` omitted entirely — both raise `KeyError` on master and build an `EnergyModel` after the fix — plus an explicit-type control. This factory path had no coverage for the missing-type case (every existing test supplies a normalized config).  ## Summary by CodeRabbit * **Bug Fixes** * Improved model loading so configurations without a fitting section no longer fail. * When a fitting type is not specified, it now defaults to an energy model. * Explicitly provided fitting types continue to be respected. * **Tests** * Added coverage for missing fitting settings and missing fitting type defaults. * Added a check to confirm explicitly configured fitting types are preserved.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

…5719) ## Problem Fixes #5673. CLI deep-eval commands (`dp test`, `dp eval-desc`, `dp embed`, `dp model-devi`) normalize their `-m` argument through `format_model_suffix()`, which decided whether a name is supported by comparing `Path(filename).suffix` against the registered backend suffixes. `PretrainedBackend` registers built-in model names as whole-name (lowercased) suffix aliases such as `dpa-3.2-5m`, but `Path("DPA-3.2-5M").suffix` is `.2-5M` (and `DPA3-Omol-Large` has an empty suffix), so the alias never matched. Because the CLI always passes a `preferred_backend`, the unmatched alias did not raise; it silently got the default backend suffix appended (`DPA-3.2-5M` becomes `DPA-3.2-5M.pth`) and the subsequent load failed. Note this refines the issue's description: the observed failure is a silent suffix-mangle, not the `ValueError("Unsupported model file format")` the issue predicts — that raise branch is unreachable from these CLI commands because a preferred backend is always supplied. The Python API (`DeepPot("DPA-3.2-5M")`) already worked, because `Backend.detect_backend_by_model` matches with a case-insensitive `endswith`; the two matchers were simply inconsistent. ## Fix Align `format_model_suffix` with `Backend.match_filename`: match on a case-insensitive `endswith` over the suffix set. This recognizes both ordinary dotted suffixes (`.pth`, `.pb`, ...) and whole-name pretrained aliases (`dpa-3.2-5m`, `dpa3-omol-large`) and returns the recognized name unchanged, so `PretrainedBackend` can resolve it downstream. ## Test Adds tests to `source/tests/common/test_pretrained_backend.py` asserting that `DPA-3.2-5M`, `DPA3-Omol-Large`, and a lowercase alias pass through `format_model_suffix` unchanged, with controls that an ordinary `model.pth` is preserved and an unknown bare name still receives the preferred suffix. The alias case fails on master (mangled to `DPA-3.2-5M.pth`) and passes with the fix. This path had no prior coverage.  ## Summary by CodeRabbit * **Bug Fixes** * Improved model file suffix detection so supported formats are recognized more reliably, including case-insensitive matches and non-standard suffix aliases. * Preserved existing behavior for unknown model names, while ensuring the preferred backend suffix is still applied when needed. * **Tests** * Expanded coverage for model suffix handling to verify aliases, standard suffixes, and fallback behavior.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Problem Fixes #5681. The TensorFlow spin `natoms_not_match` path — in the se_e2_a descriptor (`deepmd/tf/descriptor/se_a.py`) and the energy model (`deepmd/tf/model/ener.py`), reached only when ghost atoms are present (`natoms[0] != natoms[1]`) — counted ghost atom types with `tf.unique_with_counts` and then indexed the returned counts and their prefix sums as if they were a dense vector indexed by type id. `tf.unique_with_counts` returns only the types that are actually present, in encounter order, so when a type is entirely absent from a rank's ghost region every subsequent count and offset is shifted and the final index runs past the end of the vector. The result is corrupted ghost spin coordinate/force slices, or a runtime out-of-bounds failure, for otherwise valid ghost atom layouts. ## Root cause and scope The ghost region that reaches the graph is already sorted in ascending type order: `DeepSpinTF::extend` (`source/api_cc/src/DeepSpinTF.cc`) places each ghost atom at `cum_ghost_type_count[type] + reset`, i.e. contiguous per-type blocks, and `session_input_tensors` appends that region verbatim. So the slicing's ascending-order assumption always holds and the only broken assumption is density — the count vector must have one slot per type id, including zero-count slots for absent types. This is exactly what the issue's first failure mode ("ghost atoms absent for some types") describes; the second ("order differs from ascending type order") does not occur for this code path because the C++ side enforces the ordering. ## Fix Replace `tf.unique_with_counts(ghost_atype)` with `tf.math.bincount(ghost_atype, minlength=self.ntypes)` in both call sites. This produces the dense, type-indexed count vector the slicing already assumes, with absent types kept as zero-count slots so all prefix-sum offsets stay aligned. When every type is present the two are equivalent, so the change is behavior-preserving for the common case. ## Test Adds `source/tests/tf/test_spin_ghost_natoms.py`, which drives `natoms_not_match` through a TF session. `test_missing_ghost_type` uses a ghost layout that omits a (non-spin) type — it fails on master with `slice index 2 of dimension 0 out of bounds` and passes with the fix; `test_all_ghost_types_present` is an all-types-present control that passes either way. This branch previously had no coverage at all because every existing spin test exercises the box path or calls the neighbor-list path with `nghost == 0`, so `natoms_not_match` was never reached.  ## Summary by CodeRabbit * **Bug Fixes** * Improved atom-type counting so missing types no longer shift downstream indexing. * Fixed ghost-atom handling to produce stable per-type offsets even when some types are absent. * **Tests** * Added regression coverage for ghost-atom counting with missing and fully present atom types. * Verified the affected TensorFlow path behaves consistently across both scenarios.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

…#5725) ## Problem Fixes #5676. The JAX and TF2 ZBL model factories (`get_zbl_model`), unlike the standard-model factories and the dpmodel ZBL path, mutated the caller's config dict in place (popping the descriptor and `fitting_net` `type`) and did not inject `type_map` into the descriptor and fitting sub-configs. The in-place mutation is inconsistent with the neighboring factories, and the missing `type_map` leaves the descriptor and fitting without the model type map that standard/dpmodel construction provides. ## Fix Deep-copy the input and inject `type_map` into the descriptor and fitting configs in both `get_zbl_model` factories, mirroring the dpmodel ZBL path. The issue also suspected the retained JAX checkpoint metadata could be corrupted by the in-place mutation. That does not actually happen: the JAX trainer passes a `deepcopy` of the model params into the factory, so the stored `model_def_script` is never mutated. No change is made for that sub-claim. ## Test Adds `source/tests/jax/test_zbl_model.py` and a TF2-gated `source/tests/consistent/test_tf2_zbl_model.py`, each asserting the factory leaves the input dict unchanged and that the constructed descriptor and fitting carry the model `type_map`. Both assertions fail on master (the input is mutated and the sub-config `type_map` is `None`) and pass with the fix. The TF2 test is gated on `INSTALLED_TF2` (`DEEPMD_TEST_TF2=1`).  ## Summary by CodeRabbit * **Bug Fixes** * Improved ZBL model setup so input configuration data is no longer modified during model creation. * Ensured type mappings are consistently applied to both descriptor and fitting settings when building ZBL models. * **Tests** * Added coverage for JAX and TF2 ZBL model factories to verify configuration immutability and correct type mapping behavior.  Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Problem Fixes #5672. `DeepPot.eval` appends its optional outputs after `(energy, force, virial)` in a fixed order: atomic `(atom_energy, atom_virial)`, then spin `(force_mag, mask_mag)`, then `hessian`, so the hessian index is `3 + 2*atomic + 2*spin`. But `dp test` read `hessian = ret[3]` unconditionally, before advancing past the atomic and spin outputs. For a hessian model evaluated with atomic output (`dp test -a`) or with spin output, `ret[3]` is the atomic energy or the magnetic force, so the hessian metric compared the wrong array against the hessian label, raising a shape error or reporting invalid hessian MAE/RMSE. The atomic and spin reads already advanced correctly; only the hessian read was hard-wired. The plain non-atomic non-spin path was correct (hessian is genuinely at `ret[3]` there), which is why this went unnoticed: `dp test` has no test that exercises hessian together with atomic or spin outputs. ## Fix Parse the optional outputs by advancing an index through the tuple in the same order `eval` constructs them, via a `_split_optional_ener_outputs` helper. Following the issue's suggestion to return a named structure internally, the helper returns an `_OptionalEnerOutputs` NamedTuple so the hessian, atomic, and spin slots are read by name rather than by a hard-coded index. `DeepPot.eval`'s public tuple return is unchanged (a dict return there would be a breaking change to a core public API and belongs in a separate PR). ## Test Adds `source/tests/common/test_dp_test_ener_split.py`, which feeds sentinel-valued `ret` tuples for every combination of atomic/spin/hessian and asserts each optional output is read from the correct advancing slot. The atomic+hessian and spin+hessian cases fail against the old `ret[3]` read (the hessian slot is `ret[5]`/`ret[7]` when those outputs are present).  ## Summary by CodeRabbit * **Bug Fixes** * Improved `dp test` handling of optional energy-related outputs to ensure values are extracted in the correct sequence when atomic, spin, and Hessian data are present together. * Prevented cases where optional fields could be read from the wrong position, improving accuracy and stability. * **Tests** * Added regression tests covering atomic-only, spin-only, Hessian-only, and combined scenarios to verify correct output splitting for each supported combination.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

…emiops (PR-C) (#5714) ## NeighborGraph PR-C — O(N) on-device graph builders (vesin / nv) Third PR of the NeighborGraph series (after #5581 foundation, #5583 PR-A dpa1 graph forward, #5604 PR-B `.pt2`/C++). Adds two **O(N) carry-all** NeighborGraph builders behind the World-2 `neighbor_graph_method` dispatcher, replacing PR-A's O(N²) `dense` search / per-frame ASE stopgap with on-device cell lists. ### What - **`build_neighbor_graph_vesin`** (`deepmd/pt_expt/utils/vesin_graph_builder.py`) — `vesin.torch` cell list. **Device-following**: runs the search on the input tensor's device (CUDA kernel on CUDA input, CPU cell list otherwise). - **`build_neighbor_graph_nv`** (`deepmd/pt_expt/utils/nv_graph_builder.py`) — nvalchemiops GPU cell list, **frame-batched** (`batch_idx`/`batch_ptr`, one kernel for all frames — no Python loop). CUDA-only. - Both structurally clone `build_neighbor_graph_ase`: search → per-frame local `(i, j, S)` → `neighbor_graph_from_ijs(...)`, which recomputes `edge_vec` **differentiably** from the original grad-carrying coords. - Wired into the pt_expt make_model graph dispatch (`neighbor_graph_method ∈ {"legacy","dense","ase","vesin","nv"}`) and DeepEval `.pt2` graph inference (new `neighbor_graph_method` kwarg, default `"dense"` → existing inference byte-identical). dpmodel/jax fail-fast on `vesin`/`nv` (torch/CUDA-only). **Perf-only:** all builders emit the SAME neighbor set as `dense` (carry-all, `sel`=normalization-only), proven by exact set-equality; energy/force/virial are unchanged (parity 1e-12 CPU / 1e-10 CUDA). ### Layering - `dpmodel` stays torch-free: vesin/nv builders live in `pt_expt`; the dpmodel dispatch only carries a fail-fast message. - vesin/nv are **optional deps, NOT in pyproject** — lazy-imported, guarded by `is_vesin_torch_available()` / `is_nv_available()`, `ImportError` with an install hint on absence. - nv decode is a faithful transcription of the tested `deepmd/pt/utils/nv_nlist.py:_matrix_to_extended_inputs` Step-1 extraction. ### Testing - **Local (CPU):** 13 passed (vesin builder 5, vesin/reject dispatch 2, DeepEval graph 6), nv self-skips. - **Remote GPU (Tesla T4), commit 78f6c24:** - `test_nv_graph_builder.py`: 4 passed (set-equality vs dense periodic+non-periodic, frame-batch, differentiable `edge_vec`). - `test_graph_builder_dispatch.py`: 3 passed on CUDA (**vesin** parity 1e-10, dpmodel reject vesin+nv, **nv** parity 1e-10). - `test_graph_deepeval.py`: 6 passed on CUDA (`.pt2` graph dense parity + vesin, AOTI compile + `torch.as_tensor` extraction). ### Known limitations - **vesin per-frame Python loop** — `vesin.torch.compute` is single-system (no batch dim), so multi-frame vesin loops over frames (each an O(N) search; `nf=1` inference has zero loop cost). **nv batches natively** — the loop-free path for batched training. - vesin/nv not in `pyproject`; no `"auto"` selector (explicit strings only). - nv `search_capacity = max(64, nloc)` initial heuristic + 1.25× grow loop (`.item()` host-sync per grow). - nv passes the *normalized* coord to `from_ijs` (differentiable lattice translation, identity gradient; `edge_vec` is lattice-invariant) — consistent with the pt nv-nlist path. - jax O(N) graph builders (matscipy/jax-md) = PR-F; attention/angles/MP = PR-D/E/G. Implements `plan_neighbor_graph_prC_implementation`; design spec: discussion wanghan-iapcm#4.  ## Summary by CodeRabbit * **New Features** * Added experimental neighbor-graph builders supporting NVIDIA CUDA (NV) and Vesin cell-list construction (vesin), including periodic shifts and frame-aware edge decoding. * **Bug Fixes** * Excluded virtual atoms (`type < 0`) from graph edges consistently for ASE-based neighbor graphs. * Improved behavior for empty/zero-neighbor inputs and ensured differentiable edge vectors. * **Tests** * Added backend parity tests (dense vs vesin/nv), NV decode regression coverage, gradient/device checks, virtual-atom exclusions, and strengthened plugin entry-point import validation.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

## Summary - remove the second qe2f multiplication when forming fix dplr efield forces - keep the existing conversion when storing constant and equal-style efield values - add real-unit regressions for constant and equal-style efield paths so qe2f != 1 is covered Closes #5646. ## Tests - git diff --check - /tmp/deepmd-check-venv/bin/ruff check . - /tmp/deepmd-check-venv/bin/ruff format --check . ## Not run - Targeted LAMMPS DPLR pytest tests could not run in the local ad hoc venv: the lammps Python module is not installed there, and pytest startup segfaulted in the temporary environment before collection.  ## Summary by CodeRabbit * **Bug Fixes** * Corrected electric-field force calculations so results are accumulated consistently in real-unit simulations. * Fixed related force and virial contributions to match expected values. * **Tests** * Added coverage for electric-field behavior in real units. * Included checks for both constant and variable field settings to confirm force results remain accurate.

## Summary - build the C package in the manylinux_2_28 CUDA 12.9 image and install TensorFlow/PyTorch from dependency groups - package the PyTorch backend plugin while excluding libtorch/CUDA runtime libraries from the tarball - add package README/download_libtorch.sh guidance and update C package tests for external PyTorch runtime - avoid CUDA 12.9 CCCL failures from -arch=all by using all-major as the CUDA 12.9 default ## Tests - ruff check . - ruff format . - git diff --check - sh -n source/install/package_c.sh - bash -n source/install/docker_package_c.sh - bash -n source/install/docker_test_package_c.sh - fork CI on bbcd58d: Build C library, Build C++, Build/upload to PyPI, CodeQL, Test C++, and Test Python all passed Fork CI: https://github.com/njzjz/deepmd-kit/actions/runs/28547373402  ## Summary by CodeRabbit * **New Features** * Expanded C library packaging and installation to support PyTorch in addition to TensorFlow and JAX. * When PyTorch is enabled, the package can generate a matching libtorch download helper and runtime setup guidance. * **Bug Fixes** * Added stronger packaging verification to ensure required shared libraries are present and to prevent accidental bundling of PyTorch/CUDA runtime libraries. * **Documentation** * Updated C library installation docs with PyTorch version/compatibility guidance and runtime environment setup (including `LD_LIBRARY_PATH`).

…5715) Implements NeighborGraph PR-D: the graph path now supports `attn_layer > 0` for dpa1/se_atten, removing the attn_layer=0-only restriction shipped in #5583. ## What - **Segment toolkit**: `segment_max` + numerically-stable, mask-aware `segment_softmax` (`deepmd/dpmodel/utils/neighbor_graph/segment.py`), built on the existing `xp_maximum_at`. - **`center_edge_pairs`** (`neighbor_graph/pairs.py`): pairs of edges sharing a center — the edge-pair axis shared with the upcoming angle machinery (PR-E). Segment-based enumeration (a global `(E,E)` boolean is deliberately avoided: `O(N²·nnei²)` memory). Two forms: compact eager (dynamic `P`, carry-all graphs) and **shape-static** (`P = n_center·nnei²`, pure arange/reshape arithmetic, no `nonzero`) for the center-major static layout — this keeps the traced/compiled/export path traceable. - **`DescrptBlockSeAtten._graph_attention`**: op-for-op ragged mirror of `GatedAttentionLayer`/`NeighborGatedAttention` — per-center `q@kᵀ` becomes per-pair `q_m·k_n`, softmax over keys becomes `segment_softmax` grouped by the query edge; head_dim QKV slicing, q/k/v normalize, temperature/scaling, smooth shift trick, post-softmax `sw` and `dotr` weighting, residual + LayerNorm per layer. - `edge_env_mat(return_sw=True)` exposes the per-edge switch (zeroed on padding) for the smooth branch. - `uses_graph_lower` widened: attention configs (concat tebd, no exclude_types) are now graph-eligible — pt_expt eager/compiled/exported paths route them through the graph lower by default. ## Numerical semantics (reviewed decision) - **Shape-static adapter path** (the dense `call` adapter, `from_dense_quartet(compact=False)` + `static_nnei`): **bit-exact vs the dense body, rtol 1e-12**, full flag matrix (attn_layer 1/2 × dotr × smooth × normalize × temperature, binding AND non-binding sel). - **Carry-all graphs**: exact for non-smooth attention. For `smooth_type_embedding=True`, the dense branch keeps sel-padding slots in the attention softmax **denominator** (weight `exp(-attnw_shift)`), which makes the dense output *depend on sel itself* (measured up to ~1e-4 with an identical physical neighbor set). The carry-all form **drops those phantom terms by design** — the sel-independent math. Pinned by a clean-divergence test; route-equivalence fixtures pin `smooth_type_embedding=False`. - se_atten_v2 (`tebd_input_mode="strip"`) remains graph-ineligible (strip mode is a later PR) — pinned by test. ## Testing - 38 new dpmodel tests (segment toolkit, pairs incl. random-vs-oracle + static-vs-compact equality, attention parity matrix, binding-sel divergence sanity). - pt_expt: `test_make_fx_graph_attn` (graph forward + autograd at attn_layer=2 traces under make_fx, both smooth branches — required since compiled training uses the graph lower); model-level graph-vs-legacy force/virial/atom-virial parity parametrized over attn_layer {0,2}. - Local CPU: common/dpmodel 583, consistent dpa1+se_atten_v2 209, pt_expt descriptor/model/utils 701 (2 failures: dpa4 export inductor error **pre-existing on upstream/master**, and a route-parity fixture fixed in-branch). - **GPU-validated (Tesla T4, cuda:0)**: dpmodel suites 38, pt_expt graph-lower/make_fx/consistency 44 (CUDA 1e-10), route-parity 6, attention AOTI export pipeline + dpa1 cross-backend consistency 105 — all passed. ## Known limitations - Strip-mode (se_atten_v2) attention stays on the dense path. - Carry-all smooth attention diverges from dense by design (see above); old behavior reachable via `neighbor_graph_method="legacy"` / explicit World-1 builders. - `num_heads == 1` assumed (dpa1 never exposes num_heads); fail-fast otherwise. - Compact `center_edge_pairs` is eager-only (`nonzero`); traced paths use the shape-static form. - 3-body angles (PR-E), jax graph force (PR-F), dpa2/3 MP (PR-G) unchanged.  ## Summary by CodeRabbit * **New Features** * Expanded graph-native attention support for additional DPA1/se_atten configurations, enabling transformer-style graph execution suitable for tracing/export. * Added center-based neighbor edge-pair enumeration with shape-static control to improve graph layout consistency. * Improved graph tracing/export with optional dynamic-shape hinting. * **Bug Fixes** * Stabilized graph attention softmax under masking/padding and ensured correct behavior for empty/no-edge cases. * **Tests** * Added/updated parity, eligibility, FX traceability, export/graph-lower, and single-atom (no edges) coverage across attention settings.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

## Summary - avoid importing TensorFlow and PyTorch unconditionally from deepmd.lmp - add runtime library paths only for installed TensorFlow and/or PyTorch backends, while preserving CUDA preload and TF<2.12 libpython preload behavior - update the pip install docs and add focused tests for missing-backend handling ## Tests - PYTHONPATH=$PWD pytest source/tests/test_lmp.py -v - ruff check . - ruff format . / commit hook ruff format - git diff --check - fork CI on njzjz/deepmd-kit branch feat/lammps-dynamic-backends: Build C library, Build C++, Build/upload to PyPI, CodeQL, Test Python, and Test C++ all passed; Mirror skipped Fork CI runs: - https://github.com/njzjz/deepmd-kit/actions/runs/28551976035 - https://github.com/njzjz/deepmd-kit/actions/runs/28551976036 - https://github.com/njzjz/deepmd-kit/actions/runs/28551976076 - https://github.com/njzjz/deepmd-kit/actions/runs/28551976053 - https://github.com/njzjz/deepmd-kit/actions/runs/28551976048 - https://github.com/njzjz/deepmd-kit/actions/runs/28551976060  ## Summary by CodeRabbit * **New Features** * Improved automatic runtime detection for LAMMPS so it can load available TensorFlow and/or PyTorch libraries more reliably at startup. * **Bug Fixes** * Better handles systems where one or more backends are missing, avoiding failed or incomplete environment setup. * Improves compatibility with older TensorFlow installs by automatically locating required Python library components. * **Documentation** * Clarified pip extras guidance for LAMMPS/i-PI, updating backend requirements wording. * **Tests** * Added Linux/macOS-only coverage to validate library-path discovery and LAMMPS environment configuration.

## Summary - add TensorFlow 2 training/freeze/compress entrypoints and trainer support - wire multi-task, validation, finetune/pretrain, TensorBoard, checkpointing, and bias adjustment paths - keep TF2 training atomic virial disabled and add performance optimizations, including DP_JIT-controlled lower-forward XLA compilation ## Validation - `python -m pytest source/tests/tf2/test_training.py -q` - `ruff check .` - `ruff format --check deepmd/tf2 source/tests/tf2` - `DP_JIT=1 srun --gres=gpu:1 dp --tf2 train input.json --skip-neighbor-stat` on `examples/water/se_e2_a` temp copy: XLA lower compiled; first step ~140s, steady windows ~0.0305 s/step through step 900 before timeout - `DP_JIT=1 srun --gres=gpu:1 dp --tf2 train input.json --skip-neighbor-stat` on fixed-selection dpa3 temp input: XLA lower compiled; first step ~227s, steady windows ~0.119 s/step through step 500 before timeout ## Notes - `DP_JIT` is opt-in for training lower forward only; the whole train/eval step is intentionally not XLA compiled because neighbor/outer training logic is too broad and previously hit unsupported XLA ops. - dpa3 lower JIT has high first-compile cost and high memory pressure, so benchmark numbers above separate warm steady-state from first-step compile cost.  ## Summary by CodeRabbit * **New Features** * Expanded TensorFlow 2 backend with new train, freeze, and checkpoint compression entry points, including full validation support. * Added multi-task model-branch selection for freeze/compress via `--head` / `--model-branch`. * Introduced optional TF2 JIT acceleration and TF2-specific automatic batch sizing. * **Bug Fixes** * Improved compression dtype handling for more consistent outputs. * Enabled additional TF2 runtime neighbor-statistics and corrected TF2 backend wiring so more workflows complete end-to-end.

## Summary Fix a JAX/Flax NNX tracing failure in `TypeEmbedNet.call()` by avoiding a hard requirement that traced values expose a readable `.device` attribute. ## Background JAX training can call `TypeEmbedNet.call()` from inside `nnx.jit` / `nnx.grad`. In that context the type-embedding weights are represented by NNX/JAX traced values such as `DynamicJaxprTracer`. The previous code passed ```python device=array_api_compat.device(sample_array) ``` to `xp.eye`, and did the same for the padding `xp.zeros`. For an NNX traced parameter, `array_api_compat.device(...)` eventually tries to read `.device`; the tracer does not provide that attribute during tracing, so the training step fails with an error like: ```text AttributeError: DynamicJaxprTracer has no attribute device ``` or, through JAX core wrapping: ```text AttributeError: 'ShapedArray' object has no attribute 'device' ``` ## Change Add a small local helper in `deepmd/dpmodel/utils/type_embed.py` that returns `array_api_compat.device(array)` when available, and falls back to `None` when the backend value has no readable device during tracing. Passing `device=None` lets JAX create the constants on its default traced device, while preserving the existing explicit-device behavior for backends that expose it normally. This keeps the change limited to the type-embedding constants that triggered the crash, rather than changing global array API device handling. ## Tests Added `source/tests/jax/test_type_embed.py` to cover both: - `TypeEmbedNet.call()` under `nnx.jit` - `TypeEmbedNet.call()` under `nnx.jit` + `nnx.grad` The old implementation fails this test during tracing before producing an output. Validation run locally: ```bash pytest source/tests/jax/test_type_embed.py -v ruff check . ruff format . ```  ## Summary by CodeRabbit * **Bug Fixes** * Improved compatibility for model execution on array backends that do not support device detection in all cases. * Reduced failures when padding or initializing arrays during type embedding operations. * **Tests** * Added JAX coverage to verify the type embedding model runs under JIT, supports gradient tracing, returns the expected output shape, and produces valid numeric results.

## Summary - add a small helper to resize and zero the DPLR PPPM fele cache - clear fele before returning from PPPMDPLR::compute when qsqsum == 0 - reuse the same helper in init and fieldforce paths This addresses #5647 as a defensive consistency fix. In normal DeePMD DPLR usage, atom charges are fixed and are not changed from nonzero to all-zero during a run, so this stale-cache path should not be triggered by standard DeePMD workflows. It would only matter for unusual external charge mutation or reinitialization paths where qsqsum is refreshed to zero after a previous nonzero-charge PPPM/DPLR step. ## Tests - git diff --check - /tmp/deepmd-check-venv/bin/ruff check . - /tmp/deepmd-check-venv/bin/ruff format --check .  ## Summary by CodeRabbit * **Bug Fixes** * Improved consistency in per-atom field/force handling by clearing the accumulator before early exits and during setup. * Reduced the chance of stale values carrying over between calculations. * **Refactor** * Consolidated repeated zeroing and resizing steps into a shared internal routine, simplifying the calculation flow and making the code easier to maintain.

Since each commits relies on each other to pass the parity tests, so this is a fusion pr.  ## Summary by CodeRabbit * **New Features** * Added `readout_layers` and native per-atom spin support across SeZM/DPA4 descriptors, including new `spin` runtime input and spin-aware scalar readout. * Introduced a public `SpinEmbedding` component for the native spin scheme. * Added `"fndc"` tensor-layout support and expanded optional fused Triton acceleration (including flash-attention-style aggregation) for supported configurations. * **Bug Fixes** * Improved descriptor behavior for zero-block/empty-edge cases; strengthened full-validation profile handling and relaxed spin label requirements when allowed. * Reduced evaluation logging verbosity and enabled AMP autocast during inference when opted in. * **Compatibility** * Model deserialization now recognizes `sezm_native_spin`. * Removed an older experimental CuTe rotation path.  --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

## Summary - switch lightweight label, mirror, TODO, and pass aggregation jobs to ubuntu-slim - switch the Python test duration aggregation job to ubuntu-slim - keep build and test execution jobs on their existing runners ## Validation - python YAML parse for all workflow files - git diff --check - ruff check . Note: ruff format --check . still reports pre-existing formatting in dpa_adapt/cli.py, which is unrelated to this workflow-only change.  ## Summary by CodeRabbit * **Chores** * Updated several GitHub Actions jobs to run on a slimmer Ubuntu runner. * Applied this environment change across build, test, packaging, labeling, mirroring, and maintenance workflows. * No workflow steps, dependencies, or job logic were changed.

## Problem The Paddle `aggregate()` helper in `deepmd/pd/model/network/utils.py` allocated its output with `paddle.zeros([num_owner, data.shape[1]])` without specifying a dtype, so it fell back to Paddle's default floating dtype (float32). It then cast the input `data` to `output.dtype` before `index_add_`. For float64 Paddle RepFlow/DPA3 models, the dynamic-selection aggregation therefore accumulated descriptor updates in float32, silently downcasting intermediate values before returning them to the descriptor path. ## Fix Allocate the output with `dtype=data.dtype` so the input precision is preserved (and the subsequent `data.astype(output.dtype)` becomes a no-op for matching dtypes). ## Test A new test aggregates float64 input and asserts the result stays float64 (and has the expected values). On the current code the output is float32; after the fix it is float64. This exercises the shared `output = paddle.zeros(..., dtype=data.dtype)` allocation via the summation path. ## Note on verification Verified locally with `paddlepaddle==3.3.1`. The CI target is a newer nightly (`paddlepaddle==3.4.0.dev20260310`), but the behavior fixed here is version-agnostic: `paddle.zeros` without `dtype` defaults to float32 in all versions, and `dtype=data.dtype` corrects it regardless. The test is scoped to the summation path to keep it independent of an unrelated `Tensor.where` API difference in the older local Paddle. Fix #5688  ## Summary by CodeRabbit * **Bug Fixes** * Preserved the input tensor’s numeric dtype when aggregating values, preventing unintended dtype fallback during summation. * **Tests** * Added unit coverage to confirm aggregation keeps `float64` output dtype and returns the expected summed values.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Problem Fixes #5679. `RunOptions` records `dp train --init-model` as `init_mode == "init_from_model"` (`deepmd/tf/train/run_options.py`), but `DPTrainer.build()` dispatched the init step on the literal `"init_model"`. The two strings never matched, so `_init_from_ckpt(...)` was skipped for `--init-model`. That pre-inspection is the step that imports the source checkpoint's meta graph and, when the checkpoint is a `compressed_model`, sets `self.ckpt_meta` before the graph is built (the graph build later consumes `ckpt_meta`). With the mismatch, a compressed-checkpoint `--init-model` run builds the graph without its checkpoint metadata. The bug was masked for the common case because uncompressed `--init-model` still works: the variables are restored later in `_init_session`, which uses the correct `"init_from_model"` literal and does not need `ckpt_meta`. So only compressed-checkpoint initialization was actually exposed, and no test exercised it. ## Fix Correct the dispatch literal to `"init_from_model"`. To make the dispatch unit-testable — the reason the mismatch went uncaught is that it lived inline in the heavyweight `build()` — the four-way init dispatch is extracted into a small `_init_from_run_opt()` helper. The restart, init-from-frozen-model, and finetune branches already used the correct literals and are unchanged in behavior. ## Test Adds `source/tests/tf/test_trainer_init_mode.py`, which drives `_init_from_run_opt` on a stub trainer with the three concrete initializers mocked and asserts each `init_mode` routes correctly. On the old literal, `init_from_model` routes to nothing (the test fails); with the fix it reaches `_init_from_ckpt(init_model)`. The test also covers `restart`, `init_from_frz_model`, `finetune`, and the scratch no-op. This dispatch previously had no coverage.  ## Summary by CodeRabbit * **Bug Fixes** * Improved training startup by centralizing initialization dispatch, ensuring the correct checkpoint/frozen-model/pretrained initialization path is selected for restore, restart, fine-tuning, and frozen-model workflows. * If training is configured for “scratch” (or an unsupported init mode), initialization pre-inspection steps are skipped and startup proceeds. * **Tests** * Added regression tests covering each supported initialization path to prevent future startup regressions.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Problem `EwaldRecp` creates a dedicated `tf.Session` in its constructor but exposes no way to close it (no `close()`, context-manager path, or destructor). `DipoleChargeModifier` creates and keeps an `EwaldRecp` instance without releasing it either. Creating and discarding these objects therefore leaks TensorFlow sessions and graph resources; long-running processes that repeatedly construct modifiers accumulate them until process exit. ## Fix Add `close()` (idempotent, guarded), context-manager support (`__enter__`/`__exit__`), and a defensive `__del__` to `EwaldRecp`, and have `DipoleChargeModifier.close()` forward to the held `EwaldRecp`. ## Test `test_ewald.py` previously exercised only `eval`, never lifecycle. New tests assert that `EwaldRecp.close()` and context-manager exit release the session (a subsequent `eval` raises on the closed session), and that `DipoleChargeModifier.close()` forwards to its `EwaldRecp.close()`. All three fail on the current code with `AttributeError` (no such methods) and pass after the fix. Fix #5685  ## Summary by CodeRabbit * **New Features** * Added explicit resource cleanup and context-manager support for TensorFlow evaluators, including Ewald reciprocal, TensorFlow DeepEval backends, and the DeepEval wrapper. * Added `close()` forwarding for the dipole-charge modifier to release its associated evaluator resources. * **Tests** * Added unit tests validating `close()` behavior, context-manager exit, and error handling after sessions are released for Ewald reciprocal and DeepEval (including wrapper forwarding and no-op behavior when nothing is cached). * Added a test ensuring dipole-charge modifier cleanup forwards correctly.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

## Problem Fixes #5680. The legacy TensorFlow spin implementation assumes spin-enabled types form a contiguous prefix of the type map. The SE-A `sel` extension takes the first `ntypes_spin` selections (`sel_a[:ntypes_spin]`), and the coordinate splitting (`deepmd/tf/descriptor/se_a.py`), force splitting (`deepmd/tf/model/ener.py`), and bias merging (`deepmd/tf/fit/ener.py`) all address the virtual block with a dense real-to-virtual offset (`i + len(use_spin)`). For a non-prefix layout such as `use_spin=[False, True]`, these read the wrong real/virtual type ranges or raise deep inside the graph, and nothing rejected the configuration up front. In practice all supported spin models list spin-enabled types first, so the broken layout went unnoticed. ## Approach The maintained PyTorch backend already handles the sparse/non-prefix layout via `Spin.spin_type`. Rather than refactor all four coupled sites in this legacy backend (which has almost no coverage and where a subtle mistake would silently corrupt training), this guards against the unsupported layout: the TF `Spin` helper now rejects a `use_spin` where a non-spin type precedes a spin-enabled one, with a message telling the user to list spin-enabled types first. This turns a silent-wrong result or an obscure crash into an actionable error and documents the invariant in one place. ## Test Adds `source/tests/tf/test_spin_prefix_guard.py`: non-prefix layouts (`[False, True]` and `[True, False, True]`) raise `ValueError`, while prefix layouts (`[True, False]`, `[True, True]`) and an all-non-spin list (`[False, False]`) are accepted. The existing TF spin model test uses a valid `[True, False]` prefix and continues to pass. The prefix requirement previously had no test. Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

…#5747) ## Summary Adds NeighborGraph (graph-native lower) support for the dpa1 descriptor with `tebd_input_mode="strip"`, closing the last descriptor-level gap that forced strip-mode models (and `se_atten_v2`, which is strip-by-construction) onto the legacy dense path. The dense strip branch factorizes the per-neighbor feature as `gg = gg_s*gg_t + gg_s` — a radial-only geometric embedding times a type-pair strip embedding (optionally switch-smoothed). Because this has **no neighbor-axis coupling**, it maps to the graph path edge-for-edge. The change is: - **Kernel** (`DescrptBlockSeAtten`): a new per-edge helper `_graph_edge_gg_strip` (op-for-op mirror of the dense strip branch, including the `center*ntypes_pad + nei` nei-fastest two-side table layout and `int64` gather indices), selected by a `concat`/`strip` branch in `call_graph`. - **Routing** (`DescrptDPA1.uses_graph_lower`): admits `strip`, while keeping compressed descriptors and `exclude_types` on the dense path (they have no graph kernel here). `se_atten_v2` inherits this and becomes graph-eligible for free. - **pt_expt**: the two graph `make_fx` export tests are parametrized over `tebd_input_mode` to prove the strip kernel is fx-traceable. No new op, no attention/`segment_sum` change, no C++/serialization change. ## Scope This PR is deliberately **independent of #5733** (graph `exclude_types`). It does **not** change `exclude_types` eligibility — the `uses_graph_lower` `and not exclude_types` gate and the `call_graph` `exclude_types` raise are both kept. When both land, whichever merges second resolves a small (2–3 line) mechanical conflict at `uses_graph_lower` / the `call_graph` guard. ## Test plan - Block-level graph-vs-dense strip parity at `rtol=atol=1e-12` over `type_one_side × smooth` (attn=0) and `type_one_side` (attn=2, non-smooth). - Descriptor-level routed-`call` vs `_call_dense` parity over `type_one_side × smooth × attn_layer` (incl. attn=2 + smooth=True, bit-exact via the `static_nnei` adapter), plus a negative-contract gate test (compressed → dense, strip+`exclude_types` → dense). - `se_atten_v2` eligibility + graph-vs-dense parity (replaces the obsolete "strip stays dense" test). - pt_expt strip `make_fx` export; cross-backend consistency strip cases now route pt_expt through the graph adapter. Validated on **CPU** and on **GPU (Tesla T4, cuda:0)**: pt_expt dpa1 50 passed, consistency strip 22 passed + `se_atten_v2` 110 passed, dpmodel strip suites 46 passed. No tolerances relaxed, no tests skipped. ## Known limitations - **Compression** stays on the dense path by design (strip-only tabulation has no graph kernel); the gate excludes `self.compress`. - **`exclude_types`** stays dense (out of scope — owned by #5733). - **jax** graph lower remains energy-only (analytical force on the graph route is a separate follow-up). - The graph path's `segment_sum`→`index_add` is atomic/non-deterministic on CUDA (1–2 fp64 ULP), inherent to atomic scatter; GPU parity validated within tolerance. - Pre-existing (not introduced here): a softmax `RuntimeWarning` on the shared attention path (max over fully-masked segments), also present on the concat path.  ## Summary by CodeRabbit ## Summary * **New Features** * Added graph-based execution support for the additional stripped type-embedding mode. * **Bug Fixes** * Updated graph routing eligibility: compressed descriptors and excluded-type configurations now reliably fall back to dense execution; graph routing is disabled when compression is enabled. * **Tests** * Added bit-exact parity tests between graph and dense paths for the new stripped mode (including routing eligibility checks). * Expanded FX graph export/trace coverage for both embedding modes. * Adjusted neighbor-list fallback validation with model-specific tolerance handling and added a new smooth variant.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

This pull request introduces support for handling `charge_spin` information in atomic and descriptor models. The main changes add new methods to query and provide default `charge_spin` values, update the `forward` methods to accept a `charge_spin` argument, and refactor code to use this argument consistently instead of the previous `fparam` field. Additionally, the DPA3 descriptor now supports default `charge_spin` values and serializes them. **Support for charge_spin in descriptor and atomic models:** * All descriptor classes (`dpa1.py`, `dpa2.py`, `dpa3.py`, `se_a.py`, `se_t_tebd.py`) now implement `get_dim_chg_spin`, `has_default_chg_spin`, and `get_default_chg_spin` methods to standardize how charge and spin information is queried. [[1]](diffhunk://#diff-677046c6055a32ff3ba8b808f69dfd32f04ab7108ec69588fff160d490920ac3R369-R380) [[2]](diffhunk://#diff-8793f48d255ddb141f53ac3debae3affd7eb19b15d4fcc3ba3868f4c191f6cb5R336-R347) [[3]](diffhunk://#diff-feb207b49daf67c5e9dc55c32caeae21e115e9d248c4f6fcf996a6c3aa087e92R456-R467) [[4]](diffhunk://#diff-f2c319589636e32eaaa6bb991e88b5646bf3124703e35c7fe4bdbcd7589d7506R122-R133) [[5]](diffhunk://#diff-92dbb0218ed79895ee9c480f74a33a6db32acbccce418fa350b05936c9480805R193-R204) * The `forward` methods in all descriptor classes, as well as in the atomic model (`dp_atomic_model.py`), now accept a `charge_spin` argument instead of (or in addition to) `fparam`, and all internal logic is updated to use `charge_spin`. [[1]](diffhunk://#diff-677046c6055a32ff3ba8b808f69dfd32f04ab7108ec69588fff160d490920ac3R642) [[2]](diffhunk://#diff-8793f48d255ddb141f53ac3debae3affd7eb19b15d4fcc3ba3868f4c191f6cb5R749-L740) [[3]](diffhunk://#diff-feb207b49daf67c5e9dc55c32caeae21e115e9d248c4f6fcf996a6c3aa087e92L544-R563) [[4]](diffhunk://#diff-f2c319589636e32eaaa6bb991e88b5646bf3124703e35c7fe4bdbcd7589d7506R304) [[5]](diffhunk://#diff-92dbb0218ed79895ee9c480f74a33a6db32acbccce418fa350b05936c9480805R453) [[6]](diffhunk://#diff-d96cfb6a75d54b57d71cf1599e1f64a79d9862353a7663c674e6f0343817f178L335-R335) **DPA3 descriptor enhancements:** * The DPA3 descriptor now supports a `default_chg_spin` parameter, validates its shape, and serializes it as part of its configuration. [[1]](diffhunk://#diff-feb207b49daf67c5e9dc55c32caeae21e115e9d248c4f6fcf996a6c3aa087e92R125) [[2]](diffhunk://#diff-feb207b49daf67c5e9dc55c32caeae21e115e9d248c4f6fcf996a6c3aa087e92R181-R185) [[3]](diffhunk://#diff-feb207b49daf67c5e9dc55c32caeae21e115e9d248c4f6fcf996a6c3aa087e92R486) * The logic for embedding charge and spin in DPA3 is updated to use the new `charge_spin` argument. These changes standardize how charge and spin information is handled across descriptors and atomic models, making the codebase more extensible and robust for future features involving charge and spin.  ## Summary by CodeRabbit * **New Features** * Added optional `charge_spin` (nf×2) support across atomic, descriptor, energy, and full model forward pipelines, including the model wrapper interface. * Descriptors and models now expose charge/spin capability and default value queries; DPA3 can use a built-in default when `charge_spin` is omitted. * Training input handling now declares, gates, and supplies `charge_spin` when supported. * **Tests** * Updated descriptor and DPA3 consistency coverage to use `charge_spin` instead of the prior parameter, and broadened backend verification.  --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: HydrogenSulfate <23737287+HydrogenSulfate@users.noreply.github.com>

…5722) ## Problem Fixes #5674. Three related defects in global DOS inference, uncovered in sequence: 1. `DeepDOS.eval` unconditionally read the atomic `dos` output and summed it, even for `atomic=False`. The dpmodel and JAX backends only return the atomic output when `atomic=True`; on the global-DOS-only path (e.g. `dp test` without atomic DOS labels) `results["dos"]` raised `KeyError`. TF and PyTorch masked this because they always include the atomic output. 2. Fixing the `KeyError` exposed that dpmodel and JAX DOS inference was broken more deeply: both `DeepEval.get_numb_dos` implementations hard-returned `0`, so the DOS reshape target was `(nframes, 0)` and inference failed on every path, not just the missing-key case. 3. On TF, the global DOS did not equal the sum of the atomic DOS for multi-frame inputs — even though, by construction of the model, it must. `deepmd/tf/model/dos.py` reduced the atomic DOS with `reshape([natoms[0], -1])` + `reduce_sum(axis=0)`, which sums across the wrong axis and mixes atoms from different frames together. Single-frame inputs happened to give the right answer, so no test caught it. ## Fix Backend-agnostic (`deep_dos.py`): prefer the atomic `dos` output and sum it whenever the backend returns it (this is the exact global DOS on TF/PT, whose reduced output is not necessarily the plain sum), and fall back to the reduced `dos_redu` only when the atomic output is absent (dpmodel/JAX at `atomic=False`). Reading `dos` unconditionally is what raised the original `KeyError`. dpmodel: add `get_numb_dos` to the dpmodel `DOSModel` (mirroring the PyTorch model), add a default `get_numb_dos` returning 0 on the shared base model so non-DOS models can still be serialized, and delegate `dpmodel/infer/deep_eval.py:get_numb_dos` to the model. JAX: the evaluator wraps a deserialized `HLO` object with no live model, so `numb_dos` is now persisted into the StableHLO export constants and exposed via `HLO.get_numb_dos`; the `dos` output is registered in the HLO `OUTPUT_DEFS` table; and `jax/infer/deep_eval.py:get_numb_dos` delegates to the model. With these, JAX DOS inference works end to end. TF: reduce the atomic DOS per frame — `reshape([-1, natoms[0], numb_dos])` + `reduce_sum(axis=1)`, mirroring the energy model — so the global DOS equals the atomic sum for multi-frame inputs. ## Test - `source/tests/common/dpmodel/test_deep_dos.py`: builds a dpmodel DOS model and evaluates it — `atomic=False` returns the global DOS (`KeyError` on master), and the global DOS equals the sum of the atomic DOS (guarding the `dos_redu == sum(dos)` invariant relied on by all backends). - `source/tests/jax/test_deep_dos.py`: exports a DOS model to `.hlo`, checks `numb_dos` survives the round trip, and evaluates the global DOS. - `source/tests/tf/test_model_dos.py`: adds `test_multiframe_global_equals_atomic_sum`, which builds a two-frame DOS graph and asserts the global DOS equals the per-frame atomic sum — this fails on the old axis-0 reduction and passes after the per-frame fix. The existing single-frame assertions were updated to the corrected output shapes. dpmodel and JAX DOS inference previously had no test, and the TF path had only single-frame coverage; DOS was effectively exercised only where the bugs were masked.  ## Summary by CodeRabbit * **Bug Fixes** * Corrected DOS inference so global DOS is reported properly instead of using a fixed default. * Fixed multi-frame DOS aggregation to keep results separated by frame and sum across atoms correctly. * Improved consistency when using the model in different backends and after export, so DOS output counts are preserved. * **New Features** * Added support for exposing DOS output counts in model inference and export workflows.  --------- Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

github-advanced-security AI found potential problems Jan 23, 2024

View reviewed changes

njzjz deleted the branch jameswind:master January 6, 2026 05:54

njzjz deleted the master branch January 6, 2026 05:54

pre-commit-ci Bot and others added 27 commits March 10, 2026 11:15

wanghan-iapcm and others added 30 commits July 1, 2026 15:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

merge from origin#2

merge from origin#2
jameswind wants to merge 3467 commits into
jameswind:masterfrom
deepmodeling:master

jameswind commented Jun 27, 2020

Uh oh!

github-advanced-security AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

17 participants

Conversation

jameswind commented Jun 27, 2020

Uh oh!

github-advanced-security AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

17 participants