fix: skip empty frame properties when dumping deepmd format (#977) (#982)

## Summary

Fixes #977.

When an ABACUS SCF run converges but does **not** compute forces/stress
(e.g. `cal_force`/`cal_stress` disabled — which is what users hit with
the new GPU `cusolver` `ks_solver`), dpdata's ABACUS parser correctly
produces a `LabeledSystem` with size-0 forces (this is an intended
state, covered by `TestABACUSLabeledOutputNoFS`).

The bug is in the **deepmd dump→load round-trip**:

- `dump` did `np.reshape(forces, [nframes, -1])`, which for empty forces
writes `force.npy` of shape `(nframes, 0)`.
- On load, `np.reshape((nframes, 0)-array, [nframes, natoms, 3])`
raises:

  ```
  ValueError: cannot reshape array of size 0 into shape (20,64,3)
  ```

This is exactly the traceback reported in #977 (surfacing in dpgen's
`post_fp_check_fail`).

## Root cause investigation

- ABACUS (LTSv3.10.0) prints forces via `ModuleIO::print_force`
independent of the KS solver; `cusolver` fills the wavefunctions just
like `genelpa`/`scalapack_gvx`, and forces are computed from the density
matrix afterward. There is no code path that disables force output for
`cusolver`/`device gpu`.
- dpdata parses the v3.10.0 `FmtTable` force block correctly **when
present**.
- The crash therefore only requires a converged log with **no
`TOTAL-FORCE` block**, which dpdata represents as empty forces — and the
deepmd round-trip then corrupts it.

## Fix

In both `dpdata/formats/deepmd/comp.py` (npy) and
`dpdata/formats/deepmd/raw.py` (raw) `dump`, skip an optional frame
property whose array is empty while the system still has frames, rather
than writing a meaningless `(nframes, 0)` array:

```python
if nframes > 0 and np.asarray(data[dtype.name]).size == 0:
    continue
```

A missing `force`/`virial` file is already a supported state on load
(`_cond_load_data` returns `None` and skips it), so the round-trip is
now consistent.

## Tests

Added two tests to `TestABACUSLabeledOutputNoFS` (reusing the existing
`INPUT.ch4-noforcestress` fixture):

- `test_noforcestress_deepmd_roundtrip`
- `test_noforcestress_deepmd_raw_roundtrip`

Both fail on `master` with the reported `ValueError` and pass with this
change. abacus + deepmd comp/raw/mixed/empty suites pass (312 tests); no
new failures in the full suite.

## Out of scope (user/dpgen side)

This makes dpdata robust to force-less ABACUS results. Producing usable
training labels still requires `cal_force 1`/`cal_stress 1` in the
generated `INPUT` and adding `cusolver` to dpgen's `ks_solver`
whitelist.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Bug Fixes
* Fixed an issue where empty optional properties (such as forces and
virials) were incorrectly exported as invalid empty arrays in DeePMD
format files. These properties are now properly skipped during export
when data is unavailable.

## Tests
* Added validation tests for proper handling of missing optional
properties during DeePMD format conversions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>

Author: wanghan-iapcm

dpdata/formats/deepmd/comp.py

@@ -152,6 +152,12 @@ def dump(folder, data, set_size=5000, comp_prec=np.float32, remove_sets=True):
                 f"Shape of {dtype.name} is not (nframes, ...), but {dtype.shape}. This type of data will not converted to deepmd/npy format."
             )
             continue
+        if nframes > 0 and np.asarray(data[dtype.name]).size == 0:
+            # an optional frame property (e.g. forces/virials when
+            # cal_force/cal_stress is disabled) may be empty while the
+            # system still has frames. Skip it instead of writing a
+            # meaningless (nframes, 0) array that cannot be reshaped on load.
+            continue
         ddata = np.reshape(data[dtype.name], [nframes, -1])
         if np.issubdtype(ddata.dtype, np.floating):
             ddata = ddata.astype(comp_prec)

dpdata/formats/deepmd/raw.py

@@ -136,5 +136,11 @@ def dump(folder, data):
                 f"Shape of {dtype.name} is not (nframes, ...), but {dtype.shape}. This type of data will not converted to deepmd/raw format."
             )
             continue
+        if nframes > 0 and np.asarray(data[dtype.name]).size == 0:
+            # an optional frame property (e.g. forces/virials when
+            # cal_force/cal_stress is disabled) may be empty while the
+            # system still has frames. Skip it instead of writing a
+            # meaningless (nframes, 0) array that cannot be reshaped on load.
+            continue
         ddata = np.reshape(data[dtype.name], [nframes, -1])
         np.savetxt(os.path.join(folder, f"{dtype.deepmd_name}.raw"), ddata)

tests/test_abacus_pw_scf.py

@@ -2,6 +2,7 @@
 
 import os
 import shutil
+import tempfile
 import unittest
 
 import numpy as np
@@ -163,6 +164,34 @@ def test_noforcestress_job(self):
         # test append self
         system_ch4.append(system_ch4)
 
+    def test_noforcestress_deepmd_roundtrip(self):
+        # a converged scf without force/stress should survive a
+        # round-trip through deepmd/npy without raising a reshape error
+        system_ch4 = dpdata.LabeledSystem("abacus.scf", fmt="abacus/scf")
+        tmp_dir = tempfile.mkdtemp()
+        try:
+            system_ch4.to("deepmd/npy", tmp_dir)
+            reloaded = dpdata.LabeledSystem(tmp_dir, fmt="deepmd/npy")
+            self.assertEqual(reloaded.get_nframes(), system_ch4.get_nframes())
+            # empty force/virial should not be written as bogus data
+            self.assertFalse(reloaded.data.get("forces", np.empty(0)).size)
+            self.assertTrue("virials" not in reloaded.data)
+        finally:
+            shutil.rmtree(tmp_dir)
+
+    def test_noforcestress_deepmd_raw_roundtrip(self):
+        # same as above but for the deepmd/raw format
+        system_ch4 = dpdata.LabeledSystem("abacus.scf", fmt="abacus/scf")
+        tmp_dir = tempfile.mkdtemp()
+        try:
+            system_ch4.to("deepmd/raw", tmp_dir)
+            reloaded = dpdata.LabeledSystem(tmp_dir, fmt="deepmd/raw")
+            self.assertEqual(reloaded.get_nframes(), system_ch4.get_nframes())
+            self.assertFalse(reloaded.data.get("forces", np.empty(0)).size)
+            self.assertTrue("virials" not in reloaded.data)
+        finally:
+            shutil.rmtree(tmp_dir)
+
 
 if __name__ == "__main__":
     unittest.main()