Sudden Increase in RMSE_e During Initial Training

[XiangYu-Ji001]

DeePMD-kit Version：
DeePMD-kit v3.0.3

Problem Description:
A sudden increase in RMSE_e is observed during the initial training process. The system consists of an aqueous NaClO₄ aqueous solution with 339 atoms. Training was conducted on 1000 frames from a single OUTCAR file, with 800 frames for training and 200 for testing. Three separate initial training runs were performed, RMSE_e showed sharp increases during or near the end of long training sequences. Is this situation normal? Could it be caused by inappropriate learning rate or loss function settings? For ease of review, I’ve attached the input.json file.
I confirm that the training set data are fully converged. The VASP parameters are set as follows: ENCUT = 400 eV, EDIFF = 1 × 10⁻⁴ eV, SIGMA = 0.2, and only the Gamma point is used for k-point sampling (simulation box side length ~15 Å).

input.json

[XiangYu-Ji001]

The issue has been resolved, which was caused by the low accuracy of AIMD used in the previous training set. After modifying the INCAR parameters to Ecut=600, SIGMA=0.05, EDIFF=5E-6 with 2*2*2 Kpoints, no sudden Increase were observed during the training process.