Skip to content

Add NVT post-processing for TDEP input generation#698

Open
junwen94 wants to merge 15 commits into
stfc:mainfrom
junwen94:dev/tdep
Open

Add NVT post-processing for TDEP input generation#698
junwen94 wants to merge 15 commits into
stfc:mainfrom
junwen94:dev/tdep

Conversation

@junwen94
Copy link
Copy Markdown

Summary

This PR adds support for generating standard TDEP input files from NVT molecular dynamics runs in janus-core.

The new workflow allows TDEP inputs to be generated automatically through post_process_kwargs during an NVT simulation, rather than requiring a separate manual conversion step.

Changes

  • add a new TDEP processing module in janus_core/processing/tdep.py
  • add build_tdep_inputs_from_nvt for generating standard TDEP input files
  • integrate TDEP input generation into MD post-processing
  • extend PostProcessKwargs with TDEP-related options
  • add integration test coverage in tests/test_tdep.py
  • add a new Python tutorial notebook for anharmonic phonons in docs/source/tutorials/python/anharmonic_phonons.ipynb
  • register the new tutorial in docs/source/tutorials/python/index.rst

Notes

  • this is an initial integration for the NVT workflow
  • the generated files are infile.ucposcar, infile.ssposcar, infile.positions, infile.forces, infile.stat, and infile.meta

Testing

uv run pytest tests/test_tdep.py -q

Comment thread janus_core/processing/tdep.py Outdated
Comment thread janus_core/processing/tdep.py
Comment thread janus_core/processing/tdep.py Outdated
Comment thread janus_core/processing/tdep.py Outdated
Comment thread janus_core/processing/tdep.py Outdated
Comment thread janus_core/processing/tdep.py Outdated
Comment thread janus_core/processing/tdep.py Outdated
@junwen94 junwen94 requested a review from oerc0122 April 21, 2026 11:08
@junwen94
Copy link
Copy Markdown
Author

Addressed Jacob's review comments and pushed an update. Verified with uv run pytest tests/test_tdep.py -q.

oerc0122
oerc0122 previously approved these changes Apr 21, 2026
Copy link
Copy Markdown
Collaborator

@oerc0122 oerc0122 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor recommendations, otherwise looks ok.

Comment thread janus_core/processing/tdep.py Outdated
f"{n_atoms} # N atoms\n"
f"{n_steps} # N timesteps\n"
f"{timestep} # timestep in fs\n"
f"{temperature} # temperature in K\n",
Copy link
Copy Markdown
Collaborator

@oerc0122 oerc0122 Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can still avoid more repetition here:

        f"""\
{n_atoms}    # N atoms
{n_steps}    # N timesteps
{timestep}    # timestep in fs
{temperature}    # temperature in K
        """,

Comment thread janus_core/processing/tdep.py Outdated
Comment thread janus_core/processing/tdep.py Outdated
Comment on lines +128 to +139
f"{time_fs[i]:.12f} "
f"{e_tot[i]:.12f} "
f"{e_pot[i]:.12f} "
f"{e_kin[i]:.12f} "
f"{temperature_data[i]:.12f} "
f"{pressure[i]:.12f} "
f"{stress_xx[i]:.12f} "
f"{stress_yy[i]:.12f} "
f"{stress_zz[i]:.12f} "
f"{stress_xz[i]:.12f} "
f"{stress_yz[i]:.12f} "
f"{stress_xy[i]:.12f}\n"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise with the above modification

            f"{i:d} " +
            " ".join(
                out_fmt(dat[i]) for dat in
                (
                    time_fs,
                    e_tot,
                    e_pot,
                    e_kin,
                    temperature_data,
                    pressure,
                    stress_xx,
                    stress_yy,
                    stress_zz,
                    stress_xz,
                    stress_yz,
                    stress_xy,
                )
            ) + "\n"

@alinelena
Copy link
Copy Markdown
Member

be sure we add tdep in CI and we actually test it

Copy link
Copy Markdown
Member

@ElliottKasoar ElliottKasoar Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally these are run automatically via nbsphinx, so we don't usually save the output files (e.g. the png) or any of the notebook cell outputs, and it means we can test this all works when we build for the docs these on release.

Is there any reason not to do the same for this?

Comment thread .gitignore Outdated
Copy link
Copy Markdown
Member

@ElliottKasoar ElliottKasoar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, @junwen94!

Would you be able to add a (very brief) CLI tutorial for this too?

Related to that, it would be great to include these kwargs in the example configs found in docs/source/examples/full/, similarly to how we already show rdf_compute etc. This provides another example for how you can specify these.

Co-authored-by: Jacob Wilkins <46597752+oerc0122@users.noreply.github.com>
@alinelena
Copy link
Copy Markdown
Member

@junwen94 can you go through the comments please? and also we will need to install tdep in the ci to have a proper test, in the sense tdep runs and we get to test the real data.

junwen94 and others added 6 commits May 22, 2026 13:24
janus_core/processing/tdep.py (oerc0122 round 2):
- Fix syntax error in positions writer and define out_fmt = '{:.12f}'.format
- Collapse meta_path.write_text into a single triple-quoted f-string
- Refactor stat_path block to iterate data tuple through out_fmt

docs/source/tutorials/python/anharmonic_phonons.ipynb (ElliottKasoar):
- Clear all cell outputs and execution counts
- Drop the static dispersion PNG; notebook now runs TDEP via !-shell calls
  and embeds the gnuplot-generated PDF (converted via pdftoppm)
- Add nbsphinx execute=always + timeout metadata so this notebook is the
  only one executed at docs build time (global default remains 'never')

docs/source/examples/full/nvt.yml (ElliottKasoar):
- Add tdep_compute / tdep_unit_cell_file / tdep_supercell_file /
  tdep_output_dir under post_process_kwargs, mirroring rdf_compute

.gitignore (ElliottKasoar):
- Remove ! exceptions for the tutorial notebook and dispersion PNG

.github/workflows/ci.yml (alinelena):
- Install TDEP via conda-forge per upstream INSTALL.md and build with
  build_things.sh; cache the build under ~/tdep
- Add poppler-utils for pdftoppm
- Bump docs job timeout to 30 min and switch shell to bash -el
The previous run produced no PDF because gnuplot was not on the CI runner.
Use gnuplot-nox to avoid pulling in X11/Qt. Also print produced TDEP
outputs at the end of the !-shell pipeline to aid future debugging.
Previous run still failed at Image cell, meaning a silent failure earlier
in the TDEP pipeline. Bash with set -ex aborts on first error and echoes
each command, so the next CI run will show exactly which step is failing.
TDEP libolle/lo_phonon_bandstructure_on_path.f90 appends a _pdf suffix
when pdfplot is true (the -p / --pdf flag), so the script TDEP actually
writes is outfile.dispersion_relations.gnuplot_pdf, not .gnuplot.

Also drop -x from set -ex now that the failure is identified; keep -e
for fail-fast on future regressions.
Comment thread .github/workflows/ci.yml
@@ -83,11 +83,15 @@

docs:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned this is being done on the docs workflow.

Copy link
Copy Markdown
Member

@ElliottKasoar ElliottKasoar May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case it's not clear, @junwen94, the docs that are built during ci.yml (which runs on every push, and for every PR) do not run the notebook tutorials (make all), they just check the docs build from the .rst files (make html).

The tutorials will be run as part of the publish-on-pypi.yml workflow, which is triggered on new tagged releases, so you should still test locally that both make commands work.

You will need installations in ci.yml, but only for the tests section of the workflow, as we have for PLUMED (assuming, as I suggest elsewhere, we actually run tdep, and not just test the input generation).

Comment thread .github/workflows/ci.yml
Comment on lines 103 to +118
python-version: "3.12"

- name: Install pandoc
uses: pandoc/actions/setup@v1

- name: Install dependencies
- name: Install poppler-utils and gnuplot
shell: bash
run: sudo apt-get update && sudo apt-get install -y poppler-utils gnuplot-nox

- name: Set up Miniconda for TDEP
uses: conda-incubator/setup-miniconda@v3
with:
miniconda-version: latest
auto-update-conda: true
activate-environment: tdep
python-version: "3.10"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very concerned that we not only have two package managers, but two different versions of python simultaneously. That's a recipe for disaster.

Comment thread .github/workflows/ci.yml
- name: Install dependencies
- name: Install poppler-utils and gnuplot
shell: bash
run: sudo apt-get update && sudo apt-get install -y poppler-utils gnuplot-nox
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to add gnuplot? Poppler utils? Are these needed for users? none of this is documented anywhere.

Comment thread tests/test_tdep.py
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ideally we want to (optionally) test more than just the input generation.

With PLUMED, for example, we have tests/test_md_plumed.py, which is skipped if it is not installed correctly, but if it is (which we can set up in the CI), then we check it is actually integrated correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants