Conversation
* feat: add NVE energy-conservation benchmark Port the NVE energy-conservation benchmark from the internal repo. It runs short microcanonical (NVE, velocity-Verlet, no thermostat) MD trajectories for four representative systems (vacuum molecule, bulk water, two solvated peptides) and scores how well the model conserves total mechanical energy E(t) = PE(t) + KE(t). The metric is the fitted total-energy drift over the run divided by the kinetic-energy standard deviation, mapped through the standard soft-threshold scorer. - Add the benchmark module, registered and gated per-system on elements. - Add the Streamlit UI page (per-system drift curves + linear fits) and wire it into the GUI. - Mark it beta (BENCHMARKS_TO_SKIP_FOR_PUBLIC_LEADERBOARD), like the NEB benchmark. - Add unit tests and checked-in test structures, plus the generic UI-page test. - Add user-facing and API-reference docs under the General category. The NVE velocity-Verlet integrator and SimulationState.potential_energy are only on mlip's `develop` branch, not yet in a PyPI release, so pin mlip to git `develop` via [tool.uv.sources] until a release ships them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: address review comments on NVE benchmark - Drop the temporary mlip git pin now that MDIntegrator.NVE_VELOCITY_VERLET and SimulationState.potential_energy have shipped in mlip 0.2.2; bump the dependency floor to >=0.2.2 (both core and cuda extra) so resolution can't fall back to a release missing these features. - Display the NVE energy-conservation benchmark on the public leaderboard. - Document the ENERGY_DRIFT_RATIO_THRESHOLD = 1.0 rationale and why it is more lenient than the ~0.01 used for classical force fields. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: prevent NVE Time (ps) axis title from being clipped The total-energy-drift chart used a fixed height with container-width rendering, which clipped the x-axis title. Size the chart to fit within its height (including padding) so the "Time (ps)" title is fully shown. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add CHANGELOG entry for the NVE energy-conservation benchmark Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> # Conflicts: # CHANGELOG.md * fix: axis cutoff by legend --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: lwalew <l.walewski@instadeep.com>
|
* feat: NPT sims for water and solvent benchmarks * feat: NPT sims for water and solvent benchmarks * refactor: changes * refactor: changes * fix: make DEFAULT_CHARGE an int Total charge is integral, and some external calculators (e.g. UMA's FAIRChemCalculator) reject a float charge. Storing 0 instead of 0.0 fixes those without affecting mlip models (which coerce internally). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: split molecular liquids RDF and density into separate benchmarks Split the water and solvent Molecular Liquids benchmarks into four scored benchmarks: water/solvent x RDF/density. Each is its own Benchmark subclass, score, leaderboard column and UI page, grouped under the "Molecular Liquids" category. - The RDF and density benchmark of each system group share a single NPT simulation via `reusable_output_id` and identical `ModelOutput` field sets, so the simulation is only run once when both are run together. - Density is now scored on its own (relative deviation from the reference density via `compute_metric_score`), rather than only shown in the RDF UI. - Shared simulation config, data loaders and density helpers moved to `utils/molecular_liquids.py`. - New `Benchmark.data_name` lets the density benchmarks reuse the RDF input data without duplicating HuggingFace uploads. - New density UI pages plot the density time series against the reference. - Docs: added a density benchmark page and moved the RDF api-reference docs under `api_reference/molecular_liquids/`. Note: the density score threshold (DENSITY_RELATIVE_DEVIATION_THRESHOLD) is a placeholder pending sign-off from the science team. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: type Benchmark.run_mode as RunMode, drop _as_run_mode helper `Benchmark.__init__` already coerces `run_mode` to a `RunMode`, so annotate the stored attribute as `RunMode` and let the molecular-liquids helpers take a plain `RunMode`. Removes the unnecessary `_as_run_mode` normalization. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: use Leon's density scoring params (2% threshold, alpha=0.1) Match the density score to Leon's NPT density analysis: relative-deviation threshold of 0.02 and a gentle decay (DENSITY_SCORE_ALPHA=0.1), rather than the placeholder 0.05 with the steep global scoring.ALPHA=3.0. With these values the density scores reproduce the expected ranking (v1 models best on water, v2 models poor). The ideal per-solvent threshold should ultimately be based on the isothermal compressibility; still pending final sign-off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: drop verbose comment on density scoring constants Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: look up solvent structures by name in the UI, drop order comment Address review feedback that solvent dict order isn't meaningful. The solvent UI pages no longer hardcode ["CCl4", "methanol", "acetonitrile"] or index `structures` positionally; they derive the system list from the results and look structures up by name (new `ordered_structure_names` helper). Also drop the misleading "canonical order" comment/docstring, since scoring is entirely by-name and nothing depends on the config dict ordering. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: address comments Docs and make some functions private * feat: remove file divider * feat: update `data_dir` docstring * feat: more private functions * docs: add references * docs: add CHANGELOG entry for molecular liquids split; simplify solvent score mean - Add a Release 0.1.5 CHANGELOG entry describing the RDF/density split, the new water_density / solvent_density benchmarks, the shared-simulation reuse, and the new Benchmark.data_name mechanism. - Drop the unreachable `r.score if r.score is not None else 0.0` guard in the two solvent benchmarks' score aggregation; the per-structure `score` field defaults to 0.0 and is never None. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> # Conflicts: # CHANGELOG.md * docs: shorten release * docs: add references for water density * feat: update reference solvent densities * docs: add api reference for molecular liquids helpers and data_dir - Add `api_reference/utils/molecular_liquids.rst` documenting `compute_densities`, `average_equilibrated_density`, `run_water_npt_simulation`, `run_solvent_npt_simulations`, and `get_solvent_system_names`, and wire it into the api-reference toctree (the module had no page unlike the sibling utils). - Document the new `Benchmark.data_dir` property via `autoproperty`. - Reword `compute_densities` `Returns:` to clear an ambiguous `densities` cross-reference warning in the docs build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: trigger claude-review * feat: bump version to 0.1.5 * refactor: share density scoring, drop vestigial SOLVENT_PEAK_RANGE, clarify RDF docs - Add a `score_density` helper in `utils/molecular_liquids.py` and use it from both the water and solvent density benchmarks, removing the duplicated relative-deviation -> compute_metric_score block. - Drop the now-unused `SOLVENT_PEAK_RANGE` constant from the water RDF benchmark; the result's `range_of_interest` field now falls back to its `RADII_RANGE` default, which matches the range the MAE/RMSE metrics are actually computed over. - Clarify in the radial-distribution docs that the metric differs by system: water uses MAE/RMSE against the experimental RDF plus a first-peak deviation, while the other solvents use the first-peak-position deviation only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: type NPT sim helpers' force_field as ForceField | ASECalculator Match `run_water_npt_simulation` / `run_solvent_npt_simulations` to the rest of the codebase (e.g. `Benchmark.__init__`) instead of `Any`, improving type-checker coverage on the shared simulation path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Marco Carobene <marco.carobene+biontech@biontech.com> Co-authored-by: lwalew <l.walewski@instadeep.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Leon Wehrhan <l.wehrhan@instadeep.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary