Merge branch 'main' into split_vacancies_benchmark

Author: ThomasWarford

.jules/bolt.md

@@ -0,0 +1,3 @@
+## 2024-05-19 - Caching YAML Load for Model Parsing
+**Learning:** `yaml.safe_load` on large configuration files like `models.yml` is significantly slower than parsing JSON or doing other basic IO. It can become a bottleneck when called repeatedly throughout an application's lifecycle (e.g., getting subsets of models, instantiating apps).
+**Action:** Always memoize or `@lru_cache` functions that load static, read-only configuration files (like `models.yml`) to prevent repeated disk I/O and parsing overhead.

docs/source/developer_guide/add_benchmarks.rst

@@ -501,3 +501,44 @@ Building those components and their interactivity should become increasingly aut
 but less standard plots/interactions will need setting up.
 
 For now, please contact us to help with this process.
+
+Framework credit tags
++++++++++++++++++++++
+
+If a benchmark comes from an external benchmarking framework (for example,
+MLIP Arena), add a framework credit tag as follows:
+
+1. Add/update the framework entry in ``ml_peg/app/utils/frameworks.yml``.
+
+.. code-block:: yaml
+
+    mlip_arena:
+    label: MLIP Arena
+      color: "#0f766e"
+      text_color: "#ecfeff"
+      url: "https://huggingface.co/spaces/atomind/mlip-arena"
+      logo: "https://huggingface.co/front/assets/huggingface_logo-noborder.svg"
+
+2. Set ``framework_id`` in the benchmark app constructor.
+
+.. code-block:: python3
+
+    return SomeBenchmarkApp(
+        name="SomeBenchmark",
+        ...,
+        framework_id="mlip_arena",
+    )
+
+That is all that is required. The benchmark header badge and the additional
+framework pages for non-default frameworks are populated automatically from
+this metadata.
+
+Framework sections group matching benchmarks by category, omit the category
+summary table, and reuse the same benchmark tables and controls. Updating
+weights or thresholds there therefore updates the same benchmark views shown in
+the category pages.
+
+The ``logo`` field is optional. It can point to a remote image URL or a local
+Dash asset path such as ``/assets/frameworks/my_framework_logo.png``. Use a
+browser-supported image format such as ``.svg``, ``.png``, or ``.jpg``/``.jpeg``;
+``.pdf`` is not supported. For best results, use a square logo image.

docs/source/user_guide/benchmarks/surfaces.rst

@@ -102,6 +102,7 @@ Reference data:
 * Same as input data
 * PBE-D3(BJ), MPRelaxSet settings
 
+
 Elemental Slab Oxygen Adsorption
 ================================
 
@@ -184,3 +185,49 @@ Reference data:
 * Taken from the SI of the publication above (as the main text of the publication discusses mixed levels of theory). Values from the "Medium algorithm" are used in order to be consistent with the structures.
 
 * PBE without dispersion
+
+
+Graphene Wetting Under Strain
+=============================
+
+Summary
+-------
+
+Performance in predicting adsorption energies for a water molecule on graphene under varying strain conditions.
+
+Metrics
+-------
+
+MAE of adsorption energies
+
+For each combination of water molecule orientation, water-graphene distance, and strain
+condition, the adsorption energy is calculated by taking the difference between the
+energy of the combined water + graphene system and the sum of individual water and
+graphene energies. This is compared to the reference adsorption energy, calculated in the
+same way.
+
+MAE of binding energies & lengths
+
+The adsorption energies calculated above are fitted to Morse potentials, to obtain an
+effective binding energy and binding length (i.e. minimum of adsorption energy curve) for
+each strain condition. This is compared to the reference binding energy & length,
+calculated in the same way.
+
+Computational cost
+------------------
+
+Very low: tests are likely to take less than a minute to run on CPU.
+
+Data availability
+-----------------
+
+Input data:
+
+* Structures were taken from:
+
+  * D. W. Lim, X. R. Advincula, W. C. Witt, F. L. Thiemann, C. Schran, “Revealing Strain Effects on the Graphene-Water Contact Angle Using a Machine Learning Potential,” *awaiting publication* (arXiv:2601.20134)
+
+Reference data:
+
+* Same as input data
+* PBE (with D3 dispersion correction), FHI-aims "intermediate" settings