Docs: sp_composite user guide, examples, and parseability tests

docs/source/advanced.rst grows a "Composite single-point protocols
(sp_composite)" subsection covering all four YAML forms (preset /
preset+override / explicit recipe with CBS / per-species override),
interactions with sp_level, composite_method, adaptive_levels, and
conformer_sp_level, AEC routing + BAC-skipped-with-warning policy,
restart behavior, the provenance notebook + Run-All workflow, units,
and limitations. References with DOIs for HEAT, Helgaker/Halkier CBS,
Martin 3-pt, and Dunning basis-set families.

examples/Composite/ ships a README and four runnable inputs:
  * heat345q_preset            — preset by name
  * heat345q_partial_override  — preset with overrides
  * explicit_fpa               — explicit recipe incl. CBS term
  * per_species_override       — mixed inherit/null/explicit
The README flags HEAT-style post-(T) examples as illustrative and
calls out explicit_fpa as the affordable demo.

Tests: arc/level/examples_test.py YAML-parses every shipped example
and builds every sp_composite block via CompositeProtocol.from_user_input,
and asserts that all four forms appear so docs and examples stay in sync.

Author: alongd

arc/level/examples_test.py

@@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# encoding: utf-8
+
+"""
+Tests that every ``examples/Composite/*/input.yml`` example is valid YAML and
+that its ``sp_composite`` block (or per-species ``sp_composite`` entries)
+builds a valid :class:`CompositeProtocol` via
+:meth:`CompositeProtocol.from_user_input`. Keeps the docs + examples honest.
+"""
+
+import glob
+import os
+import unittest
+
+import yaml
+
+from arc.common import ARC_PATH
+from arc.level.protocol import CompositeProtocol
+
+
+EXAMPLES_DIR = os.path.join(ARC_PATH, "examples", "Composite")
+
+
+class TestCompositeExamples(unittest.TestCase):
+    """Parse every shipped example and validate its sp_composite payload."""
+
+    def _example_files(self):
+        pattern = os.path.join(EXAMPLES_DIR, "*", "input.yml")
+        return sorted(glob.glob(pattern))
+
+    def test_examples_directory_ships_at_least_four_inputs(self):
+        self.assertGreaterEqual(len(self._example_files()), 4)
+
+    def test_examples_readme_exists(self):
+        self.assertTrue(os.path.isfile(os.path.join(EXAMPLES_DIR, "README.md")))
+
+    def test_every_example_is_valid_yaml(self):
+        for path in self._example_files():
+            with self.subTest(path=path):
+                with open(path, "r") as fh:
+                    data = yaml.safe_load(fh)
+                self.assertIsInstance(data, dict)
+                self.assertIn("project", data)
+                self.assertIn("species", data)
+
+    def test_every_project_level_sp_composite_builds(self):
+        """Project-level ``sp_composite`` (if present) is parseable."""
+        for path in self._example_files():
+            with open(path, "r") as fh:
+                data = yaml.safe_load(fh)
+            sp = data.get("sp_composite")
+            if sp is None:
+                continue
+            with self.subTest(path=path):
+                protocol = CompositeProtocol.from_user_input(sp)
+                self.assertIsInstance(protocol, CompositeProtocol)
+
+    def test_every_species_sp_composite_builds_if_explicit(self):
+        """Per-species ``sp_composite`` (string/dict, not null) is parseable."""
+        for path in self._example_files():
+            with open(path, "r") as fh:
+                data = yaml.safe_load(fh)
+            for spc in data.get("species", []):
+                sp = spc.get("sp_composite", "__missing__")
+                if sp == "__missing__":
+                    continue
+                if sp is None:
+                    continue
+                with self.subTest(path=path, label=spc.get("label")):
+                    protocol = CompositeProtocol.from_user_input(sp)
+                    self.assertIsInstance(protocol, CompositeProtocol)
+
+    def test_all_four_forms_covered(self):
+        """Each of the four documented YAML forms must appear at least once."""
+        form1 = form2 = form3 = form4 = False
+        for path in self._example_files():
+            with open(path, "r") as fh:
+                data = yaml.safe_load(fh)
+            sp = data.get("sp_composite")
+            if isinstance(sp, str):
+                form1 = True
+            elif isinstance(sp, dict) and "preset" in sp:
+                form2 = True
+            elif isinstance(sp, dict) and "base" in sp:
+                form3 = True
+            for spc in data.get("species", []):
+                if "sp_composite" in spc:
+                    form4 = True
+        self.assertTrue(form1, "Form 1 (preset by name) not demonstrated.")
+        self.assertTrue(form2, "Form 2 (preset + override) not demonstrated.")
+        self.assertTrue(form3, "Form 3 (fully explicit recipe) not demonstrated.")
+        self.assertTrue(form4, "Form 4 (per-species override) not demonstrated.")
+
+    def test_explicit_recipe_example_includes_cbs_extrapolation(self):
+        path = os.path.join(EXAMPLES_DIR, "explicit_fpa", "input.yml")
+        with open(path, "r") as fh:
+            data = yaml.safe_load(fh)
+        corrections = data["sp_composite"]["corrections"]
+        term_types = {c["type"] for c in corrections}
+        self.assertIn("cbs_extrapolation", term_types)
+
+
+if __name__ == "__main__":
+    unittest.main()

docs/source/advanced.rst

@@ -248,6 +248,183 @@ ARC extracts active space parameters from Molpro CCSD output files to guide subs
 The method returns a dictionary containing the ``'e_o'`` tuple (electrons, orbitals) alongside lists of occupied (``'occ'``) and closed-shell (``'closed'``) orbitals per irreducible representation.
 
 
+Composite single-point protocols (``sp_composite``)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+``sp_composite`` expresses the final electronic energy of each stationary point
+as a sum of contributions computed at *different* levels of theory — a
+HEAT-style focal-point analysis. This is distinct from the legacy
+``composite_method`` (which means a Gaussian-style single-job composite like
+``CBS-QB3``); the two are mutually exclusive at the project level.
+
+**When is it for you?**
+When a single level of theory is insufficient for the accuracy you need on a
+transition state. A typical motivation: ``CCSD(T)-F12/cc-pVTZ-F12`` wells agree
+with ATcT, but TS barriers miss experiment by several kJ/mol. Adding small
+post-(T) corrections (``δ[CCSDT]``, ``δ[CCSDT(Q)]``), plus core-valence and
+scalar-relativistic terms, closes the gap without any empirical fitting.
+
+**Four YAML forms.**
+
+**Form 1 — preset by name.** The quickest path::
+
+    project: h2o_heat345q
+    sp_composite: HEAT-345Q
+    species:
+      - label: H2O
+        smiles: O
+
+ARC ships a few presets in ``arc/level/presets.yml``:
+
+* ``HEAT-345`` — HEAT-style recipe inspired by Tajti et al. (see references)
+* ``HEAT-345Q`` — HEAT-345 plus a ``δ[CCSDT(Q)]`` correction
+* ``FPA-min`` — minimal focal-point recipe with a CBS extrapolation term
+
+**Form 2 — preset with partial override.** Replace specific fields of named
+terms in the preset::
+
+    sp_composite:
+      preset: HEAT-345Q
+      overrides:
+        delta_T:
+          high: {method: ccsdt, basis: cc-pVTZ}
+
+The override dict keys are term labels (``base``, ``delta_T``, ``delta_Q``,
+``delta_CV``, ``delta_rel``, ...). Unknown target labels raise ``InputError``.
+
+**Form 3 — fully explicit recipe, including a CBS extrapolation term.** No
+preset, complete control::
+
+    sp_composite:
+      reference: "My recipe; DOI: 10.1234/example"
+      base:
+        method: ccsd(t)-f12
+        basis: cc-pVTZ-f12
+      corrections:
+        - label: delta_T
+          type: delta
+          high: {method: ccsdt,   basis: cc-pVDZ}
+          low:  {method: ccsd(t), basis: cc-pVDZ}
+        - label: cbs_corr
+          type: cbs_extrapolation
+          formula: helgaker_corr_2pt
+          components: total      # only "total" is currently supported
+          levels:
+            - {method: ccsd(t), basis: cc-pVTZ}
+            - {method: ccsd(t), basis: cc-pVQZ}
+
+Term types:
+
+* ``single_point`` — one absolute SP (only the ``base`` is usually one).
+* ``delta`` — ``E[high] − E[low]`` between two levels (same basis typically).
+* ``cbs_extrapolation`` — CBS extrapolation from ≥2 levels with the same
+  method but different basis cardinalities. Built-in formulas:
+  ``helgaker_hf_2pt`` (Halkier et al. 1998), ``helgaker_corr_2pt``
+  (Helgaker et al. 1997), ``martin_3pt`` (Martin 1996). Alternatively,
+  supply a user formula string referencing ``X``, ``Y``, ``Z`` (cardinals)
+  and ``E_X``, ``E_Y``, ``E_Z`` (energies); it is parsed through a
+  whitelisted AST evaluator — no ``eval()``.
+
+**Form 4 — per-species override.** Three states are distinguishable::
+
+    project: mixed
+    sp_composite: HEAT-345Q          # applies by default to every species
+    species:
+      - label: H2O                   # inherits the project-wide protocol
+        smiles: O
+      - label: H2O_uncorrected
+        smiles: O
+        sp_composite: null           # opt out — use plain sp_level
+      - label: TS1
+        xyz: ...
+        sp_composite:                # species-specific override
+          base: {method: mp2, basis: cc-pVTZ}
+          corrections: []
+
+Internally each species is in one of three states: ``"inherit"`` (key absent),
+``"opt_out"`` (explicit ``null``), ``"explicit"`` (preset name or recipe).
+These three survive ``as_dict`` / ``from_dict`` and restart-dict round-trip.
+
+**Interactions with other parameters.**
+
+* **``sp_level``** — coexists. If you omit ``sp_level`` while setting
+  ``sp_composite``, ARC derives ``sp_level`` from ``sp_composite.base.level``
+  so downstream code that reads ``sp_level`` (opt-out species, legacy paths)
+  keeps working. If you supply ``sp_level`` explicitly, it is preserved.
+* **``composite_method`` (legacy)** — mutually exclusive with ``sp_composite``.
+  Project fails to start with ``InputError`` if both are set.
+* **``adaptive_levels``** — mutually exclusive in the current release. Raises
+  ``InputError``. A future release may allow compatible combinations.
+* **``conformer_sp_level``** — unaffected. Conformer ranking stays at its own
+  level; ``sp_composite`` kicks in only at the final SP stage on the
+  optimized geometry.
+
+**AEC / BAC behavior.**
+When ``sp_composite`` is active, ARC automatically routes Arkane's AEC lookup
+through ``sp_composite.base.level``. The BAC lookup is **skipped entirely**
+with a single warning — BAC was derived for a single LoT and is not meaningful
+on top of a δ-corrected composite. If you need BAC, compute it externally
+against the base level and add it as a literal term in the recipe.
+
+Known limitation: per-species AEC is *not* implemented. When species carry
+mixed per-species protocols, the global AEC lookup uses the *project-level*
+``sp_composite.base.level``. Users who need per-species AEC should set
+``arkane_level_of_theory`` explicitly per project.
+
+**Restart behavior.**
+Composite sub-jobs are tracked in the persistent output dict
+(``output[label]['paths']['sp_composite']: {sub_label → path}``). Restart
+re-runs only the sub-jobs missing from that dict. On init the scheduler
+*validates* every recorded path (file exists, ``parse_e_elect`` returns a
+number); invalidated entries are pushed back to pending with a warning. After
+seeding, the scheduler kick-starts any pending sub-jobs for species with prior
+composite progress, so a restart with no other events still makes forward
+progress.
+
+**Provenance notebook.**
+Every time a composite finalizes, ARC regenerates a single project-level
+Jupyter notebook at ``<project>/output/sp_composite.ipynb``. It is
+**unexecuted on write**: it contains cell sources but no outputs. The user
+opens the notebook and runs "Run All" to independently verify the result —
+each section reconstructs its ``CompositeProtocol`` from a literal recipe
+dict, re-parses every sub-job QM output via ``arc.parser.parse_e_elect``, and
+re-evaluates the total. Citations (with DOI when supplied) carry through
+from ``presets.yml`` (or from the user's explicit ``reference:`` key) into the
+notebook's markdown.
+
+**Units.**
+``arc.parser.parse_e_elect`` returns kJ/mol. ``CompositeProtocol.evaluate``
+is a pass-through sum and preserves whatever units its inputs use. ARC always
+stores ``species.e_elect`` in kJ/mol. Hartree is used only at display /
+logging boundaries (division by ``arc.constants.E_h_kJmol``) and in the
+Arkane species-file renderer, which converts once when writing the numeric
+``energy = <Hartree>`` assignment.
+
+**Known limitations.**
+
+* **MRCC adapter**: the composite framework is ESS-agnostic, but ARC does not
+  yet ship a dedicated MRCC adapter. For ``CCSDT(Q)``, route through CFOUR
+  (NCC module) or Molpro.
+* **Per-species AEC/BAC**: see the AEC/BAC section above.
+* **``adaptive_levels`` interaction**: currently rejected; may relax later.
+
+**References.**
+
+* Allen, East, Császár — focal-point analysis review (general FPA methodology).
+* Tajti, Szalay, Császár, Kállay, Gauss, Valeev, Flowers, Vázquez, Stanton,
+  *J. Chem. Phys.* **121**, 11599 (2004). DOI: 10.1063/1.1804498 — HEAT protocol.
+* Helgaker, Klopper, Koch, Noga, *J. Chem. Phys.* **106**, 9639 (1997).
+  DOI: 10.1063/1.473863 — two-point correlation CBS extrapolation.
+* Halkier, Helgaker, Jørgensen, Klopper, Koch, Olsen, Wilson,
+  *Chem. Phys. Lett.* **286**, 243-252 (1998). DOI: 10.1016/S0009-2614(98)00111-0
+  — two-point HF CBS extrapolation; fitted ``α = 1.63``.
+* Martin, *Chem. Phys. Lett.* **259**, 669-678 (1996). DOI: 10.1016/0009-2614(96)00898-6
+  — three-point Schwartz-style extrapolation.
+* Dunning, *J. Chem. Phys.* **90**, 1007 (1989). DOI: 10.1063/1.456153 —
+  correlation-consistent basis-set families; cardinal-number convention used
+  by ``cardinal_from_basis``.
+
+
 Adaptive levels of theory
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 ARC allows users to adapt the level of theory to the size of the molecule.

examples/Composite/README.md

@@ -0,0 +1,51 @@
+# `sp_composite` examples
+
+These inputs demonstrate the four YAML forms accepted by ARC's `sp_composite`
+feature — composite single-point protocols for refined electronic energies
+(HEAT-style focal-point analysis and CBS extrapolation).
+
+| File | Demonstrates |
+|---|---|
+| `heat345q_preset/input.yml` | **Form 1** — preset by name (`HEAT-345Q`). Smallest possible composite input. |
+| `heat345q_partial_override/input.yml` | **Form 2** — preset with partial override: swap one basis set on a single term. |
+| `explicit_fpa/input.yml` | **Form 3** — fully explicit recipe, including a `cbs_extrapolation` term (Helgaker 2-pt correlation). |
+| `per_species_override/input.yml` | **Form 4** — per-species override: one species keeps the project default, one opts out via `null`, one uses a species-specific protocol. |
+
+## Running
+
+Activate the ARC conda environment (`environment.yml`), then from the repo root:
+
+    python ARC.py examples/Composite/heat345q_preset/input.yml
+
+After the run finishes, a provenance notebook is generated at
+`<project_directory>/output/sp_composite.ipynb`. Open it in Jupyter or VS Code
+and select **Run All** — each section re-parses the actual QM output files via
+`arc.parser.parse_e_elect` and re-evaluates the `CompositeProtocol` to verify
+the final `e_elect` matches what ARC recorded in `output.yml`.
+
+## A note on cost
+
+The HEAT-style examples include `CCSDT` and `CCSDT(Q)` post-(T) corrections
+that require the CFOUR (NCC module) or Molpro adapters to actually execute.
+These are *illustrative*: the recipes are scientifically meaningful for small
+molecules (4–6 atoms, tight TSs) but become prohibitive quickly. The minimal
+`heat345q_preset` example uses `H2` and `O` as smoke-test species; adapt the
+level of theory (or drop expensive terms via overrides) for larger systems.
+
+For small methodological demos that do not require an expensive post-(T)
+reference calculation, see `explicit_fpa/input.yml`, which shows the CBS
+extrapolation form using only CCSD(T)/cc-pV{T,Q}Z.
+
+## Units
+
+`species.e_elect` is stored in kJ/mol throughout. The notebook and ARC log
+display Hartree only at boundaries via division by `E_h_kJmol`
+(≈ 2625.4996 kJ/mol/Hartree). The Arkane species file (under
+`<project>/output/Species/<label>/arkane/species.py`) is rendered with a bare
+`energy = <Hartree>` assignment when `sp_composite` is active — matching
+Arkane's numeric-energy convention.
+
+## More
+
+Full documentation: `docs/source/advanced.rst`, section
+*Composite single-point protocols (sp_composite)*.

examples/Composite/explicit_fpa/input.yml

@@ -0,0 +1,41 @@
+# Form 3 — fully explicit recipe, with a CBS extrapolation term.
+#
+# A minimal Allen-style focal-point analysis:
+#   E_final = E[CCSD(T)-F12/cc-pVTZ-F12]            (base)
+#           + E_CBS(CCSD(T) corr) − E[CCSD(T)/cc-pVTZ]   (CBS extrapolation)
+#           + δ[CCSDT]                              (post-(T) correction)
+#
+# Does NOT require MRCC. CCSDT at cc-pVDZ runs in CFOUR (NCC) or Molpro.
+# CBS term uses Helgaker 2-point correlation-energy formula:
+#   (X^3·E_X − Y^3·E_Y) / (X^3 − Y^3)
+# See Helgaker, Klopper, Koch, Noga, J. Chem. Phys. 106, 9639 (1997).
+#
+# The top-level ``reference`` key carries a user-supplied citation; it flows
+# into the generated provenance notebook's References block.
+project: composite_explicit_fpa
+
+sp_composite:
+  reference: >-
+    Minimal focal-point analysis; cites Helgaker et al.
+    J. Chem. Phys. 106, 9639 (1997); DOI: 10.1063/1.473863.
+  base:
+    method: ccsd(t)-f12
+    basis: cc-pVTZ-f12
+  corrections:
+    - label: cbs_corr
+      type: cbs_extrapolation
+      formula: helgaker_corr_2pt
+      # Only ``components: total`` is currently supported — adapter-level
+      # correlation-only parsing is a future addition. Formula name documents intent.
+      components: total
+      levels:
+        - {method: ccsd(t), basis: cc-pVTZ}
+        - {method: ccsd(t), basis: cc-pVQZ}
+    - label: delta_T
+      type: delta
+      high: {method: ccsdt,   basis: cc-pVDZ}
+      low:  {method: ccsd(t), basis: cc-pVDZ}
+
+species:
+  - label: OH
+    smiles: '[OH]'