feat(piecewise): active_fill for partial active gates (alt. to #797, #796) (#798)

* feat(piecewise): partial `active` gate support via active_gate helper (#796)

`add_piecewise_formulation(active=...)` assumed the gate covered the whole
indexed dimension. A gate defined over only a subset of coordinate labels
(or with masked entries) silently forced the uncovered entries to zero —
breaking mixed committable / non-committable formulations (PyPSA#1755).

- Add `linopy.active_gate(active, coords, fill_value=1)`: pads a partial
  gate to full coverage, treating missing/masked entries as always-active
  (1) or always-off (0). Lives in its own module `linopy/_active_gate.py`
  as a temporary legacy stopgap; under v1 the bare
  `active.reindex(coords).fillna(fill_value)` idiom suffices and the helper
  is expected to be deprecated.
- `add_piecewise_formulation` now rejects an under-defined `active` (strict
  subset or masked) with an actionable error instead of mis-solving. A
  lower-dimensional gate still broadcasts and is accepted.
- Docs (api, piecewise guide, release notes) and tests in a dedicated
  `test/test_piecewise_active_gate.py` (parametrized over both partial
  shapes and incremental/sos2/disjunctive paths).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(piecewise): use LinearExpression.has_terms for gate gap detection

Apply review suggestion (#797): replace the hand-rolled vars/term-dim
reduction with the public `has_terms` property in `active_gate` and the
coverage validation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(piecewise): replace active_gate helper with active_fill parameter

Per PR #797 review: instead of a standalone `active_gate` helper, gate a
partial `active` via an `active_fill` parameter on `add_piecewise_formulation`.
The function derives its own coordinate, so the caller supplies nothing extra;
`active_fill=None` (default) keeps the function strict (partial `active`
raises), and `0`/`1` opt into always-off / always-on.

- Drop `linopy/_active_gate.py` and the `active_gate` export.
- Add `active_fill: int | None` to `add_piecewise_formulation`; fold the
  guard + padding into `_resolve_active`.
- `active_fill` is transitional (removed once v1 makes
  `active.reindex(coords).fillna(value)` sufficient) — documented as such.
- Tests renamed to test_piecewise_active_fill.py; docs updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(piecewise): state explicitly that a partial active = subset labels or masked

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(piecewise): type active_fill as int to satisfy mypy

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: FBumann

doc/piecewise-linear-constraints.rst

@@ -565,6 +565,24 @@ manual gating.
    variable: combined with the ``y ≤ 0`` constraint from deactivation,
    this forces ``y = 0`` automatically.
 
+Partial gates
+^^^^^^^^^^^^^
+
+``active`` must cover the formulation's full coordinate; a gate defined
+over only a subset (or with masked entries) is rejected unless
+``active_fill`` is set. ``active_fill`` gates the missing entries as
+always-active (``1``) or always-off (``0``) — handy when one formulation
+mixes committable and non-committable units sharing a single ``status``:
+
+.. code-block:: python
+
+    m.add_piecewise_formulation(
+        (power, [30, 60, 100]), (fuel, [40, 90, 170]), active=status, active_fill=1
+    )
+
+``active_fill`` is transitional: under v1 semantics, pad ``active``
+explicitly with ``active.reindex(coords).fillna(value)`` instead.
+
 Auto-broadcasting
 ~~~~~~~~~~~~~~~~~
 

doc/release_notes.rst

@@ -20,6 +20,7 @@ Upcoming Version
 *Other*
 
 * ``add_variables(binary=True, ...)`` now accepts ``lower``/``upper`` bounds, as long as they are 0 or 1. Previously binary bounds could only be set via the ``.lower``/``.upper`` setters after creation. (https://github.com/PyPSA/linopy/issues/776)
+* ``add_piecewise_formulation`` gained an ``active_fill`` parameter that gates a partial ``active`` (defined over a subset of the indexed dimension, or masked) as always-active (``1``) or always-off (``0``); without it, a partial ``active`` — which was previously zeroed silently — now raises. Useful when one formulation mixes gated and ungated entities (e.g. committable and non-committable units sharing a ``status``). ``active_fill`` is transitional and will be removed once v1 semantics make ``active.reindex(coords).fillna(value)`` sufficient. (https://github.com/PyPSA/linopy/issues/796)
 
 **Deprecations**
 

linopy/piecewise.py

@@ -838,6 +838,44 @@ def tangent_lines(
 # ---------------------------------------------------------------------------
 
 
+def _resolve_active(
+    active: LinearExpression, reference: DataArray, active_fill: int | None
+) -> LinearExpression:
+    """
+    Resolve a possibly-partial ``active`` gate against the formulation.
+
+    A gate defined over only a subset of the indexed dimension (or with
+    masked entries) would otherwise be gated as if ``active=0`` and forced
+    to zero. With ``active_fill is None`` such a gate is rejected; otherwise
+    the gaps are filled with ``active_fill`` (``1`` = always active, ``0`` =
+    always off). Dimensions absent from ``active`` broadcast and are left
+    untouched.
+    """
+    skip = {BREAKPOINT_DIM, SEGMENT_DIM} | set(HELPER_DIMS)
+    indexers = {
+        d: reference.indexes[d]
+        for d in active.coord_dims
+        if d in reference.indexes and d not in skip
+    }
+    aligned = active.reindex(indexers) if indexers else active
+
+    if active_fill is not None:
+        return aligned.where(aligned.has_terms, active_fill)
+
+    term_dims = [d for d in aligned.vars.dims if d not in aligned.coord_dims]
+    dangling = ((aligned.vars < 0) & aligned.coeffs.notnull()).any(term_dims)
+    covered = aligned.has_terms | (aligned.const.notnull() & ~dangling)
+    if not bool(covered.all()):
+        raise ValueError(
+            "`active` is not defined over the full coordinate of the "
+            "piecewise formulation: it is missing labels (a subset of the "
+            "coordinate) or has masked entries, which would be gated to "
+            "zero. Pass `active_fill=1` to treat those entries as always "
+            "active (or `0` as always off), or pass a fully-defined `active`."
+        )
+    return active
+
+
 def _validate_breakpoint_shapes(bp_a: DataArray, bp_b: DataArray) -> bool:
     """
     Validate that two breakpoint arrays have compatible shapes.
@@ -1035,6 +1073,7 @@ def add_piecewise_formulation(
     | tuple[LinExprLike, BreaksOrSlopes, Literal["==", "<=", ">="]],
     method: PWL_METHOD = "auto",
     active: LinExprLike | None = None,
+    active_fill: int | None = None,
     name: str | None = None,
 ) -> PiecewiseFormulation:
     r"""
@@ -1118,6 +1157,11 @@ def add_piecewise_formulation(
         ``active=0``, all auxiliary variables are forced to zero.
         Not supported with ``method="lp"``.
 
+        ``active`` must cover the formulation's full coordinate.  A
+        *partial* gate — one defined over only a subset of the coordinate's
+        labels, or carrying masked entries — is rejected unless
+        ``active_fill`` is set (see below).
+
         With all-equality tuples (the default), the output is then pinned
         to ``0``.  With a bounded tuple (``"<="`` / ``">="``), deactivation
         only pushes the signed bound to ``0`` (the output is ≤ 0 or ≥ 0
@@ -1129,6 +1173,16 @@ def add_piecewise_formulation(
         automatically.  For outputs that genuinely need both signs you
         must add the complementary bound yourself (e.g., a big-M
         coupling ``y`` with ``active``).
+    active_fill : int, optional
+        Fill value for the gap entries of a partial ``active`` — those where
+        ``active`` has no label (a subset of the coordinate) or is masked:
+        ``1`` treats them as always active (ungated), ``0`` as always off.
+        When ``None`` (the default) a partial ``active`` is rejected instead.
+        Useful when one formulation mixes gated and ungated entities (e.g.
+        committable and non-committable units sharing a ``status``).
+        Transitional convenience: under v1 semantics, pad ``active``
+        explicitly with ``active.reindex(coords).fillna(value)`` instead —
+        this parameter is slated for removal then.
     name : str, optional
         Base name for generated variables/constraints.
 
@@ -1285,7 +1339,12 @@ def add_piecewise_formulation(
             # can't collide with the synthetic coord for an unnamed expr.
             link_coords.append(f"_pwl_{i}")
 
-    active_expr = _to_linexpr(active) if active is not None else None
+    if active is None:
+        if active_fill is not None:
+            raise ValueError("`active_fill` has no effect without `active`.")
+        active_expr = None
+    else:
+        active_expr = _resolve_active(_to_linexpr(active), bp_list[0], active_fill)
 
     if signed_idx is None:
         inputs = _PwlInputs(

test/test_piecewise_active_fill.py

@@ -0,0 +1,222 @@
+"""
+Tests for the ``active_fill`` parameter of ``add_piecewise_formulation`` (#796).
+
+``active_fill`` is a transitional convenience: it pads a partial ``active``
+gate (a subset of the indexed dimension, or a masked gate) to full coverage.
+It is slated for removal once the v1 arithmetic semantics (#717) make
+``active.reindex(coords).fillna(value)`` correct on its own, so these tests
+live in a dedicated module that can be dropped with the parameter.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from typing import Any, Literal, TypeAlias
+
+import numpy as np
+import pandas as pd
+import pytest
+import xarray as xr
+
+from linopy import Model, available_solvers, segments
+from linopy.piecewise import _resolve_active
+from linopy.solver_capabilities import (
+    SolverFeature,
+    get_available_solvers_with_feature,
+)
+
+Method: TypeAlias = Literal["sos2", "incremental", "lp", "auto"]
+GateBuilder: TypeAlias = Callable[[Model], Any]
+
+_any_solvers = [
+    s for s in ["highs", "gurobi", "glpk", "cplex"] if s in available_solvers
+]
+_sos2_solvers = get_available_solvers_with_feature(
+    SolverFeature.SOS_CONSTRAINTS, available_solvers
+)
+
+
+# ``active`` is meaningful only for the committable subset {a, c}; "b" stays
+# ungated. The partial-gate shapes below all leave "b" as the gap.
+_PWL_GENS = pd.Index(["a", "b", "c"], name="gen")
+_COMMITTABLE = pd.Index(["a", "c"], name="gen")
+
+
+def _subset_gate(m: Model) -> Any:
+    """``active`` indexed over a strict subset of the formulation's dim."""
+    return m.add_variables(binary=True, coords=[_COMMITTABLE], name="u")
+
+
+def _masked_gate(m: Model) -> Any:
+    """``active`` over the full dim but masked where it does not apply."""
+    mask = pd.Series([True, False, True], index=_PWL_GENS)
+    return m.add_variables(binary=True, coords=[_PWL_GENS], name="u", mask=mask)
+
+
+def _full_gate(m: Model) -> Any:
+    return m.add_variables(binary=True, coords=[_PWL_GENS], name="u")
+
+
+def _scalar_gate(m: Model) -> Any:
+    return m.add_variables(binary=True, name="u")
+
+
+_PARTIAL_GATES = [
+    pytest.param(_subset_gate, id="strict-subset"),
+    pytest.param(_masked_gate, id="masked"),
+]
+
+# (builder, active_fill, should_raise): partial gates raise unless active_fill
+# is set; full/scalar gates are always fine.
+_COVERAGE_CASES = [
+    pytest.param(_subset_gate, None, True, id="subset-None-raises"),
+    pytest.param(_masked_gate, None, True, id="masked-None-raises"),
+    pytest.param(_subset_gate, 1, False, id="subset-fill1-ok"),
+    pytest.param(_masked_gate, 1, False, id="masked-fill1-ok"),
+    pytest.param(_subset_gate, 0, False, id="subset-fill0-ok"),
+    pytest.param(_full_gate, None, False, id="full-ok"),
+    pytest.param(_scalar_gate, None, False, id="scalar-ok"),
+]
+
+
+def _solve_partial_gate(
+    solver_name: str,
+    make_active: GateBuilder,
+    *,
+    method: Method,
+    disjunctive: bool = False,
+) -> None:
+    """Fill a partial gate, force the committable units off, demand "b" runs."""
+    m = Model()
+    x = m.add_variables(lower=0, upper=100, coords=[_PWL_GENS], name="x")
+    y = m.add_variables(lower=0, coords=[_PWL_GENS], name="y")
+    u = make_active(m)
+    if disjunctive:
+        m.add_piecewise_formulation(
+            (x, segments([[0.0, 50.0], [50.0, 100.0]])),
+            (y, segments([[0.0, 10.0], [10.0, 50.0]])),
+            active=u,
+            active_fill=1,
+        )
+    else:
+        m.add_piecewise_formulation(
+            (x, [0, 50, 100]),
+            (y, [0, 10, 50]),
+            active=u,
+            active_fill=1,
+            method=method,
+        )
+    m.add_constraints(u <= 0, name="force_off")
+    m.add_constraints(x.sel(gen="b") >= 50, name="demand")
+    m.add_objective(y.sum(), sense="min")
+    status, _ = m.solve(solver_name=solver_name)
+    assert status == "ok"
+    np.testing.assert_allclose(float(x.solution.sel(gen="a")), 0, atol=1e-4)
+    np.testing.assert_allclose(float(x.solution.sel(gen="c")), 0, atol=1e-4)
+    np.testing.assert_allclose(float(x.solution.sel(gen="b")), 50, atol=1e-4)
+    np.testing.assert_allclose(float(y.solution.sel(gen="b")), 10, atol=1e-4)
+
+
+class TestResolveActiveFill:
+    """The private ``_resolve_active`` fills gaps with ``active_fill``."""
+
+    @pytest.mark.parametrize("fill_value", [1, 0])
+    @pytest.mark.parametrize("make_active", _PARTIAL_GATES)
+    def test_fills_gap(self, make_active: GateBuilder, fill_value: int) -> None:
+        reference = xr.DataArray(np.zeros(len(_PWL_GENS)), coords=[_PWL_GENS])
+        gate = _resolve_active(1 * make_active(Model()), reference, fill_value)
+        assert gate.const.sel(gen="b").item() == fill_value
+        assert bool((gate.vars.sel(gen="b") < 0).all())  # no variable at "b"
+        assert bool((gate.vars.sel(gen="a") >= 0).any())  # variable kept at "a"
+
+
+class TestActiveFillValidation:
+    """``add_piecewise_formulation`` gates a partial ``active`` via ``active_fill``."""
+
+    @pytest.mark.parametrize("make_active, active_fill, should_raise", _COVERAGE_CASES)
+    def test_coverage(
+        self,
+        make_active: GateBuilder,
+        active_fill: int | None,
+        should_raise: bool,
+    ) -> None:
+        m = Model()
+        x = m.add_variables(lower=0, upper=100, coords=[_PWL_GENS], name="x")
+        y = m.add_variables(lower=0, coords=[_PWL_GENS], name="y")
+
+        def build() -> None:
+            m.add_piecewise_formulation(
+                (x, [0, 50, 100]),
+                (y, [0, 10, 50]),
+                active=make_active(m),
+                active_fill=active_fill,
+                method="incremental",
+            )
+
+        if should_raise:
+            with pytest.raises(ValueError, match="active_fill"):
+                build()
+        else:
+            build()
+
+    def test_active_fill_without_active_raises(self) -> None:
+        m = Model()
+        x = m.add_variables(lower=0, upper=100, coords=[_PWL_GENS], name="x")
+        y = m.add_variables(lower=0, coords=[_PWL_GENS], name="y")
+        with pytest.raises(ValueError, match="without `active`"):
+            m.add_piecewise_formulation(
+                (x, [0, 50, 100]),
+                (y, [0, 10, 50]),
+                active_fill=1,
+                method="incremental",
+            )
+
+    def test_lower_dimensional_active_broadcasts(self) -> None:
+        """A gate missing an entire dim broadcasts and must not be rejected."""
+        ts = pd.Index([0, 1], name="t")
+        m = Model()
+        x = m.add_variables(lower=0, upper=100, coords=[_PWL_GENS, ts], name="x")
+        y = m.add_variables(lower=0, coords=[_PWL_GENS, ts], name="y")
+        u = m.add_variables(binary=True, coords=[_PWL_GENS], name="u")
+        m.add_piecewise_formulation(
+            (x, [0, 50, 100]), (y, [0, 10, 50]), active=u, method="incremental"
+        )
+
+
+@pytest.mark.skipif(len(_any_solvers) == 0, reason="No solver available")
+class TestSolverActiveFill:
+    """End-to-end: ``active_fill`` leaves ungated units free (#796)."""
+
+    @pytest.fixture(params=_any_solvers)
+    def solver_name(self, request: pytest.FixtureRequest) -> str:
+        return request.param
+
+    @pytest.mark.parametrize("make_active", _PARTIAL_GATES)
+    def test_incremental(self, solver_name: str, make_active: GateBuilder) -> None:
+        _solve_partial_gate(solver_name, make_active, method="incremental")
+
+
+@pytest.mark.skipif(len(_sos2_solvers) == 0, reason="No SOS2-capable solver")
+class TestSolverActiveFillSOS2:
+    @pytest.fixture(params=_sos2_solvers)
+    def solver_name(self, request: pytest.FixtureRequest) -> str:
+        return request.param
+
+    @pytest.mark.parametrize("make_active", _PARTIAL_GATES)
+    @pytest.mark.parametrize(
+        "method, disjunctive",
+        [
+            pytest.param("sos2", False, id="sos2"),
+            pytest.param("auto", True, id="disjunctive"),
+        ],
+    )
+    def test_solves(
+        self,
+        solver_name: str,
+        make_active: GateBuilder,
+        method: Method,
+        disjunctive: bool,
+    ) -> None:
+        _solve_partial_gate(
+            solver_name, make_active, method=method, disjunctive=disjunctive
+        )