Merge branch 'master' into pre-commit-ci-update-config

Author: FabianHofmann

.github/workflows/test-notebooks.yml

@@ -0,0 +1,58 @@
+name: Test Notebooks
+
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+    branches: [ '*' ]
+  schedule:
+  - cron: "0 5 * * TUE"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  notebooks:
+    name: Test documentation notebooks
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v6
+      with:
+        fetch-depth: 0
+
+    - name: Set up Python 3.12
+      uses: actions/setup-python@v6
+      with:
+        python-version: "3.12"
+
+    - name: Install package and dependencies
+      run: |
+        python -m pip install uv
+        uv pip install --system -e ".[docs]"
+
+    - name: Execute notebooks
+      run: |
+        EXIT_CODE=0
+        for notebook in examples/*.ipynb; do
+          name=$(basename "$notebook")
+
+          # Skip notebooks that require credentials or special setup
+          case "$name" in
+            solve-on-oetc.ipynb|solve-on-remote.ipynb)
+              echo "Skipping $name (requires credentials or special setup)"
+              continue
+              ;;
+          esac
+
+          echo "::group::Running $name"
+          if jupyter nbconvert --to notebook --execute --ExecutePreprocessor.timeout=600 "$notebook"; then
+            echo "✓ $name passed"
+          else
+            echo "::error::✗ $name failed"
+            EXIT_CODE=1
+          fi
+          echo "::endgroup::"
+        done
+        exit $EXIT_CODE

doc/conf.py

@@ -91,16 +91,13 @@
 
 """
 
-nbsphinx_allow_errors = True
+nbsphinx_allow_errors = False
 nbsphinx_execute = "auto"
 nbsphinx_execute_arguments = [
     "--InlineBackend.figure_formats={'svg', 'pdf'}",
     "--InlineBackend.rc={'figure.dpi': 96}",
 ]
 
-# Exclude notebooks that require credentials or special setup
-nbsphinx_execute_never = ["**/solve-on-oetc*"]
-
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for

doc/release_notes.rst

@@ -18,6 +18,8 @@ Upcoming Version
 * Add the `sphinx-copybutton` to the documentation
 * Add SOS1 and SOS2 reformulations for solvers not supporting them.
 * Add semi-continous variables for solvers that support them
+* Add ``OetcSettings.from_env()`` classmethod to create OETC settings from environment variables (``OETC_EMAIL``, ``OETC_PASSWORD``, ``OETC_NAME``, ``OETC_AUTH_URL``, ``OETC_ORCHESTRATOR_URL``, ``OETC_CPU_CORES``, ``OETC_DISK_SPACE_GB``, ``OETC_DELETE_WORKER_ON_ERROR``).
+* Forward ``solver_name`` and ``**solver_options`` from ``Model.solve()`` to OETC handler. Call-level options override settings-level defaults.
 * Improve handling of CPLEX solver quality attributes to ensure metrics such are extracted correctly when available.
 * Fix Xpress IIS label mapping for masked constraints and add a regression test for matching infeasible coordinates.
 * Enable quadratic problems with SCIP on windows.

examples/piecewise-linear-constraints.ipynb

@@ -3,7 +3,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": "# Piecewise Linear Constraints Tutorial\n\nThis notebook demonstrates linopy's piecewise linear (PWL) constraint formulations.\nEach example builds a separate dispatch model where a single power plant must meet\na time-varying demand.\n\n| Example | Plant | Limitation | Formulation |\n|---------|-------|------------|-------------|\n| 1 | Gas turbine (0–100 MW) | Convex heat rate | SOS2 |\n| 2 | Coal plant (0–150 MW) | Monotonic heat rate | Incremental |\n| 3 | Diesel generator (off or 50–80 MW) | Forbidden zone | Disjunctive |\n| 4 | Concave efficiency curve | Inequality bound | LP |\n| 5 | Gas unit with commitment | On/off + min load | Incremental + `active` |\n\n**Note:** The `piecewise(...)` expression can appear on either side of\nthe comparison operator (`==`, `<=`, `>=`). For example, both\n`linopy.piecewise(x, x_pts, y_pts) == y` and `y == linopy.piecewise(...)` work."
+   "source": "# Piecewise Linear Constraints Tutorial\n\nThis notebook demonstrates linopy's piecewise linear (PWL) constraint formulations.\nEach example builds a separate dispatch model where a single power plant must meet\na time-varying demand.\n\n| Example | Plant | Limitation | Formulation |\n|---------|-------|------------|-------------|\n| 1 | Gas turbine (0\u2013100 MW) | Convex heat rate | SOS2 |\n| 2 | Coal plant (0\u2013150 MW) | Monotonic heat rate | Incremental |\n| 3 | Diesel generator (off or 50\u201380 MW) | Forbidden zone | Disjunctive |\n| 4 | Concave efficiency curve | Inequality bound | LP |\n| 5 | Gas unit with commitment | On/off + min load | Incremental + `active` |\n\n**Note:** The `piecewise(...)` expression can appear on either side of\nthe comparison operator (`==`, `<=`, `>=`). For example, both\n`linopy.piecewise(x, x_pts, y_pts) == y` and `y == linopy.piecewise(...)` work."
   },
   {
    "cell_type": "code",
@@ -90,7 +90,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 1. SOS2 formulation — Gas turbine\n",
+    "## 1. SOS2 formulation \u2014 Gas turbine\n",
     "\n",
     "The gas turbine has a **convex** heat rate: efficient at moderate load,\n",
     "increasingly fuel-hungry at high output. We use the **SOS2** formulation\n",
@@ -173,7 +173,7 @@
     }
    },
    "source": [
-    "m1.solve()"
+    "m1.solve(reformulate_sos=\"auto\")"
    ],
    "outputs": [],
    "execution_count": null
@@ -224,11 +224,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 2. Incremental formulation — Coal plant\n",
+    "## 2. Incremental formulation \u2014 Coal plant\n",
     "\n",
     "The coal plant has a **monotonically increasing** heat rate. Since all\n",
     "breakpoints are strictly monotonic, we can use the **incremental**\n",
-    "formulation — which uses fill-fraction variables with binary indicators."
+    "formulation \u2014 which uses fill-fraction variables with binary indicators."
    ]
   },
   {
@@ -306,7 +306,7 @@
     }
    },
    "source": [
-    "m2.solve();"
+    "m2.solve(reformulate_sos=\"auto\");"
    ],
    "outputs": [],
    "execution_count": null
@@ -357,10 +357,10 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 3. Disjunctive formulation — Diesel generator\n",
+    "## 3. Disjunctive formulation \u2014 Diesel generator\n",
     "\n",
     "The diesel generator has a **forbidden operating zone**: it must either\n",
-    "be off (0 MW) or run between 50–80 MW. Because of this gap, we use\n",
+    "be off (0 MW) or run between 50\u201380 MW. Because of this gap, we use\n",
     "**disjunctive** piecewise constraints via `linopy.segments()` and add a\n",
     "high-cost **backup** source to cover demand when the diesel is off or\n",
     "at its maximum.\n",
@@ -446,7 +446,7 @@
     }
    },
    "source": [
-    "m3.solve()"
+    "m3.solve(reformulate_sos=\"auto\")"
    ],
    "outputs": [],
    "execution_count": null
@@ -476,11 +476,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 4. LP formulation — Concave efficiency bound\n",
+    "## 4. LP formulation \u2014 Concave efficiency bound\n",
     "\n",
     "When the piecewise function is **concave** and we use a `>=` constraint\n",
     "(i.e. `pw >= y`, meaning y is bounded above by pw), linopy can use a\n",
-    "pure **LP** formulation with tangent-line constraints — no SOS2 or\n",
+    "pure **LP** formulation with tangent-line constraints \u2014 no SOS2 or\n",
     "binary variables needed. This is the fastest to solve.\n",
     "\n",
     "For this formulation, the x-breakpoints must be in **strictly increasing**\n",
@@ -514,7 +514,7 @@
     "power = m4.add_variables(name=\"power\", lower=0, upper=120, coords=[time])\n",
     "fuel = m4.add_variables(name=\"fuel\", lower=0, coords=[time])\n",
     "\n",
-    "# pw >= fuel means fuel <= concave_function(power) → auto-selects LP method\n",
+    "# pw >= fuel means fuel <= concave_function(power) \u2192 auto-selects LP method\n",
     "m4.add_piecewise_constraints(\n",
     "    linopy.piecewise(power, x_pts4, y_pts4) >= fuel,\n",
     "    name=\"pwl\",\n",
@@ -544,7 +544,7 @@
     }
    },
    "source": [
-    "m4.solve()"
+    "m4.solve(reformulate_sos=\"auto\")"
    ],
    "outputs": [],
    "execution_count": null
@@ -595,7 +595,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 5. Slopes mode — Building breakpoints from slopes\n",
+    "## 5. Slopes mode \u2014 Building breakpoints from slopes\n",
     "\n",
     "Sometimes you know the **slope** of each segment rather than the y-values\n",
     "at each breakpoint. The `breakpoints()` factory can compute y-values from\n",
@@ -628,7 +628,7 @@
   },
   {
    "cell_type": "markdown",
-   "source": "## 6. Active parameter — Unit commitment with piecewise efficiency\n\nIn unit commitment problems, a binary variable $u_t$ controls whether a\nunit is **on** or **off**. When off, both power output and fuel consumption\nmust be zero. When on, the unit operates within its piecewise-linear\nefficiency curve between $P_{min}$ and $P_{max}$.\n\nThe `active` parameter on `piecewise()` handles this by gating the\ninternal PWL formulation with the commitment binary:\n\n- **Incremental:** delta bounds tighten from $\\delta_i \\leq 1$ to\n  $\\delta_i \\leq u$, and base terms are multiplied by $u$\n- **SOS2:** convexity constraint becomes $\\sum \\lambda_i = u$\n- **Disjunctive:** segment selection becomes $\\sum z_k = u$\n\nThis is the only gating behavior expressible with pure linear constraints.\nSelectively *relaxing* the PWL (letting x, y float freely when off) would\nrequire big-M or indicator constraints.",
+   "source": "## 6. Active parameter \u2014 Unit commitment with piecewise efficiency\n\nIn unit commitment problems, a binary variable $u_t$ controls whether a\nunit is **on** or **off**. When off, both power output and fuel consumption\nmust be zero. When on, the unit operates within its piecewise-linear\nefficiency curve between $P_{min}$ and $P_{max}$.\n\nThe `active` parameter on `piecewise()` handles this by gating the\ninternal PWL formulation with the commitment binary:\n\n- **Incremental:** delta bounds tighten from $\\delta_i \\leq 1$ to\n  $\\delta_i \\leq u$, and base terms are multiplied by $u$\n- **SOS2:** convexity constraint becomes $\\sum \\lambda_i = u$\n- **Disjunctive:** segment selection becomes $\\sum z_k = u$\n\nThis is the only gating behavior expressible with pure linear constraints.\nSelectively *relaxing* the PWL (letting x, y float freely when off) would\nrequire big-M or indicator constraints.",
    "metadata": {}
   },
   {
@@ -666,7 +666,7 @@
   },
   {
    "cell_type": "code",
-   "source": "m6.solve()",
+   "source": "m6.solve(reformulate_sos=\"auto\")",
    "metadata": {
     "ExecuteTime": {
      "end_time": "2026-03-09T10:17:28.878112Z",
@@ -855,7 +855,7 @@
   },
   {
    "cell_type": "markdown",
-   "source": "At **t=1**, demand (15 MW) is below the minimum load (30 MW). The solver\nkeeps the unit off (`commit=0`), so `power=0` and `fuel=0` — the `active`\nparameter enforces this. Demand is met by the backup source.\n\nAt **t=2** and **t=3**, the unit commits and operates on the PWL curve.",
+   "source": "At **t=1**, demand (15 MW) is below the minimum load (30 MW). The solver\nkeeps the unit off (`commit=0`), so `power=0` and `fuel=0` \u2014 the `active`\nparameter enforces this. Demand is met by the backup source.\n\nAt **t=2** and **t=3**, the unit commits and operates on the PWL curve.",
    "metadata": {}
   }
  ],

examples/solve-on-oetc.ipynb

@@ -30,6 +30,13 @@
     "All of these steps are handled automatically by linopy's `OetcHandler`."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "> **Note:** This notebook requires Google Cloud credentials and access to the OETC platform. It is not executed during the documentation build, so no cell outputs are shown. To run it yourself, install the `linopy[oetc]` extra and configure your credentials."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -79,7 +86,12 @@
    "source": [
     "## Configure OETC Settings\n",
     "\n",
-    "Next, we need to configure the OETC settings including credentials and compute requirements:"
+    "There are two ways to configure OETC settings:\n",
+    "\n",
+    "1. **Manual construction** \u2014 build `OetcCredentials` and `OetcSettings` explicitly\n",
+    "2. **`OetcSettings.from_env()`** \u2014 resolve credentials and options from environment variables\n",
+    "\n",
+    "### Option 1: Manual Construction"
    ]
   },
   {
@@ -123,6 +135,48 @@
     "print(f\"Disk space: {settings.disk_space_gb} GB\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Option 2: Create Settings from Environment Variables\n",
+    "\n",
+    "`OetcSettings.from_env()` reads configuration from environment variables,\n",
+    "with optional keyword overrides. This is the recommended approach for\n",
+    "CI/CD pipelines and production deployments.\n",
+    "\n",
+    "| Environment Variable | Required | Description |\n",
+    "|---|---|---|\n",
+    "| `OETC_EMAIL` | Yes | Account email |\n",
+    "| `OETC_PASSWORD` | Yes | Account password |\n",
+    "| `OETC_NAME` | Yes | Job name |\n",
+    "| `OETC_AUTH_URL` | Yes | Authentication server URL |\n",
+    "| `OETC_ORCHESTRATOR_URL` | Yes | Orchestrator server URL |\n",
+    "| `OETC_CPU_CORES` | No | CPU cores (default: 2) |\n",
+    "| `OETC_DISK_SPACE_GB` | No | Disk space in GB (default: 10) |\n",
+    "| `OETC_DELETE_WORKER_ON_ERROR` | No | Delete worker on error (default: false) |\n",
+    "\n",
+    "Keyword arguments take precedence over environment variables."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create settings from environment variables\n",
+    "# All required env vars must be set: OETC_EMAIL, OETC_PASSWORD,\n",
+    "# OETC_NAME, OETC_AUTH_URL, OETC_ORCHESTRATOR_URL\n",
+    "settings = OetcSettings.from_env()\n",
+    "\n",
+    "# Or override specific values via keyword arguments\n",
+    "settings = OetcSettings.from_env(\n",
+    "    cpu_cores=8,\n",
+    "    disk_space_gb=50,\n",
+    ")"
+   ],
+   "execution_count": null
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -221,38 +275,49 @@
     "\n",
     "### Solver Options\n",
     "\n",
-    "You can pass solver-specific options through the `solver_options` parameter:"
+    "Solver name and options can be configured at two levels:\n",
+    "\n",
+    "1. **Settings level** \u2014 defaults stored in `OetcSettings.solver` and `OetcSettings.solver_options`\n",
+    "2. **Call level** \u2014 passed via `m.solve(solver_name=..., **solver_options)`\n",
+    "\n",
+    "Call-level options **override** settings-level options. The two dicts are\n",
+    "merged (call-time takes precedence), and the original settings are never\n",
+    "mutated."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Example with advanced solver options\n",
+    "# Settings-level defaults\n",
     "advanced_settings = OetcSettings(\n",
     "    credentials=credentials,\n",
     "    name=\"advanced-linopy-job\",\n",
     "    authentication_server_url=\"https://auth.oetcloud.com\",\n",
     "    orchestrator_server_url=\"https://orchestrator.oetcloud.com\",\n",
-    "    solver=\"gurobi\",  # Using Gurobi solver\n",
+    "    solver=\"gurobi\",\n",
     "    solver_options={\n",
-    "        \"TimeLimit\": 600,  # 10 minutes\n",
-    "        \"MIPGap\": 0.01,  # 1% optimality gap\n",
-    "        \"Threads\": 4,  # Use 4 threads\n",
-    "        \"OutputFlag\": 1,  # Enable solver output\n",
+    "        \"TimeLimit\": 600,\n",
+    "        \"MIPGap\": 0.01,\n",
     "    },\n",
-    "    cpu_cores=8,  # More CPU cores for larger problems\n",
-    "    disk_space_gb=50,  # More disk space\n",
+    "    cpu_cores=8,\n",
+    "    disk_space_gb=50,\n",
     ")\n",
     "\n",
-    "print(\"Advanced OETC settings:\")\n",
-    "print(f\"Solver: {advanced_settings.solver}\")\n",
-    "print(f\"Solver options: {advanced_settings.solver_options}\")\n",
-    "print(f\"CPU cores: {advanced_settings.cpu_cores}\")\n",
-    "print(f\"Disk space: {advanced_settings.disk_space_gb} GB\")"
-   ]
+    "advanced_handler = OetcHandler(advanced_settings)\n",
+    "\n",
+    "# Call-level overrides: solver_name and solver_options are forwarded\n",
+    "# to OETC and merged with the settings defaults.\n",
+    "# Here MIPGap from settings (0.01) is kept, TimeLimit is overridden to 300.\n",
+    "status, condition = m.solve(\n",
+    "    remote=advanced_handler,\n",
+    "    solver_name=\"gurobi\",\n",
+    "    TimeLimit=300,\n",
+    "    Threads=4,\n",
+    ")"
+   ],
+   "execution_count": null
   },
   {
    "cell_type": "markdown",
@@ -356,6 +421,9 @@
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
    "version": "3.12.3"
+  },
+  "nbsphinx": {
+   "execute": "never"
   }
  },
  "nbformat": 4,