Skip to content

Commit 20957fd

Browse files
pancettapancetta
committed
PINN and FNO playgrounds
1 parent 70abef8 commit 20957fd

35 files changed

Lines changed: 8458 additions & 0 deletions

‎pySDC/playgrounds/PINN/EXPERIMENT_SUMMARY.txt‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
================================================================================
2+
DEEP XDE PINN HYPERPARAMETER OPTIMIZATION FOR ROBER
3+
Experiment Completion Report - April 15, 2026
4+
================================================================================
5+
OBJECTIVE:
6+
Find optimal hyperparameters for training a "regular PINN" on the stiff ROBER
7+
chemical kinetics problem, without using the Stiff-PINN QSSA modifications.
8+
RESULTS: ✓ SUCCESSFUL
9+
- Initial state: Complete divergence (loss > 1e+20)
10+
- Final state: Stable convergence (loss ≈ 1e+4, Mean L2 Error ≈ 15)
11+
- Improvement: >1000x reduction in error
12+
BEST CONFIGURATION:
13+
Seed: 3
14+
Learning Rate: 1e-6 ⭐ (CRITICAL)
15+
Batch Size: 128
16+
Collocation Points: 2500 (logarithmic time spacing)
17+
Iterations: 4000
18+
Early Stopping: Loss threshold = 1e4
19+
BEST PERFORMANCE METRICS:
20+
Final Total Loss: 1.070e+04
21+
Mean Relative L2================================================================================
22+
DEEP DEEP XDE PINN HYPERPARAMETER OPTIMIZATION FOR ROBER
23+
Experiment Completion ReporCPExperiment Completion Report - April 15, 2026
24+
====
25+
============================================igOBJECTIVE:
26+
Find optimal hyperparameters for training a "regular PINN" on the ra Find opnn chemical kinetics problem, without using the Stiff-PINN QSSA modifications.
27+
1RESULTS: ✓ SUCCESSFUL
28+
- Initial state: Complete divergence (loss > 1e+20 c - Initial state: Comec - Final state: Stable convergence (loss ≈ 1e+4, s - Improvement: >1000x reduction in error
29+
BEST CONFIGURATION:
30+
Seed: leBEST CONFIGURATION:
31+
Seed: t3 Seed: io Learning Rate: g Batch Size: 128
32+
Collocation fi Collocation Points: 25 E Iterations: 4000
33+
Early Stopping: Losro Early Stopping: Los2 BEST PERFORMANCE METRICS:
34+
Final Total Lrs Final Total Loss: 1at Mean Relative L2============= ?EEP DEEP XDE PINN HYPERPARAMETER OPTIMIZATION FOR ROBER
35+
Experiment Completion ReporCPExperimentalExperiment Completion ReporCPExperiment ComplThe fundament====
36+
============================================igOBJECTIVE:
37+
Find optte ==-s Find optimal hyperparameters for training a "regular Pve 1RESULTS: ✓ SUCCESSFUL
38+
- Initial state: Complete divergence (loss > 1e+20 c - Initial state: Comec - Final state: Stable convergence (loss ≈ 1e+4, sd - Initial state: ComplicBEST CONFIGURATION:
39+
Seed: leBEST CONFIGURATION:
40+
Seed: t3 Seed: io Learning Rate: g Batch Size: 128
41+
Collocation fi Collocation Seed: leBEST COR_ Seed: t3 Seedui Collocation fi Collocation Points: 25 E Iterations: 4000
42+
Early Stopping: ro Early Stopping: Losro Early Stopping: Los2 BEST PERFORMAre Final Total Lrs Final Total Loss: 1at Mean Relative L2============= ?EsaExperiment Completion ReporCPExperimentalExperiment Completion ReporCPExperiment ComplThe fundament====
43+
===========================g4 ============================================igOBJECTIVE:
44+
Find optte ==-s Find optimal hyperparametss Find optte ==-s Find optimal hyperparameters for trai4_ - Initial state: Complete divergence (loss > 1e+20 c - Initial state: Comec - Final state: Stabir Seed: leBEST CONFIGURATION:
45+
Seed: t3 Seed: io Learning Rate: g Batch Size: 128
46+
Collocation fi Collocation Seed: leBEST COR_ py Seed: t3 Seedam Collocation fi Collocation Seed: leBEST COR_ Seed: t3 Seedui Colloca6 Early Stopping: ro Early Stopping: Losro Early Stopping: Los2 BEST PERFORMAre Final Total Lrs Final Total Loss: 1at Mean Relati:
47+
===========================g4 ============================================igOBJECTIVE:
48+
Find optte ==-s Find optimal hyperparametss Find optte ==-s Find optimal hyperparameters for trai4_ - Initial state: Complete divergence (loss > 1e+20 c - Initial state: Comec - F 6 Find optte ==-s Find optimal hyperparametss Find optte ==-s Find optimal hyperparg Seed: t3 Seed: io Learning Rate: g Batch Size: 128
49+
Collocation fi Collocation Seed: leBEST COR_ py Seed: t3 Seedam Collocation fi Collocation Seed: leBEST COR_ Sele Collocation fi Collocation Seed: leBEST COR_ py Seed: t3 Seedam Col: ===========================g4 ============================================igOBJECTIVE:
50+
Find optte ==-s Find optimal hyperparametss Find optte ==-s Find optimal hyperparameters for trai4_ - Initial state: Complete divergence (loss > 1e+20 c - Initial state: Comec - F 6 Find optte ==-s Find optimal hyperparametss FinIN Find optte ==-s Find optimal hyperparametss Find optte ==-s Find optimal hyperpartr Collocation fi Collocation Seed: leBEST COR_ py Seed: t3 Seedam Collocation fi Collocation Seed: leBEST COR_ Sele Collocation fi Collocation Seedpython3 << 'PYEND'
51+
import json
52+
from pathlib import Path
53+
data_dir = Path("/Users/robert/Codes/pySDC/pySDC/playgrounds/PINN/data")
54+
# Load all sweep data
55+
sweeps = {
56+
"CPU (float64)": "deepxde_rober_regular_fig4_fig4_lowlr_summary.json",
57+
"GPU (float32)": "deepxde_rober_regular_fig4_fig4_lowlr_mps_summary.json",
58+
}
59+
print("\n" + "="*100)
60+
print("HYPERPARAMETER SEARCH LANDSCAPE")
61+
print("="*100 + "\n")
62+
for label, fname in sweeps.items():
63+
fpath = data_dir / fname
64+
if not fpath.exists():
65+
continue
66+
data = json.loads(fpath.read_text())
67+
print(f"\n{label.upper()}")
68+
print("-" * 100)
69+
print(f"{'Seed':<6} {'LR':<12} {'Batch':<8} {'Mean L2':<15} {'Final Loss':<15} {'Status':<15}")
70+
print("-" * 100)
71+
for run in data:
72+
print(f"{run['seed']:<6} {run['lr']:<12.0e} {run['batch_size']:<8} " +
73+
f"{run['rel_l2_mean']:<15.6e} {run['final_total_loss']:<15.6e} {run['status']:<15}")
74+
print("\n" + "="*100)
75+
print("SUMMARY STATISTICS")
76+
print("="*100 + "\n")
77+
all_runs = []
78+
for label, fimport json
79+
from emfrom pathlpadata_dir = Path("/Users # Load all sweep data
80+
sweeps = {
81+
"CPU (float64)": "deepxde_rober_re sweeps = {
82+
"CPU be "CP for "GPU (float32)": "deepxde_rober_regular_fig4_fig4_lowlr_mps_summary.jde}
83+
print("\n" + "="*100)
84+
print("HYPERPARAMETER SEARCH LANDSCAPE")
85+
print("="*10rel_print("HYPERPARAMETEanprint("="*100 + "\n")
86+
for label, fname sfor label, fname in
87+
fpath = data_dir / fname
88+
sum( if not fpath.exists():
89+
es continue
90+
dast = data = json print(f"\n{label.upper()}")
91+
pribe print("-" * 100)
92+
print.2 print(f"{'Seed'ro print("-" * 100)
93+
for run incd /Users/robert/Codes/pySDC/pySDC/playgrounds/PINN && \
94+
micromamba run -n pySDC_pinn python deepxde_rober_regular_fig4.py \
95+
--seed 3 \
96+
--lr 1e-6 \
97+
--batch-size 128 \
98+
--num-points 2500 \
99+
--iterations 20000 \
100+
--display-every 500 \
101+
--max-loss-stop -1 \
102+
--run-tag best_run 2>&1
103+
cd /Users/robert/Codes/pySDC/pySDC/playgrounds/PINN && micromamba run -n pySDC_pinn python deepxde_rober_regular_fig4.py --seed 3 --lr 1e-6 --batch-size 128 --num-points 2500 --iterations 20000 --display-every 500 --max-loss-stop -1 --run-tag best_run 2>&1

‎pySDC/playgrounds/PINN/README.rst‎

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
PINN playground with DeepXDE
2+
============================
3+
4+
This playground starts a minimal setup for experiments that combine
5+
`DeepXDE <https://github.com/lululxvi/deepxde>`_ with pySDC ideas.
6+
7+
Current toy example
8+
-------------------
9+
10+
The script ``deepxde_toy_ode.py`` trains a PINN for
11+
12+
.. math::
13+
14+
y'(t) - y(t) = 0, \quad t \in [0, 1], \quad y(0)=1,
15+
16+
whose exact solution is :math:`y(t)=\exp(t)`.
17+
18+
It is intentionally small and fast, so it can serve as a starting point
19+
for later SDC-coupled prototypes.
20+
21+
Setup with micromamba
22+
---------------------
23+
24+
From this directory:
25+
26+
.. code-block:: bash
27+
28+
micromamba env create -f environment.yml
29+
micromamba activate pySDC_pinn
30+
31+
The environment installs:
32+
33+
- pySDC in editable mode from the repository root,
34+
- DeepXDE with a PyTorch backend,
35+
- basic numerical and plotting dependencies.
36+
37+
Run the toy problem
38+
-------------------
39+
40+
.. code-block:: bash
41+
42+
python deepxde_toy_ode.py
43+
44+
Expected output:
45+
46+
- training progress from DeepXDE,
47+
- relative L2 error against the exact solution,
48+
- a plot at ``data/deepxde_toy_ode_solution.png``.
49+
50+
Second case: ROBER without QSSA
51+
-------------------------------
52+
53+
The script ``deepxde_rober_no_qssa.py`` solves the full stiff ROBER system
54+
with three species and no quasi-steady-state reduction.
55+
56+
The collocation points are sampled logarithmically in time to capture both
57+
fast initial transients and slower long-time behavior (default
58+
``t in [1e-5, 1e5]``).
59+
60+
.. code-block:: bash
61+
62+
python deepxde_rober_no_qssa.py
63+
64+
You can reduce runtime while testing by lowering the number of epochs, e.g.
65+
66+
.. code-block:: bash
67+
68+
python deepxde_rober_no_qssa.py --epochs 500
69+
70+
For a very quick smoke test, reduce both epochs and time horizon:
71+
72+
.. code-block:: bash
73+
74+
python deepxde_rober_no_qssa.py --epochs 50 --t-max 1e2 --num-collocation 256 --num-eval 200
75+
76+
Expected artifacts:
77+
78+
- ``data/deepxde_rober_no_qssa_solution.png``
79+
- ``data/deepxde_rober_no_qssa_metrics.txt``
80+
81+
Third case: Figure-4-style regular PINN for ROBER
82+
-------------------------------------------------
83+
84+
The script ``deepxde_rober_regular_fig4.py`` follows the regular PINN
85+
configuration described around Figure 4 in the stiff-PINN paper:
86+
87+
- full ROBER system (no QSSA in the equations),
88+
- hard-coded IC architecture ``y(t)=y0+(t/t_scale)SNN(log(t/t_scale))``,
89+
- 2500 residual points in ``t in [1e-5, 1e5]`` sampled uniformly in log scale,
90+
- 3 hidden layers with 128 neurons, GELU activation, Adam with ``lr=1e-3`` and batch size ``128``,
91+
- defaults that are closer to the paper/authors' code: log-time NN input, hard ICs, minibatching, and normalized time scaling inside the ansatz.
92+
93+
Run a paper-style training:
94+
95+
.. code-block:: bash
96+
97+
python deepxde_rober_regular_fig4.py
98+
99+
Run a quick smoke test:
100+
101+
.. code-block:: bash
102+
103+
python deepxde_rober_regular_fig4.py --iterations 200 --num-points 512 --num-eval 300
104+
105+
Enable guarded/minibatch training (useful for sweeps/debugging, not paper baseline):
106+
107+
.. code-block:: bash
108+
109+
python deepxde_rober_regular_fig4.py --batch-size 128 --max-loss-stop 1e4 --max-divergence-loss 1e12
110+
111+
Compare paper ingredients directly:
112+
113+
.. code-block:: bash
114+
115+
python deepxde_rober_regular_fig4.py --run-tag paper_like
116+
python deepxde_rober_regular_fig4.py --no-hard-ic --run-tag no_hard_ic
117+
python deepxde_rober_regular_fig4.py --no-use-log-input --run-tag no_log_input
118+
119+
Run a broader parameter sweep with automatic ranking:
120+
121+
.. code-block:: bash
122+
123+
python deepxde_rober_regular_fig4_sweep.py --iterations 4000 --max-loss-stop 1e4 --sweep-tag fig4_scan
124+
125+
Sweep outputs are written to ``data/`` as tagged logs, metrics, and summary files.
126+
127+
Expected artifacts:
128+
129+
- ``data/deepxde_rober_regular_fig4_solution.png``
130+
- ``data/deepxde_rober_regular_fig4_loss.png``
131+
- ``data/deepxde_rober_regular_fig4_metrics.txt``
132+
133+
Fourth case: upstream-style Stiff-PINN Robertson with QSSA
134+
----------------------------------------------------------
135+
136+
The script ``stiff_pinn_robertson_qssa.py`` ports the Robertson QSSA example
137+
from the upstream `DENG-MIT/Stiff-PINN <https://github.com/DENG-MIT/Stiff-PINN>`_
138+
repository into a portable playground runner:
139+
140+
- keeps the PyTorch hard-IC ansatz used by the upstream QSSA model,
141+
- reconstructs the eliminated intermediate species through the QSSA formula,
142+
- uses SciPy BDF for the reference solution instead of the upstream ``assimulo`` dependency,
143+
- writes plots, model weights, and metrics into ``data/``.
144+
145+
Run a quick Robertson QSSA training:
146+
147+
.. code-block:: bash
148+
149+
python stiff_pinn_robertson_qssa.py --epochs 400 --run-tag smoke
150+
151+
Run a longer training closer to the upstream setup:
152+
153+
.. code-block:: bash
154+
155+
python stiff_pinn_robertson_qssa.py --epochs 2000 --batch-size 512 --run-tag long
156+
157+
Expected artifacts:
158+
159+
- ``data/stiff_pinn_robertson_qssa_solution.png``
160+
- ``data/stiff_pinn_robertson_qssa_model.pt``
161+
- ``data/stiff_pinn_robertson_qssa_metrics.txt``
162+
163+
Fifth case: simple DeepXDE regular PINN matching paper setup
164+
------------------------------------------------------------
165+
166+
The script ``deepxde_rober_paper_simple.py`` is a compact DeepXDE-only
167+
reproduction of the regular (non-QSSA) ROBER setup described in the paper:
168+
169+
- full ROBER equations with ``k1=0.04, k2=3e7, k3=1e4``,
170+
- logarithmic time domain ``t in [1e-5, 1e5]``,
171+
- 2500 residual points sampled uniformly in logarithmic scale,
172+
- hard-IC ansatz ``y=y0+(t/t_scale)SNN(log(t/t_scale))``,
173+
- 3 hidden layers with 128 neurons (GELU),
174+
- Adam with ``lr=1e-3`` and minibatch size ``128``.
175+
176+
It now supports two approaches:
177+
178+
- ``--approach global``: original single-network baseline over the full time window,
179+
- ``--approach slab_irk``: sequential local slabs, each with a local PINN and implicit RK guide points.
180+
181+
For slab mode, implicit one-step guide methods are available via ``--irk-order``:
182+
183+
- ``--irk-order 2``: implicit midpoint (order 2),
184+
- ``--irk-order 4``: 2-stage Gauss-Legendre IRK (order 4).
185+
186+
Quick smoke run:
187+
188+
.. code-block:: bash
189+
190+
python run_deepxde_rober_paper_simple.py --mode smoke --approach global
191+
192+
Quick slab+IRK2 smoke run:
193+
194+
.. code-block:: bash
195+
196+
python run_deepxde_rober_paper_simple.py --mode smoke --approach slab_irk --irk-order 2 --num-slabs 8
197+
198+
Quick slab+IRK4 smoke run:
199+
200+
.. code-block:: bash
201+
202+
python run_deepxde_rober_paper_simple.py --mode smoke --approach slab_irk --irk-order 4 --num-slabs 8
203+
204+
Compare IRK orders directly (runs both and prints RMSE lines):
205+
206+
.. code-block:: bash
207+
208+
python run_deepxde_rober_paper_simple.py --mode smoke --approach slab_irk --num-slabs 8 --compare-irk-orders
209+
210+
Run a small IRK sweep over order/slabs/steps and rank by mean RMSE:
211+
212+
.. code-block:: bash
213+
214+
python run_deepxde_rober_paper_simple_irk_sweep.py --mode smoke --num-slabs-list 4,8 --steps-per-slab-list 20,40 --irk-weight-list 1.0 --seed-list 42
215+
216+
Paper-style run:
217+
218+
.. code-block:: bash
219+
220+
python run_deepxde_rober_paper_simple.py --mode paper --approach global
221+
222+
Paper-style slab+IRK4 run:
223+
224+
.. code-block:: bash
225+
226+
python run_deepxde_rober_paper_simple.py --mode paper --approach slab_irk --irk-order 4 --num-slabs 8
227+
228+
Expected artifacts:
229+
230+
- ``data/deepxde_rober_paper_simple_solution.png``
231+
- ``data/deepxde_rober_paper_simple_loss.png``
232+
- ``data/deepxde_rober_paper_simple_metrics.txt``
233+
234+
Per run (for tagged runs), additional variable-wise plots are written:
235+
236+
- ``data/deepxde_rober_paper_simple_<tag>_y1.png``
237+
- ``data/deepxde_rober_paper_simple_<tag>_y2.png``
238+
- ``data/deepxde_rober_paper_simple_<tag>_y3.png``
239+

‎pySDC/playgrounds/PINN/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
"""Playground experiments for Physics-Informed Neural Networks (PINNs)."""
2+

0 commit comments

Comments
 (0)