Skip to content

Commit 69bb3ce

Browse files
committed
BUG: fix Monte Carlo seeding race and non-reproducible SeedSequence
Addresses review feedback on RocketPy-Team#1054. Parallel workers claimed the next index with an unlocked keep_simulating() + increment(), so near the end of a run two workers could both pass the count < n check and then claim sim_idx == n; the per-index child_seeds lookup turned that into an IndexError (before, it only wrote one extra record). Move the claim into a _claim_next_index helper that holds the shared mutex across the check and the increment, so each index is handed out once and the counter never overshoots. A deterministic unit test (a barrier plus a widened check-to-increment window) over-claims and fails if the lock is dropped. __root_seed_sequence returned the caller's SeedSequence, and spawn() advances its child counter, so passing the same object to simulate() twice produced different children. Copy it from its full state instead, which leaves the caller untouched and keeps repeated calls reproducible. Also drop Generator/BitGenerator from the accepted types: a stateful generator is not a seed, and reducing it to its underlying SeedSequence ignores how far it has been consumed. random_seed now takes an int, a sequence of ints, or a SeedSequence. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
1 parent 52cbf0b commit 69bb3ce

3 files changed

Lines changed: 150 additions & 51 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@ Attention: The newest changes should be on top -->
3333
### Added
3434

3535
- ENH: BUG/MNT: pre-release v1.13.0 review fixes [#1074](https://github.com/RocketPy-Team/RocketPy/pull/1074)
36+
- ENH: reproducible Monte Carlo runs via a random_seed argument [#1054](https://github.com/RocketPy-Team/RocketPy/pull/1054)
37+
3638
### Changed
3739

3840
### Fixed

rocketpy/simulation/monte_carlo.py

Lines changed: 49 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -191,15 +191,15 @@ def simulate(
191191
number of workers will be equal to the number of CPUs available.
192192
A minimum of 2 workers is required for parallel mode.
193193
Default is None.
194-
random_seed : int, numpy.random.SeedSequence, numpy.random.Generator, optional
194+
random_seed : int or numpy.random.SeedSequence, optional
195195
Root seed for the run. When provided, the sampled inputs are
196196
reproducible and identical across serial and parallel execution
197197
and across any number of workers, because each simulation index
198-
draws from its own child stream spawned from this root
199-
(``SeedSequence(random_seed).spawn(number_of_simulations)``). It
200-
accepts an int, a ``SeedSequence`` or a ``Generator``. Default is
201-
None, which draws fresh entropy (not reproducible), preserving the
202-
previous behavior.
198+
draws from its own child stream spawned from this root. It accepts
199+
an int or a ``SeedSequence``; a supplied ``SeedSequence`` is copied
200+
rather than consumed, so repeated calls with the same seed produce
201+
the same inputs. Default is None, which draws fresh entropy (not
202+
reproducible), preserving the previous behavior.
203203
kwargs : dict
204204
Custom arguments for simulation export of the ``inputs`` file. Options
205205
are:
@@ -280,19 +280,26 @@ def __setup_files(self, append):
280280

281281
@staticmethod
282282
def __root_seed_sequence(random_seed):
283-
"""Return a ``SeedSequence`` root from a flexible seed argument.
284-
285-
Accepts what the scientific-Python SPEC 7 seeding convention accepts
286-
(an int, a ``SeedSequence``, a ``Generator`` or ``BitGenerator``, or
287-
None for fresh entropy) and returns a ``SeedSequence`` so it can be
288-
spawned into one independent child stream per simulation index.
283+
"""Build a fresh ``SeedSequence`` root from ``random_seed``.
284+
285+
``random_seed`` may be an int (or any entropy ``numpy.random.SeedSequence``
286+
accepts), an existing ``SeedSequence``, or ``None`` for fresh entropy. A
287+
supplied ``SeedSequence`` is copied from its full ``state``, so the
288+
spawning below neither mutates the caller's object nor advances a shared
289+
child counter between calls; repeated ``simulate`` calls with the same
290+
seed then stay reproducible. A stateful ``Generator``/``BitGenerator`` is
291+
not accepted, since using it as an immutable seed would contradict its
292+
consume-on-use semantics; pass ``rng.bit_generator.seed_seq`` to seed
293+
from an existing generator's stream.
289294
"""
290295
if isinstance(random_seed, np.random.SeedSequence):
291-
return random_seed
292-
if isinstance(random_seed, np.random.Generator):
293-
return random_seed.bit_generator.seed_seq
294-
if isinstance(random_seed, np.random.BitGenerator):
295-
return random_seed.seed_seq
296+
return np.random.SeedSequence(**random_seed.state)
297+
if isinstance(random_seed, (np.random.Generator, np.random.BitGenerator)):
298+
raise TypeError(
299+
"random_seed must be an int or a numpy.random.SeedSequence, not "
300+
f"a {type(random_seed).__name__}; to seed from an existing "
301+
"generator pass rng.bit_generator.seed_seq."
302+
)
296303
return np.random.SeedSequence(random_seed)
297304

298305
def __seed_simulation(self, child_seed):
@@ -315,7 +322,7 @@ def __run_in_serial(self, random_seed=None): # pylint: disable=too-many-stateme
315322
316323
Parameters
317324
----------
318-
random_seed : int, SeedSequence, Generator, optional
325+
random_seed : int or SeedSequence, optional
319326
Root seed for the run. See ``simulate``.
320327
321328
Returns
@@ -369,7 +376,7 @@ def __run_in_parallel(self, n_workers=None, random_seed=None):
369376
n_workers: int, optional
370377
Number of workers to be used. If None, the number of workers
371378
will be equal to the number of CPUs available. Default is None.
372-
random_seed : int, SeedSequence, Generator, optional
379+
random_seed : int or SeedSequence, optional
373380
Root seed for the run. See ``simulate``.
374381
375382
Returns
@@ -464,8 +471,11 @@ def __sim_producer(self, child_seeds, sim_monitor, mutex, error_event): # pylin
464471
Event signaling an error occurred during the simulation.
465472
"""
466473
try:
467-
while sim_monitor.keep_simulating():
468-
sim_idx = sim_monitor.increment() - 1
474+
while True:
475+
sim_idx = _claim_next_index(sim_monitor, mutex)
476+
if sim_idx is None:
477+
break
478+
469479
inputs_json, outputs_json = "", ""
470480

471481
self.__seed_simulation(child_seeds[sim_idx])
@@ -1665,6 +1675,24 @@ def export_errors_to_json(self, filename):
16651675
self._write_log_to_json(self.errors_log, filename)
16661676

16671677

1678+
def _claim_next_index(sim_monitor, mutex):
1679+
"""Atomically claim the next 0-based simulation index, or ``None`` if done.
1680+
1681+
``keep_simulating()`` and ``increment()`` are two separate manager calls, so
1682+
the shared ``mutex`` has to be held across both. Without it, two workers can
1683+
each pass the ``count < number_of_simulations`` check at the tail before
1684+
either increments, and both then claim an index, overrunning the requested
1685+
number of simulations and indexing past the per-index seed list.
1686+
"""
1687+
mutex.acquire()
1688+
try:
1689+
if not sim_monitor.keep_simulating():
1690+
return None
1691+
return sim_monitor.increment() - 1
1692+
finally:
1693+
mutex.release()
1694+
1695+
16681696
def _import_multiprocess():
16691697
"""Import the necessary modules and submodules for the
16701698
multiprocess library.

tests/unit/simulation/test_monte_carlo_determinism.py

Lines changed: 99 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,9 @@
44
turning the run's root seed into one independent child stream per simulation
55
index. Two private helpers do the work:
66
7-
* ``__root_seed_sequence`` normalizes the flexible ``random_seed`` argument (int,
8-
``SeedSequence``, ``Generator``, ``BitGenerator`` or None) into a
9-
``SeedSequence`` that can be spawned;
7+
* ``__root_seed_sequence`` normalizes the ``random_seed`` argument (an int, a
8+
sequence of ints, a ``SeedSequence`` or None) into a fresh ``SeedSequence``
9+
that can be spawned;
1010
* ``__seed_simulation`` splits one per-index child seed three ways so the
1111
environment, rocket and flight draw from independent streams.
1212
@@ -19,12 +19,15 @@
1919
it lets the seeding invariants be asserted without running a Monte Carlo.
2020
"""
2121

22+
import threading
23+
import time
2224
from types import SimpleNamespace
2325

2426
import numpy as np
2527
import pytest
2628

2729
from rocketpy.simulation import MonteCarlo
30+
from rocketpy.simulation.monte_carlo import _SimMonitor, _claim_next_index
2831

2932
_root_seed_sequence = MonteCarlo._MonteCarlo__root_seed_sequence
3033
_seed_simulation = MonteCarlo._MonteCarlo__seed_simulation
@@ -44,17 +47,17 @@ def _entropy(seed_sequence, n=4):
4447
"make_seed",
4548
[
4649
pytest.param(lambda: 12345, id="int"),
50+
pytest.param(lambda: np.int64(12345), id="numpy-int"),
51+
pytest.param(lambda: [1, 2, 3], id="sequence"),
4752
pytest.param(lambda: np.random.SeedSequence(12345), id="seedsequence"),
48-
pytest.param(lambda: np.random.default_rng(12345), id="generator"),
49-
pytest.param(lambda: np.random.PCG64(12345), id="bitgenerator"),
5053
],
5154
)
52-
def test_root_seed_sequence_accepts_supported_types(make_seed):
53-
"""int, SeedSequence, Generator and BitGenerator all normalize to the same
54-
root SeedSequence stream for an equivalent seed value."""
55+
def test_root_seed_sequence_accepts_seed_like_values(make_seed):
56+
"""An int, a numpy integer, a sequence of ints and a SeedSequence are all
57+
accepted, normalize to a SeedSequence, and are reproducible."""
5558
root = _root_seed_sequence(make_seed())
5659
assert isinstance(root, np.random.SeedSequence)
57-
assert _entropy(root) == _entropy(_root_seed_sequence(12345))
60+
assert _entropy(root) == _entropy(_root_seed_sequence(make_seed()))
5861

5962

6063
def test_root_seed_sequence_none_draws_fresh_entropy():
@@ -65,30 +68,42 @@ def test_root_seed_sequence_none_draws_fresh_entropy():
6568

6669

6770
@pytest.mark.parametrize(
68-
"make_seed, resolve",
71+
"make_generator",
6972
[
70-
pytest.param(
71-
lambda: np.random.SeedSequence(999),
72-
lambda seed: seed,
73-
id="seedsequence",
74-
),
75-
pytest.param(
76-
lambda: np.random.default_rng(999),
77-
lambda seed: seed.bit_generator.seed_seq,
78-
id="generator",
79-
),
80-
pytest.param(
81-
lambda: np.random.PCG64(999),
82-
lambda seed: seed.seed_seq,
83-
id="bitgenerator",
84-
),
73+
pytest.param(lambda: np.random.default_rng(999), id="generator"),
74+
pytest.param(lambda: np.random.PCG64(999), id="bitgenerator"),
8575
],
8676
)
87-
def test_root_seed_sequence_reuses_existing_seed_sequence(make_seed, resolve):
88-
"""When given something that already carries a SeedSequence, the helper
89-
reuses that object rather than copying it."""
90-
seed = make_seed()
91-
assert _root_seed_sequence(seed) is resolve(seed)
77+
def test_root_seed_sequence_rejects_stateful_generators(make_generator):
78+
"""A Generator/BitGenerator is a stateful RNG, not a seed value, so it is
79+
rejected instead of being reduced to its underlying SeedSequence."""
80+
with pytest.raises(TypeError, match="SeedSequence"):
81+
_root_seed_sequence(make_generator())
82+
83+
84+
def test_root_seed_sequence_copies_seedsequence_without_mutating_it():
85+
"""A supplied SeedSequence is copied from its full state: repeated calls with
86+
the same object reproduce the same children, the caller's spawn counter is
87+
left untouched, and a spawned child (non-empty spawn_key) round-trips too."""
88+
89+
def children(seed_sequence):
90+
return [
91+
_entropy(child) for child in _root_seed_sequence(seed_sequence).spawn(3)
92+
]
93+
94+
# A SeedSequence that has already spawned children, so its counter is not 0.
95+
seed = np.random.SeedSequence(2024)
96+
seed.spawn(5)
97+
counter_before = seed.n_children_spawned
98+
99+
assert children(seed) == children(seed), "same object twice must reproduce"
100+
assert seed.n_children_spawned == counter_before, "caller must not be mutated"
101+
assert _root_seed_sequence(seed) is not seed, "must return a copy, not the caller"
102+
103+
# A spawned child carries a non-empty spawn_key that the copy must preserve.
104+
child = np.random.SeedSequence(2024).spawn(1)[0]
105+
assert child.spawn_key != ()
106+
assert children(child) == children(child)
92107

93108

94109
# --------------------------------------------------------------------------- #
@@ -138,3 +153,57 @@ def split(child):
138153
return [_entropy(env[0]), _entropy(rocket[0]), _entropy(flight[0])]
139154

140155
assert split(np.random.SeedSequence(2024)) == split(np.random.SeedSequence(2024))
156+
157+
158+
# --------------------------------------------------------------------------- #
159+
# _claim_next_index: atomic hand-out of the next simulation index #
160+
# --------------------------------------------------------------------------- #
161+
162+
163+
def test_claim_next_index_hands_out_each_index_once_under_contention():
164+
"""Holding the mutex across keep_simulating() and increment() must hand out
165+
each index exactly once, even when every worker reaches the claim together.
166+
167+
A barrier releases all workers at once and a widened check-to-increment
168+
window would let an unlocked claim run several workers past the count < n
169+
check before any increments; the lock is what keeps the result to exactly
170+
n_simulations indices (0..n-1, none repeated) and the counter from
171+
overshooting.
172+
"""
173+
n_simulations = 5
174+
n_workers = 8
175+
monitor = _SimMonitor(initial_count=0, n_simulations=n_simulations, start_time=0.0)
176+
177+
# Widen the window between the check and the increment so that, without the
178+
# lock, several workers could pass count < n before any of them increments.
179+
real_keep_simulating = monitor.keep_simulating
180+
181+
def slow_keep_simulating():
182+
result = real_keep_simulating()
183+
time.sleep(0.02)
184+
return result
185+
186+
monitor.keep_simulating = slow_keep_simulating
187+
188+
mutex = threading.Lock()
189+
barrier = threading.Barrier(n_workers)
190+
claimed = []
191+
claimed_lock = threading.Lock()
192+
193+
def worker():
194+
barrier.wait()
195+
while True:
196+
index = _claim_next_index(monitor, mutex)
197+
if index is None:
198+
break
199+
with claimed_lock:
200+
claimed.append(index)
201+
202+
workers = [threading.Thread(target=worker) for _ in range(n_workers)]
203+
for thread in workers:
204+
thread.start()
205+
for thread in workers:
206+
thread.join()
207+
208+
assert sorted(claimed) == list(range(n_simulations))
209+
assert monitor.count == n_simulations

0 commit comments

Comments
 (0)