Skip to content

Commit 817c852

Browse files
committed
fixed examples
1 parent ba6bdb3 commit 817c852

2 files changed

Lines changed: 17 additions & 29 deletions

File tree

docs/guides/seeding.md

Lines changed: 13 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -40,35 +40,24 @@ This creates a `DefaultSeedGenerator` internally and passes it to all setup meth
4040

4141
### Disabling Seeding
4242

43-
When you don't pass a `seed` parameter (or pass `seed=None`), seeding is disabled:
43+
If you don't need seeding, you can simply ignore the seed generators. However, in workflows where you mix seeded and non-seeded runs, you can disable seeding without writing `if/else` statements to check whether a seed is provided.
4444

45-
```python
46-
# No seed = seeding disabled
47-
benchmark = MyBenchmark()
48-
49-
# Explicit None = also disabled
50-
benchmark = MyBenchmark(seed=None)
51-
```
52-
53-
When seeding is disabled:
45+
To disable seeding, omit the `seed` parameter when creating your `Benchmark` or `DefaultSeedGenerator` (or pass `seed=None`):
5446

5547
1. A `DefaultSeedGenerator(global_seed=None)` is still created internally
56-
2. Setup methods still receive a `seed_generator` parameter (it's never `None`)
48+
2. Setup methods still receive a `seed_generator` parameter
5749
3. `derive_seed()` returns `None` instead of an integer
58-
4. This `None` flows directly to model adapters (which accept `Optional[int]`)
59-
60-
This design simplifies setup method implementations—you don't need `if seed_generator is not None:` checks:
6150

6251
```python
63-
def setup_agents(self, agent_data, environment, task, user, seed_generator):
64-
# Always works - seed_generator is never None
65-
agent_gen = seed_generator.child("agents")
66-
67-
# Returns None if seeding disabled, int if enabled
68-
agent_seed = agent_gen.derive_seed("orchestrator")
52+
class MyBenchmark(Benchmark):
53+
...
54+
def setup_agents(self, agent_data, environment, task, user, seed_generator):
55+
# Always works - seed_generator is never None
56+
agent = MyAgent(seed=seed_generator("agents/orchestrator"))
57+
...
6958

70-
# Model adapters accept Optional[int], so None works fine
71-
model = self.get_model_adapter(model_id, seed=agent_seed)
59+
# No seed = seeding disabled
60+
benchmark = MyBenchmark(seed=None)
7261
```
7362

7463
### Using Seeds in Setup Methods
@@ -93,9 +82,8 @@ class MyBenchmark(Benchmark):
9382
agent_gen = seed_generator.child("agents")
9483
agent_seed = agent_gen.derive_seed("orchestrator")
9584

96-
# Pass seed to model adapter (adapters accept Optional[int])
97-
model = self.get_model_adapter(model_id, seed=agent_seed)
98-
agent = MyAgent(model=model)
85+
# Pass seed directly to your agent
86+
agent = MyAgent(seed=agent_seed)
9987
# ... rest of setup
10088
```
10189

maseval/core/seeding.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@ def setup_agents(self, agent_data, environment, task, user, seed_generator):
2929
experimental_seed = agent_gen.derive_seed("experimental", per_repetition=True)
3030
baseline_seed = agent_gen.derive_seed("baseline", per_repetition=False)
3131
32-
# Seeds flow directly to model adapters (which accept Optional[int])
33-
model = self.get_model_adapter(model_id, seed=orchestrator_seed)
32+
# Pass seed directly to your agent
33+
agent = MyAgent(seed=orchestrator_seed)
3434
```
3535
"""
3636

@@ -190,8 +190,8 @@ class DefaultSeedGenerator(SeedGenerator):
190190
orchestrator_seed = agent_gen.derive_seed("orchestrator") # Path: "agents/orchestrator"
191191
baseline_seed = agent_gen.derive_seed("baseline", per_repetition=False) # Constant
192192
193-
# Seeds flow directly to model adapters
194-
model = get_model_adapter(model_id, seed=orchestrator_seed) # Works with None
193+
# Pass seed directly to your agent
194+
agent = MyAgent(seed=orchestrator_seed) # Works with None
195195
```
196196
197197
Thread Safety:

0 commit comments

Comments
 (0)