Improvement of Non-Seeding Workflow (#27)

  Simplify seeding API with null-safe pattern                                                                                                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                                                                              
  Change seed_generator from Optional[SeedGenerator] to SeedGenerator in all                                                                                                                                                                                                                                                  
  benchmark setup methods. When seeding is disabled (seed=None), derive_seed()                                                                                                                                                                                                                                                
  returns None instead of the generator being None. This eliminates conditional                                                                                                                                                                                                                                               
  checks throughout the codebase - the same code works whether seeding is                                                                                                                                                                                                                                                     
  enabled or disabled.                                                                                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                                                                                                              
  - Update Benchmark base class and all setup method signatures                                                                                                                                                                                                                                                               
  - Update DefaultSeedGenerator to accept global_seed=None                                                                                                                                                                                                                                                                    
  - Update all benchmarks (GAIA2, MACS, MultiAgentBench, Tau2)                                                                                                                                                                                                                                                                
  - Update seeding documentation and examples                                                                                                                                                                                                                                                                                 
  - Update all tests to use new pattern

Author: cemde

CHANGELOG.md

@@ -49,6 +49,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+**Core**
+
+- Simplified seeding API: `seed_generator` parameter in setup methods is now always non-None (`SeedGenerator` instead of `Optional[SeedGenerator]`). When seeding is disabled (`seed=None`), `derive_seed()` returns `None` instead of raising an error. This eliminates all `if seed_generator is not None:` conditional checks - the same code path works whether seeding is enabled or disabled. (PR: #27)
+
 **Benchmarks**
 
 - `MACSBenchmark` and `Tau2Benchmark` benchmarks now actively use the seeding system by deriving seeds for model adapters. Seeds are passed to agents, user simulators, tool simulators, and LLM-based evaluators for reproducible runs. (PR: #26)

docs/benchmark/gaia2.md

@@ -46,7 +46,7 @@ configure_model_ids(tasks, evaluator_model_id="gpt-4o")
 
 # Create your framework-specific benchmark subclass
 class MyGaia2Benchmark(Gaia2Benchmark):
-    def setup_agents(self, agent_data, environment, task, user, seed_generator=None):
+    def setup_agents(self, agent_data, environment, task, user, seed_generator):
         tools = environment.create_tools()
         # Create your agent with these tools
         ...

docs/guides/seeding.md

@@ -38,13 +38,34 @@ results = benchmark.run(tasks, agent_data=config)
 
 This creates a `DefaultSeedGenerator` internally and passes it to all setup methods.
 
+### Disabling Seeding
+
+If you don't need seeding, you can simply ignore the seed generators. However, in workflows where you mix seeded and non-seeded runs, you can disable seeding without writing `if/else` statements to check whether a seed is provided.
+
+To disable seeding, omit the `seed` parameter when creating your `Benchmark` or `DefaultSeedGenerator` (or pass `seed=None`):
+
+1. A `DefaultSeedGenerator(global_seed=None)` is still created internally
+2. Setup methods still receive a `seed_generator` parameter
+3. `derive_seed()` returns `None` instead of an integer
+
+```python
+class MyBenchmark(Benchmark):
+    ...
+    def setup_agents(self, agent_data, environment, task, user, seed_generator):
+        # Always works - seed_generator is never None
+        agent = MyAgent(seed=seed_generator("agents/orchestrator"))
+        ...
+
+# No seed = seeding disabled
+benchmark = MyBenchmark(seed=None)
+```
+
 ### Using Seeds in Setup Methods
 
-All setup methods receive an optional `seed_generator` parameter. Use it to derive seeds for your components:
+All setup methods receive a `seed_generator` parameter. Use it to derive seeds for your components. When seeding is disabled (no `seed` passed to benchmark), `derive_seed()` returns `None`:
 
 ```python
 from maseval import Benchmark, SeedGenerator
-from typing import Optional
 
 class MyBenchmark(Benchmark):
     def setup_agents(
@@ -53,18 +74,16 @@ class MyBenchmark(Benchmark):
         environment,
         task,
         user,
-        seed_generator: Optional[SeedGenerator] = None,
+        seed_generator: SeedGenerator,
     ):
         # Derive a seed for your agent using hierarchical paths
-        agent_seed = None
-        if seed_generator is not None:
-            # Use child() to create logical namespaces - results in "agents/orchestrator"
-            agent_gen = seed_generator.child("agents")
-            agent_seed = agent_gen.derive_seed("orchestrator")
-
-        # Pass seed to model adapter
-        model = self.get_model_adapter(model_id, seed=agent_seed)
-        agent = MyAgent(model=model)
+        # Returns None if seeding is disabled (global_seed=None)
+        # Use child() to create logical namespaces - results in "agents/orchestrator"
+        agent_gen = seed_generator.child("agents")
+        agent_seed = agent_gen.derive_seed("orchestrator")
+
+        # Pass seed directly to your agent
+        agent = MyAgent(seed=agent_seed)
         # ... rest of setup
 ```
 
@@ -75,18 +94,17 @@ Seeds are derived from hierarchical paths, so `derive_seed("orchestrator")` with
 When running multiple repetitions of the same task, you may want some components to vary while others remain constant. The `per_repetition` flag controls this:
 
 ```python
-def setup_agents(self, agent_data, environment, task, user, seed_generator=None):
-    if seed_generator is not None:
-        # Use child() to group agent seeds under "agents/" namespace
-        agent_gen = seed_generator.child("agents")
+def setup_agents(self, agent_data, environment, task, user, seed_generator):
+    # Use child() to group agent seeds under "agents/" namespace
+    agent_gen = seed_generator.child("agents")
 
-        # Varies per repetition - different seed for rep 0, 1, 2, ...
-        # Results in path: "agents/experimental"
-        experimental_seed = agent_gen.derive_seed("experimental", per_repetition=True)
+    # Varies per repetition - different seed for rep 0, 1, 2, ...
+    # Results in path: "agents/experimental"
+    experimental_seed = agent_gen.derive_seed("experimental", per_repetition=True)
 
-        # Constant across repetitions - same seed for rep 0, 1, 2, ...
-        # Results in path: "agents/baseline"
-        baseline_seed = agent_gen.derive_seed("baseline", per_repetition=False)
+    # Constant across repetitions - same seed for rep 0, 1, 2, ...
+    # Results in path: "agents/baseline"
+    baseline_seed = agent_gen.derive_seed("baseline", per_repetition=False)
 ```
 
 **Use cases:**
@@ -101,26 +119,24 @@ def setup_agents(self, agent_data, environment, task, user, seed_generator=None)
 For complex systems with many components, use `child()` to create hierarchical namespaces:
 
 ```python
-def setup_environment(self, agent_data, task, seed_generator=None):
-    if seed_generator is not None:
-        # Create a child generator for environment components
-        env_gen = seed_generator.child("environment")
-
-        # Further nest tools under "environment/tools/"
-        tools_gen = env_gen.child("tools")
-        weather_seed = tools_gen.derive_seed("weather")  # "environment/tools/weather"
-        search_seed = tools_gen.derive_seed("search")    # "environment/tools/search"
-
-def setup_agents(self, agent_data, environment, task, user, seed_generator=None):
-    if seed_generator is not None:
-        # Create a child generator for agents
-        agent_gen = seed_generator.child("agents")
+def setup_environment(self, agent_data, task, seed_generator):
+    # Create a child generator for environment components
+    env_gen = seed_generator.child("environment")
+
+    # Further nest tools under "environment/tools/"
+    tools_gen = env_gen.child("tools")
+    weather_seed = tools_gen.derive_seed("weather")  # "environment/tools/weather"
+    search_seed = tools_gen.derive_seed("search")    # "environment/tools/search"
+
+def setup_agents(self, agent_data, environment, task, user, seed_generator):
+    # Create a child generator for agents
+    agent_gen = seed_generator.child("agents")
 
-        orchestrator_seed = agent_gen.derive_seed("orchestrator")  # "agents/orchestrator"
+    orchestrator_seed = agent_gen.derive_seed("orchestrator")  # "agents/orchestrator"
 
-        # Nest workers under "agents/workers/"
-        worker_gen = agent_gen.child("workers")
-        analyst_seed = worker_gen.derive_seed("analyst")           # "agents/workers/analyst"
+    # Nest workers under "agents/workers/"
+    worker_gen = agent_gen.child("workers")
+    analyst_seed = worker_gen.derive_seed("analyst")           # "agents/workers/analyst"
 ```
 
 Child generators share the same seed log, so all derived seeds are recorded together.

examples/five_a_day_benchmark/five_a_day_benchmark.ipynb

@@ -523,7 +523,7 @@
    "id": "70c66cd0",
    "metadata": {},
    "outputs": [],
-   "source": "class FiveADayBenchmark(Benchmark):\n    \"\"\"5-A-Day benchmark with multi-agent support.\"\"\"\n\n    def setup_environment(self, agent_data: Dict[str, Any], task: Task, seed_generator: Optional[SeedGenerator] = None) -> Environment:\n        \"\"\"Create environment from task data.\"\"\"\n        task_data = {\n            \"environment_data\": task.environment_data,\n            \"query\": task.query,\n            \"evaluation_data\": task.evaluation_data,\n            \"metadata\": task.metadata,\n        }\n\n        environment = FiveADayEnvironment(task_data)\n\n        # Register all tools for tracing\n        for tool_name, tool_adapter in environment.get_tools().items():\n            self.register(\"tools\", tool_name, tool_adapter)\n\n        return environment\n\n    def setup_agents(\n        self,\n        agent_data: Dict[str, Any],\n        environment: Environment,\n        task: Task,\n        user=None,\n        seed_generator: Optional[SeedGenerator] = None,\n    ) -> tuple[list[SmolAgentAdapter], Dict[str, SmolAgentAdapter]]:\n        \"\"\"Create multi-agent system with orchestrator and specialists.\n\n        If seed_generator is provided, seeds are derived for each agent\n        using the benchmark's seeding system with hierarchical paths.\n        \"\"\"\n        # Build seeds dict if seed_generator is available\n        # Use child(\"agents\") to create logical paths like \"agents/primary_agent\"\n        seeds = None\n        if seed_generator is not None:\n            agent_gen = seed_generator.child(\"agents\")\n            seeds = {}\n            for agent_spec in agent_data[\"agents\"]:\n                seeds[agent_spec[\"agent_id\"]] = agent_gen.derive_seed(agent_spec[\"agent_id\"])\n\n        agents_to_run, agents_to_monitor = build_agents(agent_data, environment, seeds)\n\n        # Create adapters for the primary agent(s) to run\n        adapters_to_run = [SmolAgentAdapter(agent, agent.name) for agent in agents_to_run]\n\n        # This ensures all agent traces are collected by the benchmark\n        all_agents = {agent.name: agent for agent in agents_to_run} | agents_to_monitor\n        adapters_to_monitor = {name: SmolAgentAdapter(agent, name) for name, agent in all_agents.items()}\n        return adapters_to_run, adapters_to_monitor\n\n    def setup_evaluators(self, environment, task, agents, user, seed_generator: Optional[SeedGenerator] = None) -> Sequence[Evaluator]:\n        \"\"\"Create evaluators based on task's evaluation criteria.\"\"\"\n        if not task.evaluation_data[\"evaluators\"]:\n            return []\n\n        evaluator_instances = []\n        for name in task.evaluation_data[\"evaluators\"]:\n            evaluator_class = getattr(evaluators, name)\n            evaluator_instances.append(evaluator_class(task, environment, user))\n\n        return evaluator_instances\n\n    def run_agents(self, agents: Sequence[AgentAdapter], task: Task, environment: Environment, query: str) -> Sequence[Any]:\n        \"\"\"Execute agents and return their final answers.\"\"\"\n        answers = [agent.run(query) for agent in agents]\n        return answers\n\n    def get_model_adapter(self, model_id: str, **kwargs) -> ModelAdapter:\n        \"\"\"Return a model adapter for benchmark components that need LLM access.\n\n        This benchmark doesn't use simulated tools, user simulators, or LLM judges,\n        so this method is not called during execution.\n        \"\"\"\n        raise NotImplementedError(\"This benchmark doesn't use model adapters for tools/users/evaluators.\")\n\n    def evaluate(\n        self,\n        evaluators: Sequence[Evaluator],\n        agents: Dict[str, AgentAdapter],\n        final_answer: Any,\n        traces: Dict[str, Any],\n    ) -> list[Dict[str, Any]]:\n        \"\"\"Evaluate agent performance.\"\"\"\n        results = []\n        for evaluator in evaluators:\n            filtered_traces = evaluator.filter_traces(traces)\n            results.append(evaluator(filtered_traces, final_answer))\n        return results"
+   "source": "class FiveADayBenchmark(Benchmark):\n    \"\"\"5-A-Day benchmark with multi-agent support.\"\"\"\n\n    def setup_environment(self, agent_data: Dict[str, Any], task: Task, seed_generator: SeedGenerator) -> Environment:\n        \"\"\"Create environment from task data.\"\"\"\n        task_data = {\n            \"environment_data\": task.environment_data,\n            \"query\": task.query,\n            \"evaluation_data\": task.evaluation_data,\n            \"metadata\": task.metadata,\n        }\n\n        environment = FiveADayEnvironment(task_data)\n\n        # Register all tools for tracing\n        for tool_name, tool_adapter in environment.get_tools().items():\n            self.register(\"tools\", tool_name, tool_adapter)\n\n        return environment\n\n    def setup_agents(\n        self,\n        agent_data: Dict[str, Any],\n        environment: Environment,\n        task: Task,\n        user,\n        seed_generator: SeedGenerator,\n    ) -> tuple[list[SmolAgentAdapter], Dict[str, SmolAgentAdapter]]:\n        \"\"\"Create multi-agent system with orchestrator and specialists.\n\n        Seeds are derived for each agent using the benchmark's seeding system\n        with hierarchical paths. derive_seed() returns None if seeding is disabled.\n        \"\"\"\n        # Build seeds dict using seed_generator\n        # Use child(\"agents\") to create logical paths like \"agents/primary_agent\"\n        agent_gen = seed_generator.child(\"agents\")\n        seeds = {}\n        for agent_spec in agent_data[\"agents\"]:\n            seeds[agent_spec[\"agent_id\"]] = agent_gen.derive_seed(agent_spec[\"agent_id\"])\n\n        agents_to_run, agents_to_monitor = build_agents(agent_data, environment, seeds)\n\n        # Create adapters for the primary agent(s) to run\n        adapters_to_run = [SmolAgentAdapter(agent, agent.name) for agent in agents_to_run]\n\n        # This ensures all agent traces are collected by the benchmark\n        all_agents = {agent.name: agent for agent in agents_to_run} | agents_to_monitor\n        adapters_to_monitor = {name: SmolAgentAdapter(agent, name) for name, agent in all_agents.items()}\n        return adapters_to_run, adapters_to_monitor\n\n    def setup_evaluators(self, environment, task, agents, user, seed_generator: SeedGenerator) -> Sequence[Evaluator]:\n        \"\"\"Create evaluators based on task's evaluation criteria.\"\"\"\n        if not task.evaluation_data[\"evaluators\"]:\n            return []\n\n        evaluator_instances = []\n        for name in task.evaluation_data[\"evaluators\"]:\n            evaluator_class = getattr(evaluators, name)\n            evaluator_instances.append(evaluator_class(task, environment, user))\n\n        return evaluator_instances\n\n    def run_agents(self, agents: Sequence[AgentAdapter], task: Task, environment: Environment, query: str) -> Sequence[Any]:\n        \"\"\"Execute agents and return their final answers.\"\"\"\n        answers = [agent.run(query) for agent in agents]\n        return answers\n\n    def get_model_adapter(self, model_id: str, **kwargs) -> ModelAdapter:\n        \"\"\"Return a model adapter for benchmark components that need LLM access.\n\n        This benchmark doesn't use simulated tools, user simulators, or LLM judges,\n        so this method is not called during execution.\n        \"\"\"\n        raise NotImplementedError(\"This benchmark doesn't use model adapters for tools/users/evaluators.\")\n\n    def evaluate(\n        self,\n        evaluators: Sequence[Evaluator],\n        agents: Dict[str, AgentAdapter],\n        final_answer: Any,\n        traces: Dict[str, Any],\n    ) -> list[Dict[str, Any]]:\n        \"\"\"Evaluate agent performance.\"\"\"\n        results = []\n        for evaluator in evaluators:\n            filtered_traces = evaluator.filter_traces(traces)\n            results.append(evaluator(filtered_traces, final_answer))\n        return results"
   },
   {
    "cell_type": "markdown",

examples/five_a_day_benchmark/five_a_day_benchmark.py

@@ -729,7 +729,7 @@ class FiveADayBenchmark(Benchmark):
     Supports single-agent and multi-agent (orchestrator+specialist) configurations.
     """
 
-    def setup_environment(self, agent_data: Dict[str, Any], task: Task, seed_generator: Optional[SeedGenerator] = None) -> Environment:
+    def setup_environment(self, agent_data: Dict[str, Any], task: Task, seed_generator: SeedGenerator) -> Environment:
         """Create environment from task data."""
         # Pass full task data to environment
         task_data = {
@@ -753,8 +753,8 @@ def setup_agents(
         agent_data: Dict[str, Any],
         environment: Environment,
         task: Task,
-        user=None,
-        seed_generator: Optional[SeedGenerator] = None,
+        user,
+        seed_generator: SeedGenerator,
     ) -> tuple[List[AgentAdapter], Dict[str, AgentAdapter]]:
         """Create framework-specific agent with tools from environment.
 
@@ -775,14 +775,13 @@ def setup_agents(
         primary_spec = next(a for a in agents_specs if a["agent_id"] == primary_agent_id)
         specialist_specs = [a for a in agents_specs if a["agent_id"] != primary_agent_id]
 
-        # Derive seeds for agents using seed_generator if available
+        # Derive seeds for agents using seed_generator
         # Use child("agents") to create logical paths like "agents/primary_agent"
-        seeds = None
-        if seed_generator is not None:
-            agent_gen = seed_generator.child("agents")
-            seeds = {primary_spec["agent_id"]: agent_gen.derive_seed(primary_spec["agent_id"])}
-            for spec in specialist_specs:
-                seeds[spec["agent_id"]] = agent_gen.derive_seed(spec["agent_id"])
+        # derive_seed() returns None if seeding is disabled
+        agent_gen = seed_generator.child("agents")
+        seeds = {primary_spec["agent_id"]: agent_gen.derive_seed(primary_spec["agent_id"])}
+        for spec in specialist_specs:
+            seeds[spec["agent_id"]] = agent_gen.derive_seed(spec["agent_id"])
 
         # Build agent using unified interface - now returns (primary_adapter, all_adapters_dict)
         builder = get_agent_builder(framework, agent_type)
@@ -791,7 +790,7 @@ def setup_agents(
         # Return primary adapter to run, and all adapters for trace registration
         return [primary_adapter], all_adapters_dict
 
-    def setup_evaluators(self, environment, task, agents, user, seed_generator: Optional[SeedGenerator] = None) -> Sequence[Evaluator]:
+    def setup_evaluators(self, environment, task, agents, user, seed_generator: SeedGenerator) -> Sequence[Evaluator]:
         """Create evaluators based on task's evaluation_data.evaluators list."""
         if not task.evaluation_data["evaluators"]:
             return []

examples/macs_benchmark/macs_benchmark.py

@@ -179,7 +179,7 @@ def setup_user(
         agent_data: Dict[str, Any],
         environment: Environment,
         task: Task,
-        seed_generator: Optional[SeedGenerator] = None,
+        seed_generator: SeedGenerator,
     ) -> SmolagentsMACSUser:
         """Create smolagents-compatible user simulator.
 
@@ -210,7 +210,7 @@ def setup_agents(
         environment: MACSEnvironment,  # type: ignore[override]
         task: Task,
         user: Optional[User],
-        seed_generator: Optional[SeedGenerator] = None,
+        seed_generator: SeedGenerator,
     ) -> Tuple[List[AgentAdapter], Dict[str, AgentAdapter]]:
         """Create smolagents multi-agent hierarchy.
 
@@ -435,7 +435,7 @@ def setup_user(
         agent_data: Dict[str, Any],
         environment: Environment,
         task: Task,
-        seed_generator: Optional[SeedGenerator] = None,
+        seed_generator: SeedGenerator,
     ) -> LangGraphMACSUser:
         """Create langgraph-compatible user simulator.
 
@@ -466,7 +466,7 @@ def setup_agents(
         environment: MACSEnvironment,  # type: ignore[override]
         task: Task,
         user: Optional[User],
-        seed_generator: Optional[SeedGenerator] = None,
+        seed_generator: SeedGenerator,
     ) -> Tuple[List[AgentAdapter], Dict[str, AgentAdapter]]:
         """Create langgraph multi-agent hierarchy.