Skip to content

Build one AsyncOpenAI and one Compactor per engine run#50

Merged
declanatinference merged 5 commits into
mainfrom
inf-2933-halo-cleanup
May 22, 2026
Merged

Build one AsyncOpenAI and one Compactor per engine run#50
declanatinference merged 5 commits into
mainfrom
inf-2933-halo-cleanup

Conversation

@declanatinference
Copy link
Copy Markdown
Collaborator

@declanatinference declanatinference commented May 19, 2026

Summary

  • Engine reads engine_config.model_provider in exactly one place (engine.main.stream_engine_async), builds the AsyncOpenAI + Compactor once per run, installs the SDK default with use_for_tracing=False, and stashes both on EngineRunState so the subagent path reuses them.
  • OpenAiAgentRunner now takes a Compactor directly — the compactor_factory indirection and its unused AgentExecution arg are gone.
  • The per-run AsyncOpenAI is closed in stream_engine_async's outer finally, wrapped in try/except so close-time errors can't mask in-flight exceptions.
  • Deletes configure_default_sdk_client, build_compactor_factory, the CompactorFactory alias, and SynthesisTool's lazy-construct branch.
  • Test cleanup: shared noop_compactor exported from tests.probes.probe_kit (collapses 7 duplicate local stubs); build_compactor monkeypatch replaces the deeper build_compactor_factory patch; the subagent test no longer needs its own monkeypatch since EngineRunState carries the compactor.

Closes INF-2933 items 8 and 9.

Test plan

  • uv run pytest -q — 276 passed (baseline 278 − 3 deleted test_configure_default_sdk_client_* + 1 new SDK-default test)
  • uv run pytest tests/integration -q — 54 passed (+ 2 new lifecycle tests in test_engine_client_lifecycle.py covering normal exit and early consumer break). One pre-existing live-API flake unrelated to this change.
  • Stale-reference greps clean: configure_default_sdk_client, build_compactor_factory, compactor_factory, CompactorFactory all zero hits across engine/ and tests/.
  • basedpyright clean on the changed engine files.

Behaviour changes (intentional)

  1. SDK default is always installed. Previously skipped when base_url, api_key, and default_headers were all None. Now always installs AsyncOpenAI(...) (equivalent to env-driven AsyncOpenAI() in that case) with use_for_tracing=False.
  2. Provider misconfiguration fails eagerly. If OPENAI_API_KEY is missing and model_provider.api_key is None, AsyncOpenAI() raises at engine startup instead of lazily on first compactor/synthesis call.
  3. Connection pool closed on run exit. await client.close() in stream_engine_async's outer finally — no behaviour change visible from the public API, just cleaner httpx lifecycle.

Note

Medium Risk
Medium risk because it changes engine startup/client lifecycle (eager AsyncOpenAI construction, global set_default_openai_client, and explicit close() teardown) and rewires compaction/synthesis call paths used by both root and subagents.

Overview
The engine now creates a single per-run AsyncOpenAI client in stream_engine_async, installs it as the Agents SDK default with use_for_tracing=False, stores it on EngineRunState, and closes it on run teardown (including early consumer break) with close failures logged but not raised.

Compaction is refactored from a pluggable callable/factory into a direct call to engine.agents.compactor.compact(client, compaction_model, item): AgentContext.compact_old_items now takes the run client, OpenAiAgentRunner takes the client directly, and the old build_compactor_factory/CompactorFactory indirection is removed. SynthesisTool similarly stops lazily constructing its own client and instead requires the injected run client (wired from run_state.openai_client).

The previous test seam via RunnerProtocol and a per-run runner arg is removed; tests/probes now patch agents.Runner.run_streamed (via a new install_fake_runner helper), add coverage for SDK default installation + client close lifecycle, and set a placeholder OPENAI_API_KEY for non-live test runs to satisfy eager client construction.

Reviewed by Cursor Bugbot for commit 44c492b. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread tests/probes/probe_kit.py
Comment thread tests/unit/test_main.py Outdated
Comment thread tests/conftest.py Outdated
Comment thread engine/main.py Outdated
Comment thread engine/main.py
Comment thread engine/agents/compactor.py Outdated


def build_compactor_factory(
def build_compactor(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Id avoid builder methods for simple functions, instead expose a pure function like async def compact(client: AsyncOpenAI, engine_config: EngineConfig, item: AgentContextItem) -> str so that we dont need to be nesting functions and passing them around. Or if too much context to drill down we can create a compactor class which is initialized in the same place we call build_compactor and then passed around.

declanatinference added a commit that referenced this pull request May 22, 2026
Per francescov1's PR #50 review: drop the build_compactor builder and the
Compactor type alias in favor of a pure async def compact(*, client,
compaction_model, item) -> str. AgentContext.compact_old_items now takes
the AsyncOpenAI directly and calls compact() with self.compaction_model,
removing the closure and the compactor field on EngineRunState.

Tests that monkeypatched build_compactor now monkeypatch
engine.agents.agent_context.compact instead; example_compaction.py
switches from a recording-callable stub to a duck-typed _FakeAsyncOpenAI
plus observable post-state assertions.
Comment thread tests/integration/test_engine_client_lifecycle.py
- Engine builds one AsyncOpenAI and installs the SDK default with
  use_for_tracing=False at run start; closed in stream_engine_async's
  outer finally so connection-pool lifecycle is deterministic.
- Compaction is a pure async def compact(*, client, compaction_model,
  item) -> str. build_compactor / CompactorFactory / Compactor type
  alias are gone; AgentContext.compact_old_items takes the client
  directly and calls compact() with self.compaction_model.
- OpenAiAgentRunner takes the client (not a compactor); the runner_protocol
  seam is deleted from production code (tests monkeypatch via the SDK seam).
- SynthesisTool requires its injected client; lazy construction is gone.
- Tests: shared probe_kit helpers updated; integration tests cover client
  lifecycle (normal exit + early consumer break); compaction tests
  monkeypatch agent_context.compact instead of build_compactor; example
  probes switched to duck-typed fakes and post-state assertions.

Closes INF-2933 items 8 and 9.
Match the real compact() signature on the keyword-only params (client,
compaction_model, item) so a signature change breaks visibly instead of
being silently absorbed by **_kwargs. Mirrors the pattern already used
by fake_compact in test_engine_compaction.py.
Comment thread tests/unit/tools/test_subagent_tool_factory.py
Match the signature shape of the sibling _capturing_run_streamed at the
top of the file: keyword-only starting_agent / input / context /
max_turns / run_config with types and a concrete return. Replaces the
`*args, **kwargs` and `**_kwargs` variants the previous monkeypatch
seam swap left in place, so a future SDK signature change breaks
visibly rather than getting absorbed by the catchall.
Closes bugbot 3269820113 (FakeRunner) and tightens the other PR-introduced
stubs to the same shape:

- FakeRunner.run_streamed: declare max_turns and run_config explicitly
  (probes already assert on calls[i]["max_turns"] / "run_config").
- _FakeCompletions.create: declare model/messages/temperature so a
  signature change in compact() breaks visibly here, not silently.
- test_engine_client_lifecycle: extract _install_stub_client helper with
  typed AsyncOpenAI / set_default_openai_client stand-ins, drops the
  lambda **kw / lambda *a, **kw pair from both lifecycle tests.
- test_main: _capture_client and the set_default recorder now declare
  the kwargs they actually receive (base_url/api_key/default_headers,
  use_for_tracing).
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 44c492b. Configure here.

async def fake_compact(*, client, compaction_model, item: AgentContextItem) -> str:
del client, compaction_model
compacted_items.append((item.item_id, item.role))
return f"summary for {item.item_id}"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test stub missing type annotations on parameters

Low Severity

The fake_compact stub in test_engine_compaction.py declares client and compaction_model without type annotations, while every other stub for the same compact function in this PR (in test_agent_context.py, test_engine_client_lifecycle.py, test_main.py) explicitly types them as client: AsyncOpenAI and compaction_model: ModelConfig. This inconsistency means a signature change in the real compact function won't be caught by static analysis at this call site.

Fix in Cursor Fix in Web

Triggered by learned rule: Use explicit typed parameters in test stubs, not **kwargs

Reviewed by Cursor Bugbot for commit 44c492b. Configure here.

@declanatinference declanatinference merged commit 3b65426 into main May 22, 2026
4 checks passed
francescov1 pushed a commit that referenced this pull request May 23, 2026
…bal default

PR #50 ("Isolate the OpenAI Agents SDK") replaced an env-var-driven
configure_default_sdk_client(provider) with an unconditional
set_default_openai_client(client, use_for_tracing=False) inside
stream_engine_async. That works in production (where every Runner.run_streamed
goes through that entrypoint) but breaks tests that invoke the call_subagent
tool directly via tests/integration/tool_isolation_kit.wired_tools — those
never enter stream_engine_async, so the SDK default stays None and
OpenAIProvider lazy-constructs its own AsyncOpenAI from OPENAI_BASE_URL /
OPENAI_API_KEY env vars, dropping default_headers and any deterministic
close path. test_call_subagent_through_sdk_adapter_live started failing on
2026-05-22 with `APIConnectionError: Connection error.` raised mid-stream
against a request that never left the runner.

Fix: pass model_provider=OpenAIProvider(openai_client=client) on every
RunConfig the engine constructs. This makes prod and tests symmetric —
both paths now wire HALO's configured AsyncOpenAI to the SDK explicitly,
no global state, no env-var fallback. The engine still owns the client's
lifecycle (constructed in stream_engine_async, closed in its finally;
the wired_tools test path owns its own instance).

Bonus: removes a layer of SDK global state, which was already a footgun
for callers running multiple engines / multiple test modules in one
process.

Closes the regression that motivated the (now-closed) workaround PR #54.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants