[P3.1] Codex adapter (pilot) — build + parse + capabilities + tests + docs

[frankbria]

Phase 3 — Pilot provider. First non-Claude adapter; proves the seam end-to-end.

Scope

• lib/agents/codex.sh:
  • Build: codex exec [prompt] (prompt via arg or stdin), --json (JSONL events),
    -o/--output-last-message <file> for the final message, -m <model>, resume via
    codex exec resume <id>, approval via -s <sandbox> / --dangerously-bypass-….
  • Parse: map Codex JSONL events -> internal analysis struct (assistant/tool/result,
    end-of-turn, usage if present, errors). Use --output-last-message for the final
    answer; consider --output-schema for a structured RALPH_STATUS if helpful.
  • Capabilities: structured=yes, token usage=verify, permission_denials=no(map to
    sandbox/approval), api_limit_detection=codex-specific or off, session=resume-id.
• Unit tests via [P2.2] harness + an E2E with a mock codex CLI.
• Docs: docs/providers/CODEX.md, README provider matrix row, .ralphrc notes.

Acceptance

• ralph --provider codex runs the loop against a mock; exit detection works via
  RALPH_STATUS; unsupported features degrade per [P2.1]. Full suite green.

Depends on: [P1.1], [P1.2], [P1.3], [P2.1], [P2.2]. Blocks: [P4.x] (as template).

[github-actions]

🏷️ Triage: Feature / Enhancement

This issue is a planned implementation task — [P3.1] Codex adapter, the pilot provider in the multi-phase multi-provider effort. Categorizing it as an enhancement (new provider adapter behind the agent-abstraction seam).

Why this category

• Not a bug — it describes net-new functionality (lib/agents/codex.sh), not a defect.
• Not a duplicate — no other open issue covers the Codex adapter. It's the first non-Claude adapter and serves as the template for the P4.x provider issues.
• Clear scope + acceptance criteria, so no additional info is needed.

Dependency context (from the open multi-provider issues)

• Blocked by: [P1.1] [P1.1] Command-construction seam: lib/agents/ registry + Claude reference adapter #312, [P1.2] [P1.2] Output-normalization seam: internal analysis struct + Claude parser #313, [P1.3] [P1.3] Provider selection surface: AGENT_PROVIDER config + CLI + tmux forwarding #314, [P2.1] [P2.1] Capabilities matrix + graceful feature degradation #315, [P2.2] [P2.2] Generic adapter test harness (mock CLIs) #316 — the abstraction seam, capabilities/degradation layer, and generic test harness should land first.
• Blocks: the Phase 4 adapters — [P4.1] Gemini [P4.1] Gemini adapter — build + parse + capabilities + tests + docs #318, [P4.2] OpenCode [P4.2] OpenCode adapter — build + parse + capabilities + tests + docs #319, [P4.3] Droid [P4.3] Droid adapter — build + parse + capabilities + tests + docs #320, [P4.4] Kilocode [P4.4] Kilocode adapter — build + parse + capabilities + tests + docs #321, [P4.5] Copilot [P4.5] Copilot adapter (text-only, degraded) — build + parse + capabilities + tests + docs #322 — which reuse this as their template.

Recommended next steps

1. Confirm the upstream dependencies ([P1.1] Command-construction seam: lib/agents/ registry + Claude reference adapter #312–[P2.2] Generic adapter test harness (mock CLIs) #316) are resolved before starting, since this proves the seam end-to-end.
2. Land docs/providers/CODEX.md + the README provider-matrix row alongside the code, per the acceptance criteria.
3. Validate via the [P2.2] [P2.2] Generic adapter test harness (mock CLIs) #316 mock-CLI harness plus the E2E mock codex CLI (remember the >1s-per-call early-failure rule).

The existing multi-provider label is retained for phase tracking.

Generated by Issue Triage Assistant for issue #317 · opus48 398.3K · ◷

[coderabbitai]

Coding Plan

Summary

• Create lib/agents/codex.sh as a self-contained adapter module following Claude's distributed pattern but consolidated into a single sourced file
• Use minimal conditional dispatch in ralph_loop.sh via --provider flag rather than a full abstraction layer, deferring P2.1's interface design until patterns emerge from this pilot
• Leverage existing test infrastructure (mock-binary-on-PATH pattern, e2e_helper.bash) rather than creating new test frameworks
• Parse Codex JSONL events into Ralph's existing internal analysis struct format, ensuring compatibility with the existing exit detection logic that expects RALPH_STATUS blocks

Design Choices

Design Choice 1: Provider abstraction approach without P2.1

Options Considered:

1. Implement full P2.1 provider abstraction as a prerequisite
2. Create minimal conditional dispatch in ralph_loop.sh that can be refactored into P2.1 later
3. Block this ticket until P2.1/P2.2 are complete

Chosen Option: 2

Rationale: Create minimal dispatch with --provider flag and conditional logic—this aligns with the "pilot" nature of the ticket, proves the seam, and informs what P2.1 should abstract.

Design Choice 2: RALPH_STATUS extraction method

Options Considered:

1. Use --output-schema to enforce RALPH_STATUS JSON format directly from Codex
2. Parse RALPH_STATUS from final message text (via --output-last-message) using existing text/regex patterns
3. Support both with schema preferred and text as fallback

Chosen Option: 3

Rationale: Attempt --output-schema if available, fall back to text parsing—this maximizes reliability while leveraging structured output when possible.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Codex Adapter Core Implementation

This phase creates the new adapter file with command construction, JSONL parsing, and capability definitions, then integrates provider selection into the main loop.

Task 1: Create Agent Adapter Directory and Codex Skeleton

Establish the lib/agents/ directory structure and create the Codex adapter file with function signatures matching the Claude adapter patterns.

• Create lib/agents/ directory as the home for all provider adapters
• Create lib/agents/codex.sh with sourcing guards and required function stubs: validate_codex_command(), build_codex_command(), parse_codex_response(), get_codex_capabilities()
• Define CODEX_CMD variable (default: codex) following the CLAUDE_CODE_CMD pattern
• Add version validation function check_codex_version() if Codex CLI provides version info

Task 2: Implement Codex Command Construction

Build the command array for Codex CLI invocation, handling all required flags and execution modes.

• Implement build_codex_command() to construct CODEX_CMD_ARGS array starting with $CODEX_CMD
• Add exec subcommand as the primary invocation mode
• Append --json flag for JSONL event output
• Append -o "$output_file" / --output-last-message "$output_file" for capturing final message
• Add -m $CODEX_MODEL when model override is specified
• Handle session resume via codex exec resume $session_id when CODEX_SESSION_ID is set
• Add approval/sandbox flags: -s $CODEX_SANDBOX or --dangerously-bypass-... based on configuration
• Implement prompt delivery (as argument or via stdin based on CLI requirements)

Task 3: Implement JSONL Response Parsing

Parse Codex's JSONL event stream and map it to Ralph's internal analysis struct format.

• Create parse_codex_jsonl() to iterate through JSONL events line by line
• Handle event types: thread.started, turn.started, turn.completed, turn.failed, item.*, error, token_count
• Extract session ID from thread.started event for resume capability
• Accumulate token usage from token_count events (input, output, reasoning tokens)
• Detect errors from turn.failed and error events
• Track file modifications from relevant item.* events
• Implement extract_final_message() to read --output-last-message file and parse RALPH_STATUS block
• Write results to .ralph/.json_parse_result in the standard format expected by analyze_response()

Task 4: Define Capabilities and Degradation Behavior

Implement capability reporting and graceful degradation for unsupported features.

• Create get_codex_capabilities() returning capability flags:
  • structured_output: true (supports JSON via --json)
  • token_usage: conditional (available via token_count events, verify at runtime)
  • permission_denials: false (Codex uses sandbox/approval model, not per-command denials)
  • api_limit_detection: off (disable or implement Codex-specific rate limit detection)
  • session_resume: true (via codex exec resume <id>)
• Implement codex_supports_capability() helper for runtime feature checks
• Add degradation logging when unsupported features are requested
• Map permission-related behavior to Codex's approval sandbox mode rather than denial tracking

Task 5: Integrate Provider Selection into Main Loop

Add the --provider flag to ralph_loop.sh and implement dispatch logic to select between Claude and Codex adapters.

• Add --provider <name> flag to CLI argument parsing (default: claude)
• Define AGENT_PROVIDER variable following the SANDBOX_PROVIDER pattern
• Source appropriate adapter file based on provider: source "$RALPH_DIR/lib/agents/codex.sh" when AGENT_PROVIDER=codex
• Create dispatch wrapper build_agent_command() that calls provider-specific build function
• Create dispatch wrapper parse_agent_response() that calls provider-specific parse function
• Update execute_claude_code() to become execute_agent() with provider-aware invocation
• Ensure .ralphrc can set AGENT_PROVIDER for default provider selection
• Handle invalid provider values with clear error message listing available providers

🤖 Prompt for AI agents

Create the Codex adapter module (`lib/agents/codex.sh`) and integrate provider
selection into `ralph_loop.sh`. The goal is a self-contained adapter file and
minimal conditional dispatch wired into the main loop.

**1. Create `lib/agents/` directory and `lib/agents/codex.sh`:**
- Add sourcing guards at the top of `codex.sh`
- Define `CODEX_CMD` variable defaulting to `codex`, following the
`CLAUDE_CODE_CMD` pattern
- Add stub/full implementations for: `validate_codex_command()`,
`build_codex_command()`, `parse_codex_response()`, `get_codex_capabilities()`,
`check_codex_version()`

**2. Implement `build_codex_command()`:**
- Build `CODEX_CMD_ARGS` array starting with `$CODEX_CMD`
- Add `exec` subcommand, `--json` flag, and `-o "$output_file"` /
`--output-last-message "$output_file"`
- Conditionally add `-m $CODEX_MODEL` when model override is set
- Handle session resume with `codex exec resume $session_id` when
`CODEX_SESSION_ID` is set
- Add approval/sandbox flags (`-s $CODEX_SANDBOX` or `--dangerously-bypass-...`)
based on configuration
- Handle prompt delivery as argument or via stdin per CLI requirements

**3. Implement `parse_codex_jsonl()`:**
- Iterate JSONL events line by line, handling types: `thread.started`,
`turn.started`, `turn.completed`, `turn.failed`, `item.*`, `error`,
`token_count`
- Extract session ID from `thread.started`; accumulate token usage (input,
output, reasoning) from `token_count`; detect errors from `turn.failed` and
`error`; track file modifications from `item.*` events
- Implement `extract_final_message()` to read the `--output-last-message` file
and parse the RALPH_STATUS block
- Write results to `.ralph/.json_parse_result` in the standard format expected
by `analyze_response()`

**4. Implement `get_codex_capabilities()` and `codex_supports_capability()`:**
- Return flags: `structured_output: true`, `token_usage: conditional`,
`permission_denials: false`, `api_limit_detection: off`, `session_resume: true`
- Add degradation logging when unsupported features are requested
- Map permission-related behavior to Codex's approval sandbox mode

**5. Update `ralph_loop.sh`:**
- Add `--provider <name>` to CLI argument parsing (default: `claude`), following
the `SANDBOX_PROVIDER` pattern for the `AGENT_PROVIDER` variable
- Source `lib/agents/codex.sh` when `AGENT_PROVIDER=codex`
- Add dispatch wrappers `build_agent_command()` and `parse_agent_response()`
that call the provider-specific functions
- Rename `execute_claude_code()` to `execute_agent()` with provider-aware
invocation
- Allow `.ralphrc` to set `AGENT_PROVIDER`
- Emit a clear error message listing available providers for invalid provider
values

Phase 2: Testing

This phase creates the mock Codex CLI infrastructure and comprehensive test coverage using existing test patterns.

Task 1: Create Mock Codex CLI Binary

Build a scriptable mock Codex CLI following the established mock-binary-on-PATH pattern used for docker/gh/e2b-python mocks.

• Create mock binary generator in tests/helpers/ or inline in test setup
• Mock should log all invocations (argv) to $TEST_DIR/codex_calls/ for assertion
• Support mode-based responses: ok, error, rate-limit, no-session
• Generate valid JSONL event sequences for success cases (thread.started → turn.started → item.* → turn.completed)
• Generate error event sequences for failure cases
• Include token_count events in success responses for usage tracking tests
• Support scripted per-call responses via numbered response files (following E2E pattern)

Task 2: Unit Tests for Codex Adapter

Create unit tests for all functions in lib/agents/codex.sh following existing lib test patterns.

• Create tests/unit/test_codex_adapter.bats
• Test validate_codex_command() with available/unavailable binary scenarios
• Test build_codex_command() generates correct argument arrays for various configurations
• Test JSONL parsing with valid event streams, error events, truncated streams
• Test capability reporting returns correct flags
• Test session ID extraction from thread.started events
• Test token usage accumulation from token_count events
• Test RALPH_STATUS extraction from final message file
• Use source pattern to load adapter without running main loop

Task 3: E2E Test for Full Loop with Codex Provider

Create end-to-end test that runs ralph_loop.sh with --provider codex against the mock CLI.

• Create tests/e2e/test_codex_e2e.bats following existing E2E patterns
• Use e2e_helper.bash patterns for project setup and teardown
• Install mock codex binary at $E2E_DIR/bin/codex
• Queue responses that include RALPH_STATUS blocks with various exit signals
• Test successful completion: loop runs, detects EXIT_SIGNAL: true, exits cleanly
• Test continuation: loop continues when EXIT_SIGNAL: false
• Test error handling: loop responds appropriately to error events
• Test session resume: verify resume command uses stored session ID
• Assert call counts and argument patterns via mock logs

🤖 Prompt for AI agents

Create the mock Codex CLI infrastructure and all test files for the Codex
adapter. The goal is comprehensive coverage using the existing
mock-binary-on-PATH and E2E patterns.

**1. Create the mock Codex CLI binary (in `tests/helpers/` or inline in test
setup):**
- Log all invocations (argv) to `$TEST_DIR/codex_calls/` for assertion
- Support mode-based responses: `ok`, `error`, `rate-limit`, `no-session`
- Emit valid JSONL event sequences for success: `thread.started` →
`turn.started` → `item.*` → `turn.completed`, including `token_count` events
- Emit error event sequences for failure cases
- Support scripted per-call responses via numbered response files (following the
existing E2E pattern)

**2. Create `tests/unit/test_codex_adapter.bats`:**
- Use the `source` pattern to load `lib/agents/codex.sh` without running the
main loop
- Write tests for: `validate_codex_command()` (available/unavailable binary),
`build_codex_command()` (correct arg arrays for various configs), JSONL parsing
(valid streams, error events, truncated streams), capability flag reporting,
session ID extraction from `thread.started`, token usage accumulation from
`token_count`, and RALPH_STATUS extraction from the final message file

**3. Create `tests/e2e/test_codex_e2e.bats`:**
- Follow `e2e_helper.bash` patterns for project setup and teardown
- Install the mock codex binary at `$E2E_DIR/bin/codex`
- Queue responses containing RALPH_STATUS blocks with various exit signals
- Test cases: successful completion (EXIT_SIGNAL: true exits cleanly),
continuation (EXIT_SIGNAL: false loops), error handling (loop responds to error
events), session resume (resume command uses stored session ID)
- Assert call counts and argument patterns via mock logs

Phase 3: Documentation

This phase creates provider documentation and updates existing docs to reflect multi-provider support.

Task 1: Create Codex Provider Documentation

Write comprehensive documentation for configuring and using the Codex provider.

• Create docs/providers/ directory (establishing the provider docs location)
• Create docs/providers/CODEX.md with:
  • Overview of Codex CLI and its capabilities
  • Configuration variables: CODEX_CMD, CODEX_MODEL, CODEX_SANDBOX
  • Command-line usage: ralph --provider codex
  • Session resume behavior and limitations
  • Capability comparison with Claude (what works, what degrades)
  • Troubleshooting common issues
  • Example .ralphrc configuration snippet

Task 2: Update README Provider Matrix

Add Codex to the README's provider documentation and create a comparison matrix.

• Add provider comparison table to README showing Claude vs Codex capabilities
• Document --provider flag in CLI options section
• Update the "supported providers" statement to include Codex
• Add cross-reference to docs/providers/CODEX.md for detailed configuration

Task 3: Update Configuration Documentation

Document Codex-related settings in .ralphrc and configuration guides.

• Update .ralphrc template/example with Codex configuration section
• Document AGENT_PROVIDER variable in docs/CLI_OPTIONS.md or equivalent
• Add Codex environment variables to any existing configuration reference
• Note which settings are provider-specific vs universal

🤖 Prompt for AI agents

Create provider documentation and update existing docs to reflect multi-provider
support. The goal is accurate, complete reference material for Codex
configuration and a clear capability comparison.

**1. Create `docs/providers/` directory and `docs/providers/CODEX.md`:**
- Include: overview of Codex CLI and its capabilities; configuration variables
(`CODEX_CMD`, `CODEX_MODEL`, `CODEX_SANDBOX`); command-line usage (`ralph
--provider codex`); session resume behavior and limitations; capability
comparison with Claude (what works, what degrades); troubleshooting common
issues; example `.ralphrc` configuration snippet

**2. Update `README`:**
- Add a provider comparison table showing Claude vs Codex capabilities
- Document the `--provider` flag in the CLI options section
- Update the "supported providers" statement to include Codex
- Add a cross-reference to `docs/providers/CODEX.md`

**3. Update configuration documentation:**
- Add a Codex configuration section to the `.ralphrc` template/example
- Document the `AGENT_PROVIDER` variable in `docs/CLI_OPTIONS.md` or equivalent
- Add Codex environment variables to any existing configuration reference
- Note which settings are provider-specific vs universal

Research

Ralph is currently a single-provider system tightly coupled to the Claude Code CLI. There is no lib/agents/ directory or provider abstraction layer—the Claude adapter logic is distributed across ralph_loop.sh (command construction via build_claude_command(), execution via execute_claude_code()) and lib/response_analyzer.sh (JSON/JSONL parsing, internal analysis struct generation). The test infrastructure uses a mock-binary-on-PATH pattern for external CLIs (docker, gh, e2b-python) and provides comprehensive helpers in tests/helpers/ (test_helper.bash, mocks.bash, fixtures.bash). The E2E harness creates executable mock CLI binaries that log invocations and replay scripted responses. RALPH_STATUS blocks in agent output drive exit detection via a dual-condition gate requiring both completion_indicators >= 2 AND EXIT_SIGNAL: true.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Create the Codex adapter module (`lib/agents/codex.sh`) and integrate provider
selection into `ralph_loop.sh`. The goal is a self-contained adapter file and
minimal conditional dispatch wired into the main loop.

**1. Create `lib/agents/` directory and `lib/agents/codex.sh`:**
- Add sourcing guards at the top of `codex.sh`
- Define `CODEX_CMD` variable defaulting to `codex`, following the
`CLAUDE_CODE_CMD` pattern
- Add stub/full implementations for: `validate_codex_command()`,
`build_codex_command()`, `parse_codex_response()`, `get_codex_capabilities()`,
`check_codex_version()`

**2. Implement `build_codex_command()`:**
- Build `CODEX_CMD_ARGS` array starting with `$CODEX_CMD`
- Add `exec` subcommand, `--json` flag, and `-o "$output_file"` /
`--output-last-message "$output_file"`
- Conditionally add `-m $CODEX_MODEL` when model override is set
- Handle session resume with `codex exec resume $session_id` when
`CODEX_SESSION_ID` is set
- Add approval/sandbox flags (`-s $CODEX_SANDBOX` or `--dangerously-bypass-...`)
based on configuration
- Handle prompt delivery as argument or via stdin per CLI requirements

**3. Implement `parse_codex_jsonl()`:**
- Iterate JSONL events line by line, handling types: `thread.started`,
`turn.started`, `turn.completed`, `turn.failed`, `item.*`, `error`,
`token_count`
- Extract session ID from `thread.started`; accumulate token usage (input,
output, reasoning) from `token_count`; detect errors from `turn.failed` and
`error`; track file modifications from `item.*` events
- Implement `extract_final_message()` to read the `--output-last-message` file
and parse the RALPH_STATUS block
- Write results to `.ralph/.json_parse_result` in the standard format expected
by `analyze_response()`

**4. Implement `get_codex_capabilities()` and `codex_supports_capability()`:**
- Return flags: `structured_output: true`, `token_usage: conditional`,
`permission_denials: false`, `api_limit_detection: off`, `session_resume: true`
- Add degradation logging when unsupported features are requested
- Map permission-related behavior to Codex's approval sandbox mode

**5. Update `ralph_loop.sh`:**
- Add `--provider <name>` to CLI argument parsing (default: `claude`), following
the `SANDBOX_PROVIDER` pattern for the `AGENT_PROVIDER` variable
- Source `lib/agents/codex.sh` when `AGENT_PROVIDER=codex`
- Add dispatch wrappers `build_agent_command()` and `parse_agent_response()`
that call the provider-specific functions
- Rename `execute_claude_code()` to `execute_agent()` with provider-aware
invocation
- Allow `.ralphrc` to set `AGENT_PROVIDER`
- Emit a clear error message listing available providers for invalid provider
values
===============================================================================

Task: 2

Create the mock Codex CLI infrastructure and all test files for the Codex
adapter. The goal is comprehensive coverage using the existing
mock-binary-on-PATH and E2E patterns.

**1. Create the mock Codex CLI binary (in `tests/helpers/` or inline in test
setup):**
- Log all invocations (argv) to `$TEST_DIR/codex_calls/` for assertion
- Support mode-based responses: `ok`, `error`, `rate-limit`, `no-session`
- Emit valid JSONL event sequences for success: `thread.started` →
`turn.started` → `item.*` → `turn.completed`, including `token_count` events
- Emit error event sequences for failure cases
- Support scripted per-call responses via numbered response files (following the
existing E2E pattern)

**2. Create `tests/unit/test_codex_adapter.bats`:**
- Use the `source` pattern to load `lib/agents/codex.sh` without running the
main loop
- Write tests for: `validate_codex_command()` (available/unavailable binary),
`build_codex_command()` (correct arg arrays for various configs), JSONL parsing
(valid streams, error events, truncated streams), capability flag reporting,
session ID extraction from `thread.started`, token usage accumulation from
`token_count`, and RALPH_STATUS extraction from the final message file

**3. Create `tests/e2e/test_codex_e2e.bats`:**
- Follow `e2e_helper.bash` patterns for project setup and teardown
- Install the mock codex binary at `$E2E_DIR/bin/codex`
- Queue responses containing RALPH_STATUS blocks with various exit signals
- Test cases: successful completion (EXIT_SIGNAL: true exits cleanly),
continuation (EXIT_SIGNAL: false loops), error handling (loop responds to error
events), session resume (resume command uses stored session ID)
- Assert call counts and argument patterns via mock logs
===============================================================================

Task: 3

Create provider documentation and update existing docs to reflect multi-provider
support. The goal is accurate, complete reference material for Codex
configuration and a clear capability comparison.

**1. Create `docs/providers/` directory and `docs/providers/CODEX.md`:**
- Include: overview of Codex CLI and its capabilities; configuration variables
(`CODEX_CMD`, `CODEX_MODEL`, `CODEX_SANDBOX`); command-line usage (`ralph
--provider codex`); session resume behavior and limitations; capability
comparison with Claude (what works, what degrades); troubleshooting common
issues; example `.ralphrc` configuration snippet

**2. Update `README`:**
- Add a provider comparison table showing Claude vs Codex capabilities
- Document the `--provider` flag in the CLI options section
- Update the "supported providers" statement to include Codex
- Add a cross-reference to `docs/providers/CODEX.md`

**3. Update configuration documentation:**
- Add a Codex configuration section to the `.ralphrc` template/example
- Document the `AGENT_PROVIDER` variable in `docs/CLI_OPTIONS.md` or equivalent
- Add Codex environment variables to any existing configuration reference
- Note which settings are provider-specific vs universal

💡 Iterate on the plan with: `@coderabbitai` <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!