Skip to content

docs: add new page for CLI Backend Protocol (--cli-backend, backends list, YAML cli_backend) #239

@MervinPraison

Description

@MervinPraison

Context

PraisonAI shipped a new pluggable CLI Backend Protocol in three PRs that are now merged into main:

PR What it shipped
#1521 Phase 1 — CliBackendProtocol, CliBackendConfig, ClaudeCodeBackend (Claude Code as a first-class Agent backend)
#1531 Phase 1b — YAML cli_backend: field + --cli-backend CLI flag + backends list subcommand + mutual exclusion with --external-agent
#1532 Test hardening — extracts _resolve_yaml_cli_backend() helper for testability; production behaviour unchanged

The feature is completely undocumented in PraisonAIDocs today. A repo-wide search for cli-backend, cli_backend, CliBackendProtocol, resolve_cli_backend, ClaudeCodeBackend, and CliBackendConfig returns zero matches across docs/, docs.json, and the features/ folder.

The existing docs/features/external-cli-integrations.mdx page documents the legacy --external-agent flag and the ClaudeCodeIntegration wrapper class — these are different APIs. The new CLI Backend Protocol is an additional, orthogonal feature with its own user-facing surface.

This issue is for the second-pass agent to create a new documentation page following the standards in AGENTS.md.


Decision: New content, not an update

Question Answer
Does any existing page cover this? No — the closest page (external-cli-integrations.mdx) covers a different API surface (--external-agent, ClaudeCodeIntegration)
Is this user-facing? Yes — three new user surfaces: --cli-backend flag, backends list subcommand, YAML cli_backend: field
Does it belong in docs/concepts/? No — per AGENTS.md §1.8, docs/concepts/ is human-only. Use docs/features/
Does it belong in docs/cli/? The CLI flag and backends list subcommand parts could be cross-referenced from a new docs/cli/cli-backend.mdx, but the primary home is docs/features/cli-backend-protocol.mdx because it is a feature, not just a CLI command

Recommendation: Create docs/features/cli-backend-protocol.mdx and add a sidebar entry under the Features group in docs.json. Optionally add a short cross-reference in docs/features/external-cli-integrations.mdx so users on the legacy page learn the new pluggable approach exists.


SDK ground truth (read these files first)

Per AGENTS.md §1.2 (SDK-First Documentation Cycle), the second-pass agent must read each of these files in full before writing a single line of documentation:

Core protocol (lightweight, no implementations)

  • src/praisonai-agents/praisonaiagents/cli_backend/protocols.py
    • CliBackendConfig dataclass — declarative backend configuration
    • CliSessionBinding dataclass — session state tracking
    • CliBackendResult dataclass — execute() return type
    • CliBackendDelta dataclass — stream() yield type
    • CliBackendProtocol runtime-checkable Protocol — execute(...) and stream(...) async methods

Wrapper implementation

  • src/praisonai/praisonai/cli_backends/__init__.py — lazy public API (register_cli_backend, resolve_cli_backend, list_cli_backends, ClaudeCodeBackend)
  • src/praisonai/praisonai/cli_backends/registry.py — thread-safe registry, register_cli_backend(), resolve_cli_backend(backend_id, overrides=None), list_cli_backends(), auto-registers claude-code on import
  • src/praisonai/praisonai/cli_backends/claude.pyClaudeCodeBackend class with DEFAULT_CONFIG (subprocess-based execution against the claude CLI, JSONL streaming, env sanitization, model aliases, session/resume support)

CLI integration

  • src/praisonai/praisonai/cli/main.py--cli-backend flag is added with choices from list_cli_backends(); backends / backends list subcommand; mutual-exclusion with --external-agent
    • Look for the --cli-backend action and the backends subparser
    • Look for the PYTEST_CURRENT_TEST short-circuit in parse_args (informative only — do not document it)

YAML integration

  • src/praisonai/praisonai/agents_generator.py — module-level _resolve_yaml_cli_backend(cli_backend_config, logger) (lines ~143–185) — extracted in PR #1532
    • Accepts None, a string id, or a dict {id: ..., overrides: {...}}
    • Returns None (with a logged warning) on any error — YAML parsing never raises

Example YAML

  • examples/yaml/cli_backend.yaml — minimal worked example shipping in the repo

Tests (read for behavioural truth, not documentation content)

  • src/praisonai/tests/unit/test_yaml_cli_backend.py — covers all YAML shapes (string, dict, missing id, unknown id, invalid type, registered id end-to-end)
  • src/praisonai/tests/unit/test_cli_backend_flag.py — covers --cli-backend in --help, accepted backend, unknown backend rejected, mutex with --external-agent, backends list, bare backends, unknown subcommand
  • src/praisonai/tests/unit/test_external_agents_backcompat.py — proves the legacy --external-agent flag and ClaudeCodeIntegration class are unchanged
  • src/praisonai-agents/tests/unit/test_cli_backend_protocol.py — protocol-level tests

What the new page must cover

1. Frontmatter (per AGENTS.md §8)

---
title: "CLI Backend Protocol"
sidebarTitle: "CLI Backends"
description: "Plug an external CLI (Claude Code, future: Codex, Gemini) in as a first-class Agent backend"
icon: "plug"
---

2. One-line opener + hero diagram

One sentence: e.g. "CLI Backends let you run an agent's turn through an external CLI tool (like Claude Code) instead of a Python LLM client, while keeping the same Agent/Task API."

Hero graph LR Mermaid diagram with the standard color scheme from AGENTS.md §3.1:

  • 📋 Input (Agent/User) — #6366F1
  • 🧠 Resolver (resolve_cli_backend) — #F59E0B
  • 🔌 Backend (ClaudeCodeBackend) — #189AB4
  • 💻 CLI subprocess — #189AB4
  • ✅ Result — #10B981

3. Quick Start (Steps component) — agent-first per AGENTS.md §1.1 rule 9

Three steps minimum, all agent-centric — show the user-facing way first, never the bare backend instance:

Step 1 — Simplest: CLI flag

praisonai "Refactor utils.py" --cli-backend claude-code

Step 2 — Declarative: YAML

framework: praisonai
topic: coding
roles:
  coder:
    role: Code refactorer
    goal: Refactor Python modules
    backstory: Senior engineer
    cli_backend: claude-code   # string form
    tasks:
      refactor:
        description: Refactor utils.py
        expected_output: Refactored code

Step 3 — YAML with overrides (dict form)

roles:
  coder:
    # ...
    cli_backend:
      id: claude-code
      overrides:
        timeout_ms: 60000

Step 4 — Discover what's registered

praisonai backends list
# claude-code

4. How It Works (sequence diagram + brief table)

sequenceDiagram showing: User → CLI/YAML → _resolve_yaml_cli_backend (or --cli-backend argparse) → resolve_cli_backend(id, overrides=...) → factory → ClaudeCodeBackend instance → Agent uses it for the turn → subprocess to claude CLI → result back to Agent → response to user.

5. The three configuration surfaces (decision diagram)

Per AGENTS.md §6.1 ("If multiple options in one page, people might be confused… create the mermaid diagram to choose what option at what instance"):

graph TB
    Q{How do I want to configure a CLI backend?}
    Q -->|Quickest, ad-hoc command| A[--cli-backend on the CLI]
    Q -->|Versioned, per-role config| B[cli_backend in YAML - string form]
    Q -->|Need to tweak timeouts/args| C[cli_backend in YAML - dict with overrides]
    Q -->|Building a custom backend| D[register_cli_backend in Python]

6. The cli_backend YAML field reference

Document all three accepted shapes from _resolve_yaml_cli_backend():

Shape Example Behaviour
Omitted (field absent) No backend used; Agent uses normal LLM client. No warning logged.
String cli_backend: claude-code Resolves via resolve_cli_backend("claude-code")
Dict cli_backend: {id: claude-code, overrides: {timeout_ms: 60000}} Resolves via resolve_cli_backend("claude-code", overrides={...})
Dict missing id cli_backend: {overrides: {...}} Returns None + logged warning. Does not raise.
Unknown id cli_backend: nope Returns None + logged warning. Does not raise.
Invalid type (e.g. 123) cli_backend: 123 Returns None + logged warning. Does not raise.

7. Built-in backend: claude-code (config reference)

Pull the complete DEFAULT_CONFIG from src/praisonai/praisonai/cli_backends/claude.py lines 24–73 into a config table per AGENTS.md §7.2. Every field of CliBackendConfig that is set in DEFAULT_CONFIG must appear, with its default value as set by Claude's defaults (not the dataclass defaults).

Key things to highlight in plain language:

  • command: "claude" — the CLI it shells out to (must be on PATH)
  • args — the default flags passed every call
  • model_aliasesopus / sonnet / haiku shortcuts
  • clear_env — the 15 env vars sanitized before subprocess (security-relevant — list them all)
  • timeout_ms: 300_000 — 5 minute default
  • bundle_mcp: True + bundle_mcp_mode: "claude-config-file"
  • serialize: True — operations are queued

8. The --cli-backend CLI flag

Flag Type Behaviour
--cli-backend BACKEND_ID string Choices populated dynamically from list_cli_backends(). Currently: claude-code.
--cli-backend X --external-agent Y Mutually exclusive — argparse rejects with "not allowed with argument"
Unknown id (e.g. --cli-backend bogus) Rejected by argparse with "invalid choice"

9. The backends subcommand

Command Behaviour
praisonai backends list Prints each registered backend id on its own line
praisonai backends Same as list (list is the default)
praisonai backends bogus Prints [red]Unknown backends subcommand: bogus[/red] and the list of valid subcommands

10. Custom backends (advanced — registering your own)

Show a minimal example:

from praisonai.cli_backends import register_cli_backend
from praisonaiagents import CliBackendConfig

def my_backend_factory():
    from my_pkg import MyBackend
    return MyBackend(config=CliBackendConfig(command="my-cli"))

register_cli_backend("my-backend", my_backend_factory)

After registering, praisonai backends list shows it, --cli-backend my-backend accepts it, and cli_backend: my-backend works in YAML.

11. CliBackendProtocol reference (for backend authors)

Brief — the Agent-facing user does not need this. Backend authors do:

  • config: CliBackendConfig attribute
  • async def execute(prompt, *, session=None, images=None, system_prompt=None, **kwargs) -> CliBackendResult
  • async def stream(prompt, **kwargs) -> AsyncIterator[CliBackendDelta]

Link to the relevant SDK reference card per the page template in AGENTS.md §2.

12. Best Practices (<AccordionGroup>)

3–4 accordions, e.g.:

  • Prefer the YAML field for production, the --cli-backend flag for one-off commands
  • Set overrides.timeout_ms for slow CLIs rather than monkey-patching
  • Don't combine --cli-backend with --external-agent — argparse rejects it; pick one model
  • The claude CLI must be on PATH (or installed via the Claude Code SDK) — claude-code is the only built-in backend today

13. How this differs from --external-agent (call-out + cross-link)

A short <Note> block:

The legacy --external-agent claude and ClaudeCodeIntegration class still work and are unchanged (see External CLI Integrations). The CLI Backend Protocol is the new pluggable path: backends are registered by id, configured declaratively, and surfaced as a YAML field and --cli-backend flag.

14. Related (<CardGroup cols={2}>)

  • /docs/features/external-cli-integrations — the legacy class-based path
  • A future card for the SDK reference page once it exists

Style requirements (per AGENTS.md)

  • Non-developer tone — §6.1 ("After reading the code they should feel 'is it this easy to use?'")
  • Agent-centric examples first — §1.1 rule 9 ("Top of the document should always start with Agent Centric code example")
  • Use simple imports — §6.1 closing line. e.g. from praisonai.cli_backends import register_cli_backend, never deep paths
  • Mermaid diagrams for each concept on the page — §6.1 ("If there are different concepts in one page, then explain each concept using mermaid diagram")
  • No forbidden phrases — §6.3 (no "In this section, we will…", "As you can see…", etc.)
  • Every code block must run unmodified — §5.1
  • All CliBackendConfig defaults must match DEFAULT_CONFIG in src/praisonai/praisonai/cli_backends/claude.py exactly — §1.3 (SDK Source Verification)

File placement (per AGENTS.md §1.8)

Action Path
Create new page docs/features/cli-backend-protocol.mdx
Add sidebar entry docs.json under the Features group (NEVER under Concepts)
Optional cross-link Append a <Note> to docs/features/external-cli-integrations.mdx pointing at the new page

Do NOT create or modify anything in docs/concepts/ — that folder is human-only.


Acceptance checklist (the agent that creates the docs should tick these)

  • All four files in SDK ground truth above were read in full before any documentation was written
  • Every CliBackendConfig field documented matches DEFAULT_CONFIG in claude.py exactly
  • All three YAML shapes (string / dict / missing) are documented with the exact behaviour from _resolve_yaml_cli_backend
  • The --cli-backend flag and backends list subcommand are both documented
  • Mutual exclusion with --external-agent is documented
  • Hero Mermaid diagram present, using the AGENTS.md §3.1 color scheme
  • At least one decision diagram (which configuration surface to use)
  • At least one sequence diagram (resolution flow)
  • All Quick Start examples are agent-centric (CLI / YAML), not bare backend instantiation
  • Steps, AccordionGroup, CardGroup Mintlify components used per AGENTS.md §4.1
  • docs.json updated, valid JSON, page placed under Features group
  • No edits to docs/concepts/
  • Branch: claude/admiring-euler-Fibw0

Reference: triggering PR

This issue was generated automatically after PR MervinPraison/PraisonAI#1532 merged to main. PR #1532 is a test-fix PR (production behaviour unchanged), but it confirms that the CLI Backend Protocol surface from PRs #1521 and #1531 is now stable, tested, and ready for documentation.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions