Skip to content

Guard streaming prompts against context growth#36

Open
xiaowangzhixiao wants to merge 1 commit into
jd-opensource:mainfrom
xiaowangzhixiao:codex/context-budget-guard
Open

Guard streaming prompts against context growth#36
xiaowangzhixiao wants to merge 1 commit into
jd-opensource:mainfrom
xiaowangzhixiao:codex/context-budget-guard

Conversation

@xiaowangzhixiao

Copy link
Copy Markdown

Summary

  • add opt-in recent-turn and recent-image windows for main-model prompts
  • preserve full chunk/session state for summarization and long-term memory
  • roll back the current turn when prompt construction or the main-model request fails
  • expose the new settings through CLI arguments, environment variables, and start_adapter.sh
  • document the behavior in English and Chinese and add regression tests

Both limits default to 0 (unlimited), so existing deployments keep their current prompt behavior.

Fixes #35.

Validation

  • python -m unittest discover -s services/webinfer/tests -v (5 tests passed)
  • python -m py_compile services/webinfer/live_adapter.py services/webinfer/tests/test_live_adapter.py
  • bash -n services/webinfer/scripts/start_adapter.sh
  • git diff --check
  • local end-to-end adapter run with 9 sequential image turns against an OpenAI-compatible vLLM backend; the final queried turn completed successfully
  • forced backend failure followed by a retry stayed at the same turn/frame count, confirming rollback

@xiaowangzhixiao
xiaowangzhixiao marked this pull request as ready for review July 22, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming adapter can accumulate failed frames past context and image limits

1 participant