fix(captainagent): drop top_p from default LLM config to avoid temperature/top_p conflict#2881
fix(captainagent): drop top_p from default LLM config to avoid temperature/top_p conflict#2881obchain wants to merge 2 commits into
Conversation
…mperature/top_p conflict
``CaptainAgent.DEFAULT_NESTED_CONFIG`` hard-coded
``default_llm_config = {"temperature": 1, "top_p": 0.95, "max_tokens": 2048}``.
Many providers (notably Anthropic) reject requests that set both
``temperature`` and ``top_p`` simultaneously, which surfaced as
ValidationError: temperature and top_p cannot be set at the same time
on the very first invocation, even when the user had not configured either
parameter on their own ``llm_config``. The defaults made the demo
unusable on those providers.
OpenAI guidance is to set only one of the two sampling parameters; keep
``temperature`` (it is the more widely supported default) and drop
``top_p`` from the built-in config. Users that want ``top_p`` can still
opt in by overriding ``default_llm_config`` in their ``nested_config``.
Adds a regression test that pins this invariant on
``DEFAULT_NESTED_CONFIG``.
Fixes ag2ai#2102
Codecov Report✅ All modified and coverable lines are covered by tests. 🚀 New features to boost your workflow:
|
|
Thanks @obchain for digging into this and putting together a well-tested change. Things have shifted since #2102 was originally filed: the error in that issue ( So, we won't merge this specific fix, but thanks again for taking the time to work on it. |
2 participants
Summary
CaptainAgent.DEFAULT_NESTED_CONFIGset bothtemperatureandtop_pin the built-indefault_llm_config. Providers like Anthropic reject requests that set both, which broke out-of-the-boxCaptainAgentruns the moment the agent tried to spin up its expert group — even when the user had not configured either parameter.top_pfrom the default. Users that want it can still set it explicitly vianested_config.Fixes #2102
Test plan
uv run --extra test pytest test/agentchat/contrib/test_captainagent.py::test_default_nested_config_does_not_set_temperature_and_top_p_together— passed