Fix #5886: Strip cache_breakpoint from LiteLLM path for non-Anthropic providers

[devin-ai-integration]

Summary

Fixes #5886 — cache_breakpoint was leaking into the wire payload for non-Anthropic providers (Groq, OpenAI-compatible, etc.) causing API errors like:

GroqException - 'messages.0': property 'cache_breakpoint' is unsupported

Root cause: The LiteLLM-based LLM.call() code path (used by Groq and other non-native providers) does not go through BaseLLM._format_messages() which strips cache_breakpoint. Instead it uses _format_messages_for_provider() → _prepare_completion_params(), neither of which stripped the flag.

Native providers (OpenAI, Anthropic, Gemini, Azure, Bedrock) all call BaseLLM._format_messages() and are unaffected.

Fix: Strip cache_breakpoint in LLM._prepare_completion_params() before the messages are placed into the API request params. This creates copies (not in-place mutation) so the executor's reusable messages list retains markers across ReAct loop iterations.

Review & Testing Checklist for Human

• Verify the fix works end-to-end with a Groq model: LLM(model="groq/llama-3.3-70b-versatile") with a basic crew
• Confirm Anthropic prompt caching still works correctly (the Anthropic provider reads flags before super()._format_messages() strips them, so this change doesn't affect that path)
• Check that the 3 new tests in tests/llms/test_prompt_cache.py::TestLiteLLMPathStripsMarker pass

Notes

• The fix is in lib/crewai/src/crewai/llm.py — the _prepare_completion_params method
• 3 new regression tests added in lib/crewai/tests/llms/test_prompt_cache.py
• All 12 prompt cache tests pass locally

Link to Devin session: https://app.devin.ai/sessions/86df9bef10ca4e7ebfbf3c97c946f2fd

Summary by CodeRabbit

Release Notes

• Bug Fixes

  • Fixed cache breakpoint handling for non-Anthropic LLM providers by removing unsupported cache-breakpoint fields before sending requests.

• Tests

  • Added regression tests to verify cache breakpoint removal and ensure original message data remains unchanged during processing.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 726e04e6-1543-4f20-8854-9625d0601c37

📥 Commits

Reviewing files that changed from the base of the PR and between 81c21e3 and d00f7a7.

📒 Files selected for processing (2)

• lib/crewai/src/crewai/llm.py
• lib/crewai/tests/llms/test_prompt_cache.py

---

📝 Walkthrough

Walkthrough

LLM._prepare_completion_params now strips the CACHE_BREAKPOINT_KEY marker from formatted messages before sending to LiteLLM, preventing API rejections from non-Anthropic providers that don't support cache_breakpoint fields. Regression tests verify marker removal, caller-side immutability, and correct message pass-through.

Changes

Cache breakpoint marker handling for LiteLLM

Layer / File(s)	Summary
Cache breakpoint marker stripping in LiteLLM path lib/crewai/src/crewai/llm.py	Adds cleanup logic to LLM._prepare_completion_params that strips CACHE_BREAKPOINT_KEY from each formatted message's dictionary keys before building completion call parameters, preventing non-Anthropic providers from receiving unsupported cache-breakpoint fields.
Regression tests for marker stripping lib/crewai/tests/llms/test_prompt_cache.py	Introduces TestLiteLLMPathStripsMarker with three test methods: verifying cache_breakpoint is stripped from the params payload, confirming the caller's original message objects remain unmodified, and ensuring messages without markers pass through unchanged.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• crewAIInc/crewAI#5774: Both PRs modify the CACHE_BREAKPOINT_KEY cache-breakpoint marker handling by ensuring it's stripped/translated before provider requests (main PR does it in LLM._prepare_completion_params for LiteLLM; retrieved PR introduces the marker and broader stripping/stamping logic).

Suggested labels

size/L

Poem

🐰 A breakpoint marker caused quite the plight,
Non-Anthropic providers rejected it on sight,
But now with a strip in the LiteLLM way,
The messages flow cleanly, hip-hip hooray! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically summarizes the main change: stripping cache_breakpoint from the LiteLLM path for non-Anthropic providers, which directly addresses the bug fix.
Linked Issues check	✅ Passed	The PR successfully addresses all coding requirements from issue #5886 by stripping cache_breakpoint in LiteLLM path and preventing message mutation through copying, with comprehensive regression tests.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to fixing issue #5886: modifications to _prepare_completion_params and addition of regression tests for the LiteLLM cache_breakpoint stripping behavior.
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch devin/1779357962-fix-cache-breakpoint-litellm

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}