|
| 1 | +# Investigation Index |
| 2 | + |
| 3 | +## Quick Navigation |
| 4 | + |
| 5 | +### 🚀 Start Here |
| 6 | +- **[INVESTIGATION_README.md](./INVESTIGATION_README.md)** - Overview of all documents |
| 7 | + |
| 8 | +### 📋 Executive Summary |
| 9 | +- **[SUMMARY_FOR_TEAM.md](./SUMMARY_FOR_TEAM.md)** - Quick TL;DR for team (5 min read) |
| 10 | + |
| 11 | +### 🔍 Full Investigation |
| 12 | +- **[INVESTIGATION_OPUS_4_7_HAIKU_REGRESSION.md](./INVESTIGATION_OPUS_4_7_HAIKU_REGRESSION.md)** - Complete analysis (15 min read) |
| 13 | + |
| 14 | +### 💻 Technical Deep-Dive |
| 15 | +- **[CODE_PATTERNS_ANALYSIS.md](./CODE_PATTERNS_ANALYSIS.md)** - Code patterns & fixes (20 min read) |
| 16 | + |
| 17 | +### 📊 Visual Explanation |
| 18 | +- **[VISUAL_TIMELINE.md](./VISUAL_TIMELINE.md)** - Diagrams & timeline (10 min read) |
| 19 | + |
| 20 | +--- |
| 21 | + |
| 22 | +## Reading Paths |
| 23 | + |
| 24 | +### Path 1: Quick Brief (10 minutes) |
| 25 | +1. [INVESTIGATION_README.md](./INVESTIGATION_README.md) - Overview |
| 26 | +2. [SUMMARY_FOR_TEAM.md](./SUMMARY_FOR_TEAM.md) - Action items |
| 27 | + |
| 28 | +**Result:** Understand what happened and what needs to be done. |
| 29 | + |
| 30 | +### Path 2: Implementation (30 minutes) |
| 31 | +1. [SUMMARY_FOR_TEAM.md](./SUMMARY_FOR_TEAM.md) - Context |
| 32 | +2. [CODE_PATTERNS_ANALYSIS.md](./CODE_PATTERNS_ANALYSIS.md) - Specific fixes |
| 33 | +3. [VISUAL_TIMELINE.md](./VISUAL_TIMELINE.md) - Before/after comparison |
| 34 | + |
| 35 | +**Result:** Ready to implement the hotfix. |
| 36 | + |
| 37 | +### Path 3: Complete Understanding (60 minutes) |
| 38 | +1. [INVESTIGATION_README.md](./INVESTIGATION_README.md) - Start here |
| 39 | +2. [VISUAL_TIMELINE.md](./VISUAL_TIMELINE.md) - See the timeline |
| 40 | +3. [INVESTIGATION_OPUS_4_7_HAIKU_REGRESSION.md](./INVESTIGATION_OPUS_4_7_HAIKU_REGRESSION.md) - Full details |
| 41 | +4. [CODE_PATTERNS_ANALYSIS.md](./CODE_PATTERNS_ANALYSIS.md) - Technical depth |
| 42 | +5. [SUMMARY_FOR_TEAM.md](./SUMMARY_FOR_TEAM.md) - Action plan |
| 43 | + |
| 44 | +**Result:** Complete context for planning long-term fix. |
| 45 | + |
| 46 | +--- |
| 47 | + |
| 48 | +## Documents by Audience |
| 49 | + |
| 50 | +### For Engineering Managers |
| 51 | +- [SUMMARY_FOR_TEAM.md](./SUMMARY_FOR_TEAM.md) - What happened, impact, action items |
| 52 | +- [VISUAL_TIMELINE.md](./VISUAL_TIMELINE.md) - Timeline and cost impact |
| 53 | + |
| 54 | +### For Developers Implementing Fix |
| 55 | +- [CODE_PATTERNS_ANALYSIS.md](./CODE_PATTERNS_ANALYSIS.md) - Exact code changes needed |
| 56 | +- [SUMMARY_FOR_TEAM.md](./SUMMARY_FOR_TEAM.md) - Files to modify |
| 57 | + |
| 58 | +### For Architects Planning Long-term Fix |
| 59 | +- [INVESTIGATION_OPUS_4_7_HAIKU_REGRESSION.md](./INVESTIGATION_OPUS_4_7_HAIKU_REGRESSION.md) - Root cause analysis |
| 60 | +- [CODE_PATTERNS_ANALYSIS.md](./CODE_PATTERNS_ANALYSIS.md) - Architectural fix strategy |
| 61 | + |
| 62 | +### For Customer Success / Support |
| 63 | +- [VISUAL_TIMELINE.md](./VISUAL_TIMELINE.md) - What customers experienced |
| 64 | +- [SUMMARY_FOR_TEAM.md](./SUMMARY_FOR_TEAM.md) - Timeline of events |
| 65 | + |
| 66 | +--- |
| 67 | + |
| 68 | +## Key Findings Summary |
| 69 | + |
| 70 | +### The Issue |
| 71 | +Upgrading from `v1.82.3-stable.patch.4` to `v1.82.3-stable.opus-4.7` caused cost spikes on Haiku 4.5 Bedrock due to "unsupported thinking metadata" errors. |
| 72 | + |
| 73 | +### Root Cause |
| 74 | +Broad pattern matching (`"claude-haiku-4" in model`) added for Opus 4.7 support inadvertently matched Haiku 4.5, causing thinking metadata to be sent to models that don't support it. |
| 75 | + |
| 76 | +### Immediate Fix |
| 77 | +Explicitly exclude Haiku 4.5 before checking broader Claude 4.x patterns. |
| 78 | + |
| 79 | +### Long-term Fix |
| 80 | +Fix `supports_reasoning()` provider name mismatch and remove all hardcoded model version checks. |
| 81 | + |
| 82 | +--- |
| 83 | + |
| 84 | +## Files to Review in LiteLLM Repo |
| 85 | + |
| 86 | +Priority order: |
| 87 | + |
| 88 | +1. **`litellm/llms/bedrock/common_utils.py`** |
| 89 | + - Function: `is_claude_4_5_on_bedrock()` |
| 90 | + - Lines: 565-597 |
| 91 | + |
| 92 | +2. **`litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py`** |
| 93 | + - Function: `_supports_extended_thinking_on_bedrock()` |
| 94 | + |
| 95 | +3. **`litellm/llms/anthropic/chat/transformation.py`** |
| 96 | + - Functions: `_is_opus_4_7_model()`, thinking logic |
| 97 | + - Lines: 172-210, 237-248 |
| 98 | + |
| 99 | +4. **`litellm/llms/bedrock/chat/converse_transformation.py`** |
| 100 | + - Function: `get_supported_openai_params()` |
| 101 | + - Lines: 565-580 |
| 102 | + |
| 103 | +--- |
| 104 | + |
| 105 | +## Related GitHub Resources |
| 106 | + |
| 107 | +### PRs |
| 108 | +- [#25867](https://github.com/BerriAI/litellm/pull/25867) - Day 0 Opus 4.7 support (merged) |
| 109 | +- [#25876](https://github.com/BerriAI/litellm/pull/25876) - Opus 4.7 hotfix (merged April 16) |
| 110 | +- [#24053](https://github.com/BerriAI/litellm/pull/24053) - Claude 4+ thinking support (open) |
| 111 | + |
| 112 | +### Discussions |
| 113 | +- [#22555](https://github.com/BerriAI/litellm/discussions/22555) - Day 0 Model Release Gaps |
| 114 | + |
| 115 | +### Issues |
| 116 | +Referenced in investigation but no specific issue filed yet for this regression. |
| 117 | + |
| 118 | +--- |
| 119 | + |
| 120 | +## Timeline |
| 121 | + |
| 122 | +- **April 16, 2026 5:19pm** - PR #25867 merged |
| 123 | +- **April 16, 2026 7:19pm** - PR #25876 merged, v1.82.3-stable.opus-4.7 released |
| 124 | +- **April 17, 2026 7:32pm** - User reports cost spike on Slack |
| 125 | +- **April 17, 2026 7:47pm** - Investigation requested |
| 126 | +- **April 17, 2026 [Now]** - Investigation complete |
| 127 | + |
| 128 | +--- |
| 129 | + |
| 130 | +## Next Steps |
| 131 | + |
| 132 | +- [ ] LiteLLM team validates findings |
| 133 | +- [ ] Deploy immediate hotfix |
| 134 | +- [ ] Notify affected users |
| 135 | +- [ ] Plan architectural fix |
| 136 | +- [ ] Update CI/CD requirements |
| 137 | +- [ ] Document lessons learned |
| 138 | + |
| 139 | +--- |
| 140 | + |
| 141 | +**Investigation Status:** ✅ Complete |
| 142 | +**Branch:** `cursor/investigate-opus-4-7-haiku-regression-5249` |
| 143 | +**Created by:** Cursor Cloud Agent |
| 144 | +**Date:** April 17, 2026 |
0 commit comments