Commit c303c62
fix: propagate Gemini grounding metadata from ModelResponse
Merge #5661
### Link to Issue
- Closes: #5659
### Problem
LiteLLM exposes Gemini's grounding metadata on `ModelResponse.vertex_ai_grounding_metadata` rather than inside the message, and `_model_response_to_generate_content_response` in `lite_llm.py` never reads it. As a result `LlmResponse.grounding_metadata` is always `None` when Gemini is called through `LiteLlm`, breaking `event.grounding_metadata`, `after_model_callback`, and citation pipelines for the entire LiteLlm path. The native `Gemini()` path picks grounding up from `candidate.grounding_metadata` (`llm_response.py:190`).
### Solution
Pull `vertex_ai_grounding_metadata` off the `ModelResponse` after the message is converted, and attach it to `LlmResponse.grounding_metadata`. Three shapes are accepted:
- `types.GroundingMetadata` instance → used as-is.
- `dict` → validated via `types.GroundingMetadata.model_validate`.
- `list` (one entry per candidate) → first entry, then the same dispatch.
If validation fails the handler logs a warning and leaves grounding unset, so a single malformed payload doesn't break the rest of the response.
### Testing Plan
**Unit Tests** in `tests/unittests/models/test_litellm.py`:
- `test_model_response_to_generate_content_response_grounding_metadata_dict` — dict payload is propagated.
- `test_model_response_to_generate_content_response_grounding_metadata_list` — list payload uses the first entry.
- `test_model_response_to_generate_content_response_no_grounding_metadata` — missing attribute leaves grounding as `None` (no regression for non-Gemini LiteLlm models).
Full `test_litellm.py` suite passes locally:
```
259 passed in 9.36s
```
**Manual E2E**: Live Gemini API verification was not possible locally (no API key configured). The unit tests cover the conversion function the bug report points to.
### Checklist
- [x] I have read the CONTRIBUTING.md document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective.
- [x] New and existing unit tests pass locally with my changes.
- [ ] I have manually tested my changes end-to-end. (See above — requires live Gemini access.)
Co-authored-by: George Weale <gweale@google.com>
COPYBARA_INTEGRATE_REVIEW=#5661 from 1wos:fix-litellm-grounding 4657c9d
PiperOrigin-RevId: 9340100431 parent ea772b9 commit c303c62
2 files changed
Lines changed: 161 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1741 | 1741 | | |
1742 | 1742 | | |
1743 | 1743 | | |
| 1744 | + | |
| 1745 | + | |
| 1746 | + | |
| 1747 | + | |
| 1748 | + | |
| 1749 | + | |
| 1750 | + | |
| 1751 | + | |
| 1752 | + | |
| 1753 | + | |
| 1754 | + | |
| 1755 | + | |
| 1756 | + | |
| 1757 | + | |
| 1758 | + | |
| 1759 | + | |
| 1760 | + | |
| 1761 | + | |
| 1762 | + | |
| 1763 | + | |
| 1764 | + | |
| 1765 | + | |
| 1766 | + | |
| 1767 | + | |
| 1768 | + | |
| 1769 | + | |
| 1770 | + | |
| 1771 | + | |
| 1772 | + | |
| 1773 | + | |
| 1774 | + | |
| 1775 | + | |
1744 | 1776 | | |
1745 | 1777 | | |
1746 | 1778 | | |
| |||
1792 | 1824 | | |
1793 | 1825 | | |
1794 | 1826 | | |
| 1827 | + | |
| 1828 | + | |
| 1829 | + | |
| 1830 | + | |
| 1831 | + | |
1795 | 1832 | | |
1796 | 1833 | | |
1797 | 1834 | | |
| |||
2374 | 2411 | | |
2375 | 2412 | | |
2376 | 2413 | | |
| 2414 | + | |
2377 | 2415 | | |
2378 | 2416 | | |
2379 | 2417 | | |
| |||
2459 | 2497 | | |
2460 | 2498 | | |
2461 | 2499 | | |
| 2500 | + | |
| 2501 | + | |
| 2502 | + | |
| 2503 | + | |
| 2504 | + | |
2462 | 2505 | | |
2463 | 2506 | | |
2464 | 2507 | | |
| |||
2560 | 2603 | | |
2561 | 2604 | | |
2562 | 2605 | | |
| 2606 | + | |
| 2607 | + | |
2563 | 2608 | | |
2564 | 2609 | | |
2565 | 2610 | | |
2566 | 2611 | | |
2567 | 2612 | | |
| 2613 | + | |
| 2614 | + | |
| 2615 | + | |
| 2616 | + | |
2568 | 2617 | | |
2569 | 2618 | | |
2570 | 2619 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2473 | 2473 | | |
2474 | 2474 | | |
2475 | 2475 | | |
| 2476 | + | |
| 2477 | + | |
| 2478 | + | |
| 2479 | + | |
| 2480 | + | |
| 2481 | + | |
| 2482 | + | |
| 2483 | + | |
| 2484 | + | |
| 2485 | + | |
| 2486 | + | |
| 2487 | + | |
| 2488 | + | |
| 2489 | + | |
| 2490 | + | |
| 2491 | + | |
| 2492 | + | |
| 2493 | + | |
| 2494 | + | |
| 2495 | + | |
| 2496 | + | |
| 2497 | + | |
| 2498 | + | |
| 2499 | + | |
| 2500 | + | |
| 2501 | + | |
| 2502 | + | |
| 2503 | + | |
| 2504 | + | |
| 2505 | + | |
| 2506 | + | |
| 2507 | + | |
| 2508 | + | |
| 2509 | + | |
| 2510 | + | |
| 2511 | + | |
| 2512 | + | |
| 2513 | + | |
| 2514 | + | |
| 2515 | + | |
| 2516 | + | |
| 2517 | + | |
| 2518 | + | |
| 2519 | + | |
| 2520 | + | |
| 2521 | + | |
| 2522 | + | |
| 2523 | + | |
| 2524 | + | |
| 2525 | + | |
| 2526 | + | |
| 2527 | + | |
| 2528 | + | |
| 2529 | + | |
| 2530 | + | |
| 2531 | + | |
| 2532 | + | |
| 2533 | + | |
| 2534 | + | |
| 2535 | + | |
| 2536 | + | |
| 2537 | + | |
2476 | 2538 | | |
2477 | 2539 | | |
2478 | 2540 | | |
| |||
3775 | 3837 | | |
3776 | 3838 | | |
3777 | 3839 | | |
3778 | | - | |
| 3840 | + | |
| 3841 | + | |
| 3842 | + | |
| 3843 | + | |
| 3844 | + | |
| 3845 | + | |
| 3846 | + | |
| 3847 | + | |
| 3848 | + | |
| 3849 | + | |
| 3850 | + | |
| 3851 | + | |
| 3852 | + | |
| 3853 | + | |
| 3854 | + | |
| 3855 | + | |
| 3856 | + | |
| 3857 | + | |
| 3858 | + | |
| 3859 | + | |
| 3860 | + | |
| 3861 | + | |
| 3862 | + | |
| 3863 | + | |
| 3864 | + | |
| 3865 | + | |
| 3866 | + | |
| 3867 | + | |
| 3868 | + | |
| 3869 | + | |
| 3870 | + | |
| 3871 | + | |
| 3872 | + | |
| 3873 | + | |
| 3874 | + | |
| 3875 | + | |
| 3876 | + | |
| 3877 | + | |
| 3878 | + | |
| 3879 | + | |
| 3880 | + | |
| 3881 | + | |
| 3882 | + | |
| 3883 | + | |
| 3884 | + | |
| 3885 | + | |
| 3886 | + | |
| 3887 | + | |
| 3888 | + | |
| 3889 | + | |
3779 | 3890 | | |
3780 | 3891 | | |
3781 | 3892 | | |
| |||
0 commit comments