Skip to content

[issue-5931] [FE/P SDK] fix: render Google ADK attachments in prompt parts#5967

Draft
ollieagent[bot] wants to merge 3 commits into
mainfrom
ollie/issue-5931-render-attachments-correctly-879056
Draft

[issue-5931] [FE/P SDK] fix: render Google ADK attachments in prompt parts#5967
ollieagent[bot] wants to merge 3 commits into
mainfrom
ollie/issue-5931-render-attachments-correctly-879056

Conversation

@ollieagent
Copy link
Copy Markdown
Contributor

@ollieagent ollieagent Bot commented Mar 30, 2026

Details

Fixes two layered bugs that prevented Google ADK image attachments from being extracted and rendered correctly.

Python SDK — URL-safe base64 not matched by regex

When a genai_types.Part contains inline_data (a Blob with raw bytes), pydantic's model_dump(mode='json') serializes the bytes as URL-safe base64 (using - and _ instead of + and /). The AttachmentsExtractor regex only matched standard base64 characters, causing it to match only a partial substring — leaving a dangling suffix after the placeholder and producing a broken attachment.

  • attachments_extractor.py: extended the base64 character class to include - and _
  • decoder_base64.py: added a fallback to base64.urlsafe_b64decode when standard decoding fails, so already-URL-safe strings are decoded correctly

Frontend — Google format handler was null

The google entry in the PrettyLLMMessage format registry was registered as null, so Google ADK traces rendered as raw JSON instead of a structured chat view.

  • Added detector.ts: detects Google ADK contents[].parts[] input format and candidates[] output format
  • Added mapper.ts: maps parts[] to block descriptors — inline_data → image blocks, text → text blocks, function_call/function_response → code blocks; maps model role → assistant, function role → tool
  • Added index.ts: exports the googleFormat implementation
  • Registered googleFormat in registry.ts
  • PromptMessagesReadonly.tsx: added inline_data handling so the existing prompt message readonly view also renders Google-format images

Change checklist

  • User facing
  • Documentation update

Issues

Testing

  1. Create an ADK trace with an image attachment using the Python SDK:
    import base64, opik
    from opik.api_objects.attachment import AttachmentType
    
    client = opik.Opik()
    png_bytes = b'\x89PNG\r\n...'  # any PNG bytes
    b64 = base64.urlsafe_b64encode(png_bytes).decode()
    trace = client.trace(
        name="adk-test",
        input={"contents": [{"role": "user", "parts": [{"text": "Analyze"},
            {"inline_data": {"data": b64, "mime_type": "image/png"}}]}]},
        output={"text": "Done"}
    )
    trace.end()
  2. Open the trace in the UI → Messages tab.
  3. Verify:
    • The input renders as a User message with a text block and an image attachment block
    • The output renders as an Assistant message
    • No raw JSON is shown

Unit tests:

  • Python: pytest tests/unit/api_objects/attachment/test_attachments_extractor.py (31 tests)
  • Frontend: npx vitest run src/shared/PrettyLLMMessage (94 tests)

Documentation

No documentation changes required.

Screenshots

google-adk-trace-detail-messages.png

google-adk-trace-full-page.png

Douglas Blank and others added 3 commits March 30, 2026 10:15
- Add Google GenAI format detector and mapper to PrettyLLMMessage
  (handles contents/parts/inline_data structure from ADK traces)
- Register google format in provider registry (was null)
- Support inline_data rendering in PromptMessagesReadonly
- Extend base64 regex to match URL-safe base64 (- and _ chars)
- Add URL-safe base64 fallback decode in Base64AttachmentDecoder

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot added python Pull requests that update Python code Frontend Infrastructure typescript *.ts *.tsx Python SDK labels Mar 30, 2026
@dsblank dsblank marked this pull request as ready for review March 30, 2026 14:59
@dsblank dsblank requested a review from a team as a code owner March 30, 2026 14:59
@dsblank dsblank marked this pull request as draft March 30, 2026 15:20
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove these screenshots, these should be comitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Google ADK integration failing to render attachments in prompt parts

1 participant