Skip to content

Commit 26a08bd

Browse files
fix(attachments): SSRF guards + Message-instance rehydration + truthiness fallbacks
Second review pass on PR #67 (rehydrate_attachment). The previous fixup addressed pyrefly only — this commit resolves the remaining review feedback. SSRF guards (3 adapters) - Slack, Teams, Google Chat all rebuild fetch_data closures from serialized fetch_metadata["url"] in rehydrate_attachment. A tampered URL in persisted queue state could exfiltrate the workspace bot/OAuth token to an attacker-controlled host. Each adapter now validates the URL's scheme (https only) and host against a platform-specific allowlist before forwarding the auth header. Upstream TS does not validate; this is a Python-first divergence documented in docs/UPSTREAM_SYNC.md. - Slack: files.slack.com, slack.com, *.slack.com, *.slack-edge.com - Teams: Microsoft-owned hosts (graph.microsoft.com, smba.trafficmanager.net, *.sharepoint.com, *.botframework.com, *.office.com, attachments.office.net, …) - Google Chat: chat.googleapis.com, *.googleapis.com, *.googleusercontent.com, *.google.com Message-instance rehydration (P1) - Chat._rehydrate_message used to early-return on Message inputs, matching upstream TS's `raw instanceof Message` shortcut. That shortcut is safe in upstream because its state adapters return raw JSON dicts from dequeue. Our RedisStateAdapter / PostgresStateAdapter both upgrade the dequeued dict to `Message.from_json(...)` before returning, so the early return would skip rehydrate_attachment for every persistent-backend dequeue and leave fetch_data stripped. We now fall through and apply the rehydrate pass on Message inputs too (already-hydrated attachments with fetch_data are filtered out). Truthiness fallbacks (Port Rule #1) - telegram, whatsapp rehydrate_attachment and types.py dual-key fetch_metadata lookup now use explicit `is not None` instead of `or`, so an empty-dict fetch_metadata is preserved. Teams connection pooling - _build_teams_fetch_data used httpx.AsyncClient as a throwaway context manager per download. Refactored to use the shared aiohttp session (_get_http_session) that the rest of the adapter already goes through. Test hardening - test_slack_webhook.py and test_teams_adapter.py now stub the fetch path with AsyncMock, await rehydrated.fetch_data(), and assert the URL + token that were forwarded. Previously the tests only checked that `fetch_data is not None` — they would have passed even if rehydration returned a dummy closure. - New tests per adapter verify the SSRF guard rejects untrusted hosts and the allowlist accepts the intended Slack/Teams/GCP hosts. - New regression test in test_chat_faithful.py drives a Message- instance dequeue through the chat queue and asserts rehydrate_attachment still fires. Slack adapter connection pooling (deferred) - _fetch_slack_file still uses httpx.AsyncClient per call. The Slack adapter has no pooled aiohttp helper (only slack_sdk.AsyncWebClient for Slack API calls), so adding one is a larger refactor left for a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 717a2d2 commit 26a08bd

12 files changed

Lines changed: 492 additions & 33 deletions

File tree

docs/UPSTREAM_SYNC.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -458,6 +458,8 @@ stay explicit instead of being rediscovered in code review.
458458
| Teams divider rendering | `card_to_adaptive_card` hoists `separator: True` onto the next sibling (or emits a non-empty Container for a trailing divider) | `convertDividerToElement` emits an empty `Container` with `separator: True` | Upstream shares the same bug: Microsoft Teams renders an empty Container at zero height, so the separator line is effectively invisible. Python port fixes locally (issue #45) rather than blocking on upstream. |
459459
| `SlackAdapter.current_token` / `current_client` | Public `@property` accessors that return the request-context-bound token and a preconfigured `AsyncWebClient` | Not exposed (`getToken()` is private on the TS `SlackAdapter`) | Python-only addition (issue #47). Downstream code that calls Slack Web APIs from inside a handler — email resolution, user profile fetches, reaction bookkeeping — otherwise depends on underscore-prefixed helpers. |
460460
| `ConcurrencyConfig.max_concurrent` | Enforced via `asyncio.Semaphore` in the `"concurrent"` strategy path; rejects non-integer or `<= 0` values, and rejects any non-`None` `max_concurrent` paired with a non-`"concurrent"` strategy | Accepted into the config type with docstring "Default: Infinity" but never read (3 writes, 0 reads) | Silent correctness bug upstream — consumers setting `max_concurrent=N` with `strategy="concurrent"` reasonably expect an N-way bound on in-flight handlers. We honor the documented contract via a semaphore and fail-fast on misconfiguration so it's never silent. `max_concurrent=None` stays compatible with every strategy (unbounded default). |
461+
| `rehydrate_attachment` URL allowlist (Slack / Teams / Google Chat) | Validates the downloaded URL's scheme + host against a per-adapter allowlist inside the fetch closure; raises `ValidationError` on untrusted hosts before forwarding bearer tokens | No validation — `fetchData` blindly GETs `fetchMetadata.url` and forwards the workspace/bot token | SSRF + token-exfil risk upstream: after the 4.26 `rehydrateAttachment` hook lands, a crafted `fetchMetadata` in persisted state can redirect auth'd downloads to an arbitrary host. Python port enforces `CLAUDE.md`'s "Validate external URLs before requests (SSRF)" rule. Allowlist: Slack = `{files.slack.com, slack.com, *.slack.com, *.slack-edge.com}`; Teams = `{smba.trafficmanager.net, graph.microsoft.com, attachments.office.net, *.botframework.com, *.graph.microsoft.com, *.sharepoint.com, *.officeapps.live.com, *.office.com, *.office365.com, *.onedrive.com, *.microsoft.com}`; Google Chat = `{chat.googleapis.com, googleapis.com, *.googleapis.com, *.googleusercontent.com, *.google.com}`. |
462+
| `_rehydrate_message` with `Message` input | Falls through to the `rehydrate_attachment` pass even when the dequeued entry is already a `Message` instance | Early-returns on `raw instanceof Message` before rehydration | The Python port's Redis + Postgres `dequeue()` upgrade raw JSON to `Message.from_json(...)` before returning (upstream's dequeue returns the raw JSON.parse'd dict). Upstream's `instanceof Message` shortcut therefore only fires for in-memory state, but ours would fire for persistent backends too, leaving `fetch_data` stripped forever. The rehydrate pass still skips any attachment that already has `fetch_data`, so in-memory callers pay no cost. |
461463

462464
### Platform-specific gaps
463465

src/chat_sdk/adapters/google_chat/adapter.py

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
from collections.abc import AsyncIterable, Awaitable, Callable
2020
from datetime import datetime, timezone
2121
from typing import Any, NoReturn
22+
from urllib.parse import urlparse
2223

2324
from chat_sdk.adapters.google_chat.cards import card_to_google_card
2425
from chat_sdk.adapters.google_chat.format_converter import GoogleChatFormatConverter
@@ -2631,6 +2632,37 @@ def _create_attachment(self, att: dict[str, Any]) -> Attachment:
26312632
fetch_metadata=fetch_meta or None,
26322633
)
26332634

2635+
@staticmethod
2636+
def _is_trusted_gchat_download_url(url: str) -> bool:
2637+
"""Gate Google Chat attachment downloads to Google-owned hosts.
2638+
2639+
After ``rehydrate_attachment`` reconstructs the fetch closure
2640+
from serialized ``fetch_metadata``, the URL may have been
2641+
tampered with in the state store. We refuse to forward the
2642+
OAuth access token unless the host is a known Google-owned host.
2643+
2644+
This is a Python-first divergence: upstream Google Chat adapter
2645+
does not validate the URL. See ``docs/UPSTREAM_SYNC.md`` Known
2646+
Non-Parity.
2647+
"""
2648+
try:
2649+
parsed = urlparse(url)
2650+
except (ValueError, TypeError):
2651+
return False
2652+
if parsed.scheme != "https":
2653+
return False
2654+
host = (parsed.hostname or "").lower()
2655+
if not host:
2656+
return False
2657+
allowed_suffixes = (
2658+
".googleapis.com",
2659+
".googleusercontent.com",
2660+
".google.com",
2661+
)
2662+
if host.endswith(allowed_suffixes):
2663+
return True
2664+
return host in {"chat.googleapis.com", "googleapis.com"}
2665+
26342666
def _build_gchat_fetch_data(
26352667
self,
26362668
resource_name: str | None,
@@ -2656,8 +2688,15 @@ async def _fetch_data() -> bytes:
26562688
)
26572689
return await response.read()
26582690

2659-
# Fallback to direct URL fetch (downloadUri)
2691+
# Fallback to direct URL fetch (downloadUri). Validate the
2692+
# host before forwarding the OAuth token — the URL may have
2693+
# been rebuilt from serialized metadata and tampered with.
26602694
if url:
2695+
if not adapter._is_trusted_gchat_download_url(url):
2696+
raise ValidationError(
2697+
"gchat",
2698+
f"Refusing to fetch Google Chat file from untrusted URL: {url}",
2699+
)
26612700
token = await adapter._get_access_token()
26622701
session = await adapter._get_http_session()
26632702
async with session.get(
@@ -2682,7 +2721,7 @@ def rehydrate_attachment(self, attachment: Attachment) -> Attachment:
26822721
``url`` (fallback) from ``fetch_metadata``. Returns the attachment
26832722
unchanged when neither identifier is present.
26842723
"""
2685-
meta = attachment.fetch_metadata or {}
2724+
meta = attachment.fetch_metadata if attachment.fetch_metadata is not None else {}
26862725
resource_name = meta.get("resourceName")
26872726
meta_url = meta.get("url")
26882727
url = meta_url if meta_url is not None else attachment.url

src/chat_sdk/adapters/slack/adapter.py

Lines changed: 44 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
from contextvars import ContextVar
2424
from datetime import datetime, timezone
2525
from typing import Any, NoReturn, cast
26-
from urllib.parse import parse_qs
26+
from urllib.parse import parse_qs, urlparse
2727

2828
from chat_sdk.adapters.slack.cards import (
2929
card_to_block_kit,
@@ -1773,13 +1773,48 @@ async def fetch_data() -> bytes:
17731773
fetch_metadata=fetch_meta or None,
17741774
)
17751775

1776+
@staticmethod
1777+
def _is_trusted_slack_download_url(url: str) -> bool:
1778+
"""Gate Slack file downloads to known Slack-owned hosts.
1779+
1780+
We refuse to forward ``Authorization: Bearer {token}`` to an
1781+
arbitrary URL. After ``rehydrate_attachment`` reconstructs the
1782+
fetch closure from serialized ``fetch_metadata``, that URL may
1783+
have been tampered with in the state store — a crafted value
1784+
could exfiltrate the workspace bot token.
1785+
1786+
This is a Python-first divergence: upstream Slack adapter does not
1787+
validate the URL. See ``docs/UPSTREAM_SYNC.md`` Known Non-Parity.
1788+
"""
1789+
try:
1790+
parsed = urlparse(url)
1791+
except (ValueError, TypeError):
1792+
return False
1793+
if parsed.scheme != "https":
1794+
return False
1795+
host = (parsed.hostname or "").lower()
1796+
if not host:
1797+
return False
1798+
# Exact-match hosts
1799+
if host in {"files.slack.com", "slack.com"}:
1800+
return True
1801+
# Suffix match for Slack-owned subdomains
1802+
return host.endswith(".slack.com") or host.endswith(".slack-edge.com")
1803+
17761804
async def _fetch_slack_file(self, url: str, token: str) -> bytes:
17771805
"""Download a file from a Slack ``url_private`` endpoint.
17781806
17791807
Shared by :meth:`_create_attachment` (direct fetch closure) and
17801808
:meth:`rehydrate_attachment` (reconstructed closure after JSON
1781-
roundtrip).
1809+
roundtrip). Validates the host against the Slack allowlist
1810+
before forwarding the bot token (SSRF guard).
17821811
"""
1812+
if not self._is_trusted_slack_download_url(url):
1813+
raise ValidationError(
1814+
"slack",
1815+
f"Refusing to fetch Slack file from untrusted URL: {url}",
1816+
)
1817+
17831818
import httpx
17841819

17851820
async with httpx.AsyncClient() as http:
@@ -1801,10 +1836,14 @@ def rehydrate_attachment(self, attachment: Attachment) -> Attachment:
18011836
``attachment.fetch_metadata``, and rebuilds a ``fetch_data`` closure
18021837
that resolves the workspace-specific bot token at call time.
18031838
1804-
Returns the attachment unchanged when no URL is available.
1839+
Returns the attachment unchanged when no URL is available. The
1840+
URL is re-validated inside the closure (by ``_fetch_slack_file``)
1841+
rather than here so that a trusted-at-serialize-time URL still
1842+
fails closed if the allowlist tightens later.
18051843
"""
1806-
meta = attachment.fetch_metadata or {}
1807-
url = meta.get("url") or attachment.url
1844+
meta = attachment.fetch_metadata if attachment.fetch_metadata is not None else {}
1845+
meta_url = meta.get("url")
1846+
url = meta_url if meta_url is not None else attachment.url
18081847
team_id = meta.get("teamId")
18091848
if not url:
18101849
return attachment

src/chat_sdk/adapters/teams/adapter.py

Lines changed: 67 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
from collections.abc import Awaitable, Callable
1818
from datetime import datetime, timezone
1919
from typing import Any, Literal, NoReturn
20+
from urllib.parse import urlparse
2021

2122
from chat_sdk.adapters.teams.cards import card_to_adaptive_card
2223
from chat_sdk.adapters.teams.format_converter import TeamsFormatConverter
@@ -591,16 +592,70 @@ def _create_attachment(self, att: dict[str, Any]) -> Attachment:
591592
fetch_data=self._build_teams_fetch_data(url) if url else None,
592593
)
593594

595+
@staticmethod
596+
def _is_trusted_teams_download_url(url: str) -> bool:
597+
"""Gate Teams file downloads to Microsoft-owned hosts.
598+
599+
After ``rehydrate_attachment`` reconstructs the fetch closure
600+
from serialized ``fetch_metadata``, the URL may have been
601+
tampered with. We refuse to issue a direct GET unless the host
602+
is a known Microsoft/Graph download host.
603+
604+
This is a Python-first divergence: upstream Teams adapter does
605+
not validate the URL. See ``docs/UPSTREAM_SYNC.md`` Known
606+
Non-Parity.
607+
"""
608+
try:
609+
parsed = urlparse(url)
610+
except (ValueError, TypeError):
611+
return False
612+
if parsed.scheme != "https":
613+
return False
614+
host = (parsed.hostname or "").lower()
615+
if not host:
616+
return False
617+
# Microsoft Graph / Bot Framework / SharePoint / Teams file hosts
618+
allowed_suffixes = (
619+
".botframework.com",
620+
".graph.microsoft.com",
621+
".sharepoint.com",
622+
".officeapps.live.com",
623+
".office.com",
624+
".office365.com",
625+
".onedrive.com",
626+
".microsoft.com",
627+
)
628+
if host.endswith(allowed_suffixes):
629+
return True
630+
# Exact-match traffic-manager / Graph / Teams service hosts
631+
return host in {
632+
"smba.trafficmanager.net",
633+
"graph.microsoft.com",
634+
"attachments.office.net",
635+
}
636+
594637
def _build_teams_fetch_data(self, url: str) -> Callable[[], Awaitable[bytes]]:
595-
"""Build a lazy ``fetch_data`` closure for a Teams file URL."""
638+
"""Build a lazy ``fetch_data`` closure for a Teams file URL.
596639
597-
async def fetch_data() -> bytes:
598-
import httpx
640+
Uses the adapter's shared ``aiohttp.ClientSession`` (via
641+
:meth:`_get_http_session`) so downloads reuse the connection
642+
pool instead of constructing a throwaway client per request.
643+
"""
599644

600-
async with httpx.AsyncClient() as http:
601-
resp = await http.get(url)
602-
resp.raise_for_status()
603-
return resp.content
645+
async def fetch_data() -> bytes:
646+
if not self._is_trusted_teams_download_url(url):
647+
raise ValidationError(
648+
"teams",
649+
f"Refusing to fetch Teams file from untrusted URL: {url}",
650+
)
651+
session = await self._get_http_session()
652+
async with session.get(url) as resp:
653+
if resp.status >= 400:
654+
raise NetworkError(
655+
"teams",
656+
f"Failed to fetch file: {resp.status}",
657+
)
658+
return await resp.read()
604659

605660
return fetch_data
606661

@@ -611,9 +666,12 @@ def rehydrate_attachment(self, attachment: Attachment) -> Attachment:
611666
need to rebuild the download closure is the URL — either from
612667
``fetch_metadata["url"]`` or the attachment's top-level ``url``.
613668
Returns the attachment unchanged when no URL is available.
669+
The URL host is validated inside the closure, so tampered URLs
670+
raise at fetch time.
614671
"""
615-
meta = attachment.fetch_metadata or {}
616-
url = meta.get("url") or attachment.url
672+
meta = attachment.fetch_metadata if attachment.fetch_metadata is not None else {}
673+
meta_url = meta.get("url")
674+
url = meta_url if meta_url is not None else attachment.url
617675
if not url:
618676
return attachment
619677
return Attachment(

src/chat_sdk/adapters/telegram/adapter.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1376,7 +1376,7 @@ def rehydrate_attachment(self, attachment: Attachment) -> Attachment:
13761376
no file ID is present (e.g. a pre-serialized attachment that did
13771377
not originate from this adapter).
13781378
"""
1379-
meta = attachment.fetch_metadata or {}
1379+
meta = attachment.fetch_metadata if attachment.fetch_metadata is not None else {}
13801380
file_id = meta.get("fileId")
13811381
if not file_id:
13821382
return attachment

src/chat_sdk/adapters/whatsapp/adapter.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -643,7 +643,7 @@ def rehydrate_attachment(self, attachment: Attachment) -> Attachment:
643643
``download_media`` closure. Returns the attachment unchanged when
644644
no media ID is present.
645645
"""
646-
meta = attachment.fetch_metadata or {}
646+
meta = attachment.fetch_metadata if attachment.fetch_metadata is not None else {}
647647
media_id = meta.get("mediaId")
648648
if not media_id:
649649
return attachment

src/chat_sdk/chat.py

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2048,13 +2048,21 @@ def _rehydrate_message(self, raw: Any, adapter: Adapter | None = None) -> Messag
20482048
attachment that lost its ``fetch_data`` closure so downstream
20492049
handlers can still download bytes.
20502050
"""
2051-
# Matches upstream: if the entry is already a Message instance, its
2052-
# fetch_data closures never went through a JSON roundtrip, so we
2053-
# return it untouched — no rehydrate pass.
2051+
# Diverges from upstream: upstream TS has
2052+
# ``if (raw instanceof Message) return raw;`` because its Redis /
2053+
# Postgres ``dequeue()`` returns the raw ``JSON.parse(value)`` —
2054+
# never a ``Message`` instance. Our Python port's Redis +
2055+
# Postgres ``dequeue()`` already upgrade the raw dict to
2056+
# ``Message.from_json(...)`` before returning (see
2057+
# ``state/redis.py`` and ``state/postgres.py``). An early return
2058+
# here would therefore skip ``rehydrate_attachment`` for every
2059+
# dequeued Message in a persistent backend, leaving
2060+
# ``fetch_data`` stripped. We fall through and apply the
2061+
# rehydrate pass; attachments that still have ``fetch_data``
2062+
# (e.g. in-memory state) are filtered out below.
20542063
if isinstance(raw, Message):
2055-
return raw
2056-
2057-
if isinstance(raw, dict):
2064+
msg = raw
2065+
elif isinstance(raw, dict):
20582066
if raw.get("_type") == "chat:Message":
20592067
msg = _message_from_json(raw)
20602068
else:

src/chat_sdk/types.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -518,7 +518,9 @@ def from_json(cls, data: dict[str, Any] | Message) -> Message:
518518
size=att.get("size"),
519519
width=att.get("width"),
520520
height=att.get("height"),
521-
fetch_metadata=att.get("fetchMetadata") or att.get("fetch_metadata"),
521+
fetch_metadata=(
522+
att.get("fetchMetadata") if att.get("fetchMetadata") is not None else att.get("fetch_metadata")
523+
),
522524
)
523525
for att in attachments_data
524526
],
@@ -590,7 +592,9 @@ def from_json_compat(cls, data: dict[str, Any]) -> Message:
590592
size=att.get("size"),
591593
width=att.get("width"),
592594
height=att.get("height"),
593-
fetch_metadata=att.get("fetch_metadata") or att.get("fetchMetadata"),
595+
fetch_metadata=(
596+
att.get("fetch_metadata") if att.get("fetch_metadata") is not None else att.get("fetchMetadata")
597+
),
594598
)
595599
for att in attachments_data
596600
],

0 commit comments

Comments
 (0)