fix(tika-worker): stream S3 blobs to Tika instead of buffering (#189)

[jensens]

Problem

TikaWorker materialized each blob entirely in memory before sending it to Tika — peak ≈ 2–3× the blob size (S3 download_fileobj buffer + getvalue() copy + httpx request body). On a deployment with S3-tiered blobs and a small worker pod (256Mi limit), a single large PDF/image OOM-killed the worker (exit 137) into CrashLoopBackOff, stalling extraction. Raising the memory limit only masks it.

Fix

_extract() now streams the S3 path straight from S3 into the Tika request body:

obj = s3.get_object(Bucket=bucket, Key=s3_key)
headers["Content-Length"] = str(obj["ContentLength"])
content = obj["Body"].iter_chunks(chunk_size=1 MiB)   # boto3 StreamingBody
client.put(f"{tika_url}/tika", content=content, headers=headers)

Bytes flow S3 → worker → Tika in 1 MiB chunks, so the worker never holds the whole blob and its memory stays roughly constant regardless of blob size. Content-Length from get_object avoids chunked transfer-encoding.

Blob-location logic is split into _blob_source() (returns a bytes or s3 descriptor without materializing S3); _extract() streams S3 while _fetch_blob() keeps materializing for its remaining callers. The PG-bytea path (small inline blobs below the S3 tiering threshold) is unchanged.

Pairs with the configurable TIKA_WORKER_HTTP_TIMEOUT (#183).

Tests

tests/test_tika_worker_extras.py — 2 new: the S3 path hands the iter_chunks iterator (not a bytes buffer) to httpx with Content-Length and never calls download_fileobj; the PG-bytea path still sends bytes. _extract's existing tests adjusted for the _blob_source split. 18 passed, 2 skipped.

Closes #189

🤖 Generated with Claude Code