Skip to content

fix(bump/datahub): cap response body size (#65)#107

Merged
galt-tr merged 7 commits into
mainfrom
fix/issue-65-arcade-datahub-body-cap
May 1, 2026
Merged

fix(bump/datahub): cap response body size (#65)#107
galt-tr merged 7 commits into
mainfrom
fix/issue-65-arcade-datahub-body-cap

Conversation

@galt-tr
Copy link
Copy Markdown
Contributor

@galt-tr galt-tr commented May 1, 2026

Summary

  • Wraps resp.Body reads in io.LimitReader and inspects Content-Length before reading the body, so a hostile or malfunctioning DataHub cannot exhaust process memory by streaming an unbounded response.
  • Per-endpoint caps:
    • /block/<hash> (binary block metadata: 80-byte header + subtree-hash list + coinbase tx + coinbase BUMP) — default 1 GiB via bump.DefaultMaxBlockBytes. The endpoint serves block metadata only, so 1 GiB is two-plus orders of magnitude over a realistic Teranode payload while still bounding memory.
    • The 5xx-body diagnostic read (already capped at 512 bytes pre-fix) is preserved.
  • Configurable via bump_builder.datahub_max_block_bytes (BumpBuilderConfig.DataHubMaxBlockBytes); default applied via Viper. A value <= 0 falls back to bump.DefaultMaxBlockBytes so a misconfigured zero never silently disables the protection.
  • New tests in bump/datahub_test.go:
    • body larger than the cap (chunked, no Content-Length) is rejected with an error mentioning exceeds and the cap, and the error does not embed the response content;
    • body exactly at the cap is accepted (LimitReader(max+1) boundary);
    • advertised oversize Content-Length is rejected before any body bytes are read;
    • zero/negative caps fall back to the default rather than disabling the protection.
  • Mirrors the merkle-service fix in PR Merge chaintracks OpenAPI docs, add CDN support, and improve Docker configuration #21 / commit 19241ad.

Closes #65

Test plan

  • go build ./...
  • go vet ./...
  • go test ./bump/... -race
  • go test ./... -race (full suite)
  • golangci-lint run ./bump/... ./config/... ./services/bump_builder/...
  • Reviewer to confirm operational caps (1 GiB for /block/<hash> is generous; tighten if Teranode's realistic max-block-metadata payload is lower for your fleet).

#65)

Wraps DataHub responses in io.LimitReader and inspects Content-Length
before reading, returning an error instead of OOMing the process.
Caps are tuned per endpoint and configurable via DataHubConfig.
Closes F-007.
@galt-tr galt-tr requested a review from mrz1836 as a code owner May 1, 2026 18:12
@github-actions github-actions Bot added the size/L Large change (201–500 lines) label May 1, 2026
@github-actions github-actions Bot added the bug-P3 Lowest rated bug, affects nearly none or low-impact label May 1, 2026
@galt-tr galt-tr merged commit fbe1783 into main May 1, 2026
45 checks passed
@galt-tr galt-tr deleted the fix/issue-65-arcade-datahub-body-cap branch May 1, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-P3 Lowest rated bug, affects nearly none or low-impact size/L Large change (201–500 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[F-007] DataHub block fetch reads unbounded response bodies into memory

2 participants