Skip to content

fix(s3): validate x-amz-content-sha256 against received body (bug-hunt 5.4)#2294

Merged
vieiralucas merged 1 commit into
mainfrom
bh-s3sha
Jul 16, 2026
Merged

fix(s3): validate x-amz-content-sha256 against received body (bug-hunt 5.4)#2294
vieiralucas merged 1 commit into
mainfrom
bh-s3sha

Conversation

@vieiralucas

@vieiralucas vieiralucas commented Jul 16, 2026

Copy link
Copy Markdown
Member

Summary

Completes bug-hunt finding 5.4 at the correct layer. A plain (non-chunked) SigV4 PutObject carries the payload's real SHA-256 in the x-amz-content-sha256 header. fakecloud spooled the body but never compared this header against the bytes received, so a corrupt/tampered upload whose header disagreed with its body stored silently. AWS rejects the divergence with XAmzContentSHA256Mismatch (400).

Why the S3 layer, not sigv4

The earlier attempt (#2288) put this in sigv4 verification and was reverted — the signed payload is not reliably re-derivable there (aws-chunked / streaming empties the buffered body), so it wrongly rejected legitimate signed requests with IncompleteSignature. The S3 write handler holds the fully-decoded object bytes, so the hash is authoritative and there is no ambiguity.

Changes

  • core: SpooledBody gains sha256_hex, computed in the same single streaming pass as md5_hex (no extra IO). Over the decoded payload for aws-chunked bodies.
  • s3 write path: when x-amz-content-sha256 is a 64-char hex digest (not UNSIGNED-PAYLOAD, not a STREAMING-… marker, not aws-chunked), compare to the spool's sha256; return XAmzContentSHA256Mismatch on divergence.
  • Markers / streaming skip the check entirely, so correct clients (SDK, aws-cli, boto3 — which default to aws-chunked STREAMING) are unaffected.

Test plan

  • core unit: spool_computes_sha256_over_plain_payload, spool_sha256_is_over_decoded_aws_chunked_payload.
  • e2e: s3_put_object_rejects_content_sha256_mismatch — bogus hex -> 400 XAmzContentSHA256Mismatch, correct hex -> 200, UNSIGNED-PAYLOAD -> bypass.
  • regression guard (per the sigv4-body-hash lesson): iam_enforcement (48) + iam_enforcement_abac (9) — the FAKECLOUD_VERIFY_SIGV4=true paths fix(core,aws,s3): auth enforcement gaps (bug-hunt 5.1/5.3/5.4) #2288's revert restored, including the 3 tests the sigv4-layer version broke — all green.
  • s3 (80), s3_aws_chunked, s3_streaming_body, and s3 conformance 52/52 green.

No non-code surface change: behavior now matches AWS more faithfully; invisible to correct clients, no new API/flag/field, no SDK change.


Summary by cubic

Validate x-amz-content-sha256 against the received bytes for non‑chunked S3 PutObject requests. Mismatches now return 400 XAmzContentSHA256Mismatch to match AWS.

  • Bug Fixes

    • Added sha256_hex to SpooledBody, computed alongside MD5 over the decoded payload.
    • In the S3 write path, when the header is a 64‑char hex and the request is not aws‑chunked, compare it to the spool hash and return XAmzContentSHA256Mismatch (400) on mismatch. The check runs in the S3 layer, not SigV4.
    • Skip checks for UNSIGNED-PAYLOAD, STREAMING-…, and aws‑chunked bodies, so streaming SDKs remain unaffected.
  • Dependencies

    • Added sha2 (workspace).

Written for commit 99fa86b. Summary will update on new commits.

Review in cubic

…t 5.4)

A plain (non-chunked) SigV4 PutObject carries the payload's real SHA-256 in
the x-amz-content-sha256 header. fakecloud spooled the body but never compared
this header against the bytes received, so a corrupt/tampered upload whose
header disagreed with its body stored silently. AWS rejects the divergence
with XAmzContentSHA256Mismatch (400).

This lives at the S3 service layer, NOT in sigv4 verification: the signed
payload is not reliably re-derivable there (aws-chunked / streaming empties
the buffered body), which is why the earlier sigv4-layer attempt (#2288) was
reverted. The S3 handler holds the fully-decoded object bytes, so the hash is
authoritative.

- core: SpooledBody gains sha256_hex, computed in the same single streaming
  pass as md5_hex (no extra IO). Over the DECODED payload for aws-chunked.
- s3 write path: when x-amz-content-sha256 is a 64-char hex digest (not
  UNSIGNED-PAYLOAD, not a STREAMING-... marker, not aws-chunked), compare it
  to the spool's sha256 and return XAmzContentSHA256Mismatch on divergence.
- Skips markers/streaming entirely, so correct clients (SDK, aws-cli, boto3
  which default to aws-chunked STREAMING) are unaffected.

Tests: core unit tests for spool sha256 (plain + aws-chunked decoded); e2e
s3_put_object_rejects_content_sha256_mismatch (bogus hex -> 400, correct hex
-> 200, UNSIGNED-PAYLOAD -> bypass). Ran iam_enforcement + iam_enforcement_abac
(the verify_sigv4 paths #2288's revert restored), s3 (80), s3_aws_chunked,
s3_streaming_body, and s3 conformance (52/52) — all green.

No non-code surface change: behavior now matches AWS more faithfully; invisible
to correct clients, no new API/flag/field, no SDK change.
@vieiralucas vieiralucas requested a review from Copilot July 16, 2026 07:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@vieiralucas vieiralucas merged commit 5c6cc72 into main Jul 16, 2026
222 of 223 checks passed
@vieiralucas vieiralucas deleted the bh-s3sha branch July 16, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants