fix(s3): validate x-amz-content-sha256 against received body (bug-hunt 5.4)#2294
Merged
Conversation
…t 5.4) A plain (non-chunked) SigV4 PutObject carries the payload's real SHA-256 in the x-amz-content-sha256 header. fakecloud spooled the body but never compared this header against the bytes received, so a corrupt/tampered upload whose header disagreed with its body stored silently. AWS rejects the divergence with XAmzContentSHA256Mismatch (400). This lives at the S3 service layer, NOT in sigv4 verification: the signed payload is not reliably re-derivable there (aws-chunked / streaming empties the buffered body), which is why the earlier sigv4-layer attempt (#2288) was reverted. The S3 handler holds the fully-decoded object bytes, so the hash is authoritative. - core: SpooledBody gains sha256_hex, computed in the same single streaming pass as md5_hex (no extra IO). Over the DECODED payload for aws-chunked. - s3 write path: when x-amz-content-sha256 is a 64-char hex digest (not UNSIGNED-PAYLOAD, not a STREAMING-... marker, not aws-chunked), compare it to the spool's sha256 and return XAmzContentSHA256Mismatch on divergence. - Skips markers/streaming entirely, so correct clients (SDK, aws-cli, boto3 which default to aws-chunked STREAMING) are unaffected. Tests: core unit tests for spool sha256 (plain + aws-chunked decoded); e2e s3_put_object_rejects_content_sha256_mismatch (bogus hex -> 400, correct hex -> 200, UNSIGNED-PAYLOAD -> bypass). Ran iam_enforcement + iam_enforcement_abac (the verify_sigv4 paths #2288's revert restored), s3 (80), s3_aws_chunked, s3_streaming_body, and s3 conformance (52/52) — all green. No non-code surface change: behavior now matches AWS more faithfully; invisible to correct clients, no new API/flag/field, no SDK change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes bug-hunt finding 5.4 at the correct layer. A plain (non-chunked) SigV4 PutObject carries the payload's real SHA-256 in the
x-amz-content-sha256header. fakecloud spooled the body but never compared this header against the bytes received, so a corrupt/tampered upload whose header disagreed with its body stored silently. AWS rejects the divergence withXAmzContentSHA256Mismatch(400).Why the S3 layer, not sigv4
The earlier attempt (#2288) put this in sigv4 verification and was reverted — the signed payload is not reliably re-derivable there (aws-chunked / streaming empties the buffered body), so it wrongly rejected legitimate signed requests with
IncompleteSignature. The S3 write handler holds the fully-decoded object bytes, so the hash is authoritative and there is no ambiguity.Changes
SpooledBodygainssha256_hex, computed in the same single streaming pass asmd5_hex(no extra IO). Over the decoded payload for aws-chunked bodies.x-amz-content-sha256is a 64-char hex digest (notUNSIGNED-PAYLOAD, not aSTREAMING-…marker, not aws-chunked), compare to the spool's sha256; returnXAmzContentSHA256Mismatchon divergence.Test plan
spool_computes_sha256_over_plain_payload,spool_sha256_is_over_decoded_aws_chunked_payload.s3_put_object_rejects_content_sha256_mismatch— bogus hex -> 400XAmzContentSHA256Mismatch, correct hex -> 200,UNSIGNED-PAYLOAD-> bypass.iam_enforcement(48) +iam_enforcement_abac(9) — theFAKECLOUD_VERIFY_SIGV4=truepaths fix(core,aws,s3): auth enforcement gaps (bug-hunt 5.1/5.3/5.4) #2288's revert restored, including the 3 tests the sigv4-layer version broke — all green.s3(80),s3_aws_chunked,s3_streaming_body, and s3 conformance 52/52 green.No non-code surface change: behavior now matches AWS more faithfully; invisible to correct clients, no new API/flag/field, no SDK change.
Summary by cubic
Validate
x-amz-content-sha256against the received bytes for non‑chunked S3 PutObject requests. Mismatches now return 400XAmzContentSHA256Mismatchto match AWS.Bug Fixes
sha256_hextoSpooledBody, computed alongside MD5 over the decoded payload.XAmzContentSHA256Mismatch(400) on mismatch. The check runs in the S3 layer, not SigV4.UNSIGNED-PAYLOAD,STREAMING-…, and aws‑chunked bodies, so streaming SDKs remain unaffected.Dependencies
sha2(workspace).Written for commit 99fa86b. Summary will update on new commits.