fix: simplify isBinaryPayload to use encode="bytes" as binary signal

[Copilot]

Fixes #3913

Background

In TCGC, the encoding for a bytes body is determined as follows:

• For */* and application/octet-stream content types, TCGC natively sets encode to "bytes".
• For all other content types (e.g., application/json, application/xml), a bytes body defaults to "base64" encoding — unless the user explicitly annotates the body with @encode("bytes"), which overrides the encode to "bytes".

Change

Given the TCGC behavior above, isBinaryPayload is extended with an optional encode parameter. When encode === "bytes", the function returns true immediately.

This approach:

• Naturally handles the */* case — since TCGC already sets encode to "bytes" for wildcard and application/octet-stream content types, no special-casing for */* is needed anymore. The dedicated */* wildcard detection path can be removed in favor of this unified encode-based check.
• Correctly handles explicit @encode("bytes") annotations on non-binary content types (e.g., application/json + @encode("bytes") → treated as binary).
• Keeps backward compatibility with m4 → TCGC migration, since the encode field reliably reflects the wire-level intent.

Call sites updated

Three Modular call sites in operationHelpers.ts are updated to pass getEncodeForType(...) as the new encode argument:

• getDeserializePrivateFunction (response deserialization)
• buildBodyParameter (request body serialization)
• checkWrapNonModelReturn (non-model return wrapping)

Scope note

The RLC call site at transform/transformResponses.ts:176 is intentionally not updated in this PR. The RLC pipeline passes a raw TypeSpec Type (not a TCGC SdkType), so getEncodeForType is not directly available there. RLC behavior is unchanged by this PR.

Follow-up

A separate issue will track refactoring isBinaryPayload to operate on SdkType directly so we can drop the __raw dependency and the encode-based coupling. See #4001.

Tests

Added snapshot scenarios under test/modularUnit/scenarios/operations/bodyParam/bytesWithDifferentContentType.md covering:

• bytes + */* → binary
• bytes + application/xml + @encode("bytes") → binary
• bytes + application/json + @encode("bytes") → binary
• bytes + text/plain + @encode("bytes") (current behavior captured)
• bytes + application/json (no @encode) → base64 (not binary)
• Request-body variants of the above.