feat: charset-aware, binary-safe body previews in request/response logging

Body previews in the default sync and async instrumentation steps decoded
every captured payload as UTF-8. That produced mojibake for a text body
declared in another charset (a latin-1 response logged `café` as `caf?`)
and a run of U+FFFD replacement characters for a binary body (gzip, an
image), which is both unreadable and capable of corrupting log viewers.

Add a shared `BodyPreview` renderer that the two steps delegate to:

- Text bodies are decoded with the charset declared on the body's
  `MediaType`, falling back to UTF-8 when the media type omits a charset
  or names one the JVM cannot resolve.
- A short text/binary sniff (NUL byte, or a high ratio of non-whitespace
  control bytes in a leading sample) classifies binary bodies, which are
  rendered as a size-only `[binary N bytes]` summary instead of being
  decoded as text.

Decoding still substitutes the replacement character for malformed input
rather than throwing, so a snapshot truncated mid-multibyte-sequence by the
bounded capture cannot crash the log line. The bounded/streaming capture
behaviour of the loggable bodies is unchanged.

Author: OmarAlJarrah

sdk-core/src/main/kotlin/org/dexpace/sdk/core/http/pipeline/steps/BodyPreview.kt

@@ -0,0 +1,127 @@
+/*
+ * Copyright (c) 2026 dexpace and Omar Aljarrah
+ *
+ * Licensed under the MIT License. See LICENSE in the project root.
+ * SPDX-License-Identifier: MIT
+ */
+
+package org.dexpace.sdk.core.http.pipeline.steps
+
+import org.dexpace.sdk.core.http.common.MediaType
+import java.nio.charset.Charset
+
+/**
+ * Renders captured request/response body bytes into a short, log-safe preview string.
+ *
+ * The default instrumentation steps previously decoded every preview as UTF-8, which produced
+ * mojibake for a text body declared in another charset (e.g. ISO-8859-1) and a stream of
+ * `U+FFFD` replacement characters for a binary body (a gzip payload, an image, …). This helper
+ * makes previews:
+ *
+ * - **Charset-aware** — a text body is decoded with the charset declared on its [MediaType]
+ *   (`Content-Type: text/plain;charset=ISO-8859-1`). When the media type omits a charset, or
+ *   names one the JVM does not recognise, the preview falls back to [DEFAULT_CHARSET] (UTF-8),
+ *   matching the HTTP default for most text types and the previous behaviour.
+ * - **Binary-safe** — a body that does not look like text is never decoded. Instead it is
+ *   summarised as `[binary N bytes]`, so an operator sees a stable, greppable marker rather
+ *   than a line of replacement characters that can corrupt log viewers.
+ *
+ * Decoding itself never throws: [Charset]-based decoding substitutes the replacement character
+ * for malformed input, so a snapshot that ends mid-multibyte-sequence (a real possibility on a
+ * bounded capture) still yields a usable preview rather than crashing the log line.
+ *
+ * This is an `internal` seam shared by [DefaultInstrumentationStep] and
+ * [DefaultAsyncInstrumentationStep]; it has no public API surface.
+ */
+internal object BodyPreview {
+    /** Charset used when a text body declares no charset, or names an unknown one. */
+    internal val DEFAULT_CHARSET: Charset = Charsets.UTF_8
+
+    /**
+     * Number of leading bytes inspected by [isProbablyText]. A small fixed sample keeps the
+     * heuristic O(1) regardless of body size while still catching the common binary signatures
+     * (NUL bytes, control-byte runs) that appear near the start of a payload.
+     */
+    private const val SNIFF_SAMPLE_BYTES = 1024
+
+    /**
+     * Fraction of sampled bytes that may be non-text control bytes before the body is judged
+     * binary. Real text can carry the odd control byte (an ESC sequence, a stray form feed), so
+     * the cutoff sits well above zero rather than rejecting on the first control byte.
+     */
+    private const val MAX_CONTROL_BYTE_RATIO = 0.30
+
+    // Byte values used by the text/binary heuristic, named so the rule logic reads clearly and
+    // so the values are not flagged as inline magic numbers.
+    private const val UNSIGNED_BYTE_MASK = 0xFF
+    private const val NUL = 0x00
+    private const val FIRST_PRINTABLE = 0x20 // space; bytes below this are C0 control bytes
+    private const val TAB = 0x09
+    private const val LINE_FEED = 0x0A
+    private const val CARRIAGE_RETURN = 0x0D
+    private const val DEL = 0x7F
+
+    /**
+     * Renders [bytes] as a preview string.
+     *
+     * Empty input yields the empty string. A body that does not pass [isProbablyText] is rendered
+     * as a size-only `[binary N bytes]` summary. Otherwise the bytes are decoded with the charset
+     * from [mediaType] (or [DEFAULT_CHARSET] when absent/unknown).
+     */
+    internal fun render(
+        bytes: ByteArray,
+        mediaType: MediaType?,
+    ): String {
+        if (bytes.isEmpty()) return ""
+        if (!isProbablyText(bytes)) return binarySummary(bytes.size)
+        return previewText(bytes, mediaType)
+    }
+
+    /**
+     * Decodes [bytes] using the charset declared on [mediaType], falling back to
+     * [DEFAULT_CHARSET] when the media type is null, declares no charset, or names a charset the
+     * JVM cannot resolve ([MediaType.charset] returns null in the latter two cases). Invalid byte
+     * sequences are replaced rather than throwing.
+     */
+    internal fun previewText(
+        bytes: ByteArray,
+        mediaType: MediaType?,
+    ): String {
+        if (bytes.isEmpty()) return ""
+        val charset = mediaType?.charset ?: DEFAULT_CHARSET
+        return String(bytes, charset)
+    }
+
+    /**
+     * Heuristically decides whether [bytes] is text. Samples the first [SNIFF_SAMPLE_BYTES]:
+     * a single NUL byte is treated as a strong binary signal, and a control-byte ratio above
+     * [MAX_CONTROL_BYTE_RATIO] (excluding the common text whitespace `\t`, `\n`, `\r`) also marks
+     * the body binary. Bytes >= 0x80 are not counted against the body — they are legitimate in
+     * UTF-8 multibyte sequences and in single-byte charsets such as ISO-8859-1.
+     */
+    @Suppress("ReturnCount")
+    internal fun isProbablyText(bytes: ByteArray): Boolean {
+        if (bytes.isEmpty()) return true
+        val sample = minOf(bytes.size, SNIFF_SAMPLE_BYTES)
+        var controlBytes = 0
+        for (i in 0 until sample) {
+            val b = bytes[i].toInt() and UNSIGNED_BYTE_MASK
+            // A NUL byte essentially never appears in real text and is the canonical marker of a
+            // binary payload (gzip, images, protobuf, …); reject immediately.
+            if (b == NUL) return false
+            if (isControlByte(b)) controlBytes++
+        }
+        return controlBytes.toDouble() / sample <= MAX_CONTROL_BYTE_RATIO
+    }
+
+    /**
+     * True for an ASCII C0 control byte that is not one of the whitespace characters routinely
+     * found in text ([TAB], [LINE_FEED], [CARRIAGE_RETURN]), or for the [DEL] byte. High bytes
+     * (>= 0x80) are deliberately excluded — see [isProbablyText].
+     */
+    private fun isControlByte(b: Int): Boolean =
+        (b < FIRST_PRINTABLE && b != TAB && b != LINE_FEED && b != CARRIAGE_RETURN) || b == DEL
+
+    /** The size-only summary emitted for a binary body. */
+    private fun binarySummary(size: Int): String = "[binary $size bytes]"
+}

sdk-core/src/main/kotlin/org/dexpace/sdk/core/http/pipeline/steps/DefaultAsyncInstrumentationStep.kt

@@ -308,13 +308,13 @@ public class DefaultAsyncInstrumentationStep
                     if (requestBody != null) {
                         val preview = requestBody.snapshot(options.bodyPreviewMaxBytes)
                         ev.field("request.body.size", preview.size.toLong())
-                            .field("request.body.preview", utf8Preview(preview))
+                            .field("request.body.preview", BodyPreview.render(preview, requestBody.mediaType()))
                     }
                     val responseBody = response.body
                     if (responseBody is LoggableResponseBody) {
                         val preview = responseBody.snapshot(options.bodyPreviewMaxBytes)
                         ev.field("response.body.size", preview.size.toLong())
-                            .field("response.body.preview", utf8Preview(preview))
+                            .field("response.body.preview", BodyPreview.render(preview, responseBody.mediaType()))
                         responseBody.captureException?.let {
                             ev.field("response.body.drain_error", it.javaClass.simpleName ?: "Throwable")
                         }
@@ -346,7 +346,7 @@ public class DefaultAsyncInstrumentationStep
                 if (shouldCaptureBody() && requestBody != null) {
                     val preview = requestBody.snapshot(options.bodyPreviewMaxBytes)
                     ev.field("request.body.size", preview.size.toLong())
-                        .field("request.body.preview", utf8Preview(preview))
+                        .field("request.body.preview", BodyPreview.render(preview, requestBody.mediaType()))
                 }
                 ev.log()
             } catch (t: Throwable) {
@@ -416,13 +416,6 @@ public class DefaultAsyncInstrumentationStep
 
         private fun elapsedMillis(startNanos: Long): Double = (clock.monotonic() - startNanos) / NANOS_PER_MILLI_DOUBLE
 
-        private fun utf8Preview(bytes: ByteArray): String {
-            if (bytes.isEmpty()) return ""
-            // Defensive: a snapshot that ends mid-UTF-8 codepoint shouldn't crash the log line.
-            // String(bytes, charset) replaces invalid sequences rather than throwing.
-            return String(bytes, Charsets.UTF_8)
-        }
-
         private fun emitInstrumentationError(
             event: String,
             phase: String,

sdk-core/src/main/kotlin/org/dexpace/sdk/core/http/pipeline/steps/DefaultInstrumentationStep.kt

@@ -212,13 +212,13 @@ public class DefaultInstrumentationStep
                     if (requestBody != null) {
                         val preview = requestBody.snapshot(options.bodyPreviewMaxBytes)
                         ev.field("request.body.size", preview.size.toLong())
-                            .field("request.body.preview", utf8Preview(preview))
+                            .field("request.body.preview", BodyPreview.render(preview, requestBody.mediaType()))
                     }
                     val responseBody = response.body
                     if (responseBody is LoggableResponseBody) {
                         val preview = responseBody.snapshot(options.bodyPreviewMaxBytes)
                         ev.field("response.body.size", preview.size.toLong())
-                            .field("response.body.preview", utf8Preview(preview))
+                            .field("response.body.preview", BodyPreview.render(preview, responseBody.mediaType()))
                         responseBody.captureException?.let {
                             ev.field("response.body.drain_error", it.javaClass.simpleName ?: "Throwable")
                         }
@@ -250,7 +250,7 @@ public class DefaultInstrumentationStep
                 if (shouldCaptureBody() && requestBody != null) {
                     val preview = requestBody.snapshot(options.bodyPreviewMaxBytes)
                     ev.field("request.body.size", preview.size.toLong())
-                        .field("request.body.preview", utf8Preview(preview))
+                        .field("request.body.preview", BodyPreview.render(preview, requestBody.mediaType()))
                 }
                 ev.log()
             } catch (t: Throwable) {
@@ -320,13 +320,6 @@ public class DefaultInstrumentationStep
 
         private fun elapsedMillis(startNanos: Long): Double = (clock.monotonic() - startNanos) / NANOS_PER_MILLI_DOUBLE
 
-        private fun utf8Preview(bytes: ByteArray): String {
-            if (bytes.isEmpty()) return ""
-            // Defensive: a snapshot that ends mid-UTF-8 codepoint shouldn't crash the log line.
-            // String(bytes, charset) replaces invalid sequences rather than throwing.
-            return String(bytes, Charsets.UTF_8)
-        }
-
         private fun emitInstrumentationError(
             event: String,
             phase: String,

sdk-core/src/test/kotlin/org/dexpace/sdk/core/http/pipeline/steps/BodyPreviewTest.kt

@@ -0,0 +1,85 @@
+/*
+ * Copyright (c) 2026 dexpace and Omar Aljarrah
+ *
+ * Licensed under the MIT License. See LICENSE in the project root.
+ * SPDX-License-Identifier: MIT
+ */
+
+package org.dexpace.sdk.core.http.pipeline.steps
+
+import org.dexpace.sdk.core.http.common.MediaType
+import java.nio.charset.StandardCharsets
+import kotlin.test.Test
+import kotlin.test.assertEquals
+import kotlin.test.assertFalse
+import kotlin.test.assertTrue
+
+class BodyPreviewTest {
+    @Test
+    fun `empty bytes render to empty string`() {
+        assertEquals("", BodyPreview.render(ByteArray(0), MediaType.parse("text/plain")))
+    }
+
+    @Test
+    fun `text body declared as ISO-8859-1 is decoded with that charset`() {
+        // 0xE9 is 'é' in ISO-8859-1. Decoded as UTF-8 it would be the U+FFFD replacement char.
+        val bytes = "café".toByteArray(StandardCharsets.ISO_8859_1)
+        val preview = BodyPreview.render(bytes, MediaType.parse("text/plain;charset=ISO-8859-1"))
+        assertEquals("café", preview)
+        assertFalse(preview.contains('�'), "latin-1 body must not be decoded as mojibake")
+    }
+
+    @Test
+    fun `text body declared as UTF-8 is decoded with UTF-8`() {
+        val bytes = "naïve — text".toByteArray(StandardCharsets.UTF_8)
+        assertEquals("naïve — text", BodyPreview.render(bytes, MediaType.parse("text/plain;charset=utf-8")))
+    }
+
+    @Test
+    fun `text body with no declared charset falls back to UTF-8`() {
+        val bytes = "résumé".toByteArray(StandardCharsets.UTF_8)
+        // No charset parameter on the media type → documented default is UTF-8.
+        assertEquals("résumé", BodyPreview.render(bytes, MediaType.parse("text/plain")))
+        // A null media type also falls back to UTF-8.
+        assertEquals("résumé", BodyPreview.render(bytes, null))
+    }
+
+    @Test
+    fun `unknown declared charset falls back to UTF-8`() {
+        val bytes = "hello".toByteArray(StandardCharsets.UTF_8)
+        // MediaType.charset returns null for an unrecognised charset name; preview falls back.
+        assertEquals("hello", BodyPreview.render(bytes, MediaType.parse("text/plain;charset=not-a-real-charset")))
+    }
+
+    @Test
+    fun `binary body with NUL byte is summarised as size only and not decoded`() {
+        // gzip magic header (0x1f 0x8b 0x08) followed by a NUL byte — a representative binary body.
+        val bytes = byteArrayOf(0x1f, 0x8b.toByte(), 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03)
+        val preview = BodyPreview.render(bytes, MediaType.parse("application/gzip"))
+        assertEquals("[binary ${bytes.size} bytes]", preview)
+        assertFalse(preview.contains('�'), "binary body must not be rendered as replacement chars")
+    }
+
+    @Test
+    fun `binary body declared with a text charset is still detected as binary`() {
+        // Even if a server mislabels a gzip stream as text, the NUL byte keeps it from being decoded.
+        val bytes = byteArrayOf(0x1f, 0x8b.toByte(), 0x00, 0x42, 0x00, 0x13)
+        val preview = BodyPreview.render(bytes, MediaType.parse("text/plain;charset=utf-8"))
+        assertEquals("[binary ${bytes.size} bytes]", preview)
+    }
+
+    @Test
+    fun `isProbablyText accepts plain ASCII and latin-1 high bytes`() {
+        assertTrue(BodyPreview.isProbablyText("plain ascii text".toByteArray(StandardCharsets.US_ASCII)))
+        assertTrue(BodyPreview.isProbablyText("café".toByteArray(StandardCharsets.ISO_8859_1)))
+        // Common text whitespace control bytes do not flip the verdict.
+        assertTrue(BodyPreview.isProbablyText("line1\r\n\tline2".toByteArray(StandardCharsets.UTF_8)))
+    }
+
+    @Test
+    fun `isProbablyText rejects a NUL byte and a control-byte-heavy body`() {
+        assertFalse(BodyPreview.isProbablyText(byteArrayOf(0x41, 0x00, 0x42)))
+        // A run of non-whitespace C0 control bytes exceeds the ratio cutoff.
+        assertFalse(BodyPreview.isProbablyText(byteArrayOf(0x01, 0x02, 0x03, 0x04, 0x05)))
+    }
+}