inspector,http: support builtin http request bodies

[GrinZero]

Summary

This PR adds builtin http/https request-body support to network inspection, so Network.getRequestPostData can return text request bodies while preserving the existing rejection behavior for binary request bodies.

It also moves builtin http response-body tracking to a raw-byte hook before IncomingMessage decoding, so response inspection remains correct even when user code calls response.setEncoding(...).

In addition, this PR extends requestWillBeSent to accept initiator data from JS diagnostics payloads and validates structured initiator.stack objects against the inspector protocol schema before forwarding them to DevTools.

Problem

Builtin http/https network inspection currently emits:

• Network.requestWillBeSent
• Network.responseReceived
• Network.loadingFinished

However, the builtin http client path does not currently expose request-body data to the inspector. As a result, Network.getRequestPostData cannot return POST data for builtin http/https requests.

There are two related gaps as well:

• IncomingMessage 'data' events are not a stable source of raw bytes. If user code calls response.setEncoding('utf8'), the chunks observed by userland become strings, while the inspector protocol expects byte-oriented payloads.
• requestWillBeSent needs to accept JS-provided initiator data, but structured stack trace input must be validated before it is forwarded as protocol data.

Approach

1. Reuse the existing request buffering pipeline

This change does not alter the CDP schema or the existing buffering behavior in NetworkAgent.

Instead, it reuses the current pipeline already used by other transports:

Network.dataSent(...) -> NetworkAgent::getRequestPostData(...)

2. Add builtin http request-body diagnostics events

The builtin http client now publishes:

• http.client.request.bodyChunkSent
• http.client.request.bodySent

These events are emitted from the ClientRequest write path before HTTP framing is applied, so the inspector sees the original user payload rather than chunked-transfer framing bytes.

3. Capture builtin http response bytes before decoding

For responses, this PR avoids relying on IncomingMessage.on('data') in network_http.js.

Instead, it adds:

• http.client.response.bodyChunkReceived

This hook runs before Readable.setEncoding() transforms chunks for userland, so the inspector always receives raw bytes.

4. Support and validate network initiator payloads

Network.requestWillBeSent can now use JS diagnostics payloads to provide initiator data.

On the C++ side, the incoming JS object is converted into inspector protocol values, and initiator.stack is validated against the Runtime.StackTrace schema before being attached to the event. If the initiator payload is malformed, the event is rejected with a clear error instead of forwarding bad protocol data to the frontend.

If JS does not provide an initiator, the existing behavior remains unchanged: the inspector captures the current stack trace and emits a default script initiator.

5. Document the new diagnostics channels

This PR also updates diagnostics_channel docs to cover the new builtin HTTP client request/response body channels and their message shapes.

Behavior

After this change:

• builtin http and https POST requests with UTF-8 text bodies are available through Network.getRequestPostData
• binary request bodies still reject with the existing inspector error behavior
• builtin http response inspection continues to work even if user code calls response.setEncoding('utf8')
• Network.requestWillBeSent can carry JS-provided initiator stack data
• malformed initiator stack input is rejected by C++ schema validation

Tests

This PR adds and extends coverage in:

• test/parallel/test-diagnostics-channel-http.js
• test/parallel/test-inspector-network-http.js
• test/parallel/test-inspector-network-arbitrary-data.js
• test/parallel/test-inspector-emit-protocol-event-errors.js

The updated tests cover:

• request body chunk and request body finished diagnostics events
• response body raw-byte diagnostics events
• text request bodies split across write() and end()
• binary request bodies
• http and https Network.getRequestPostData
• binary request-body rejection semantics
• response inspection when the client calls response.setEncoding('utf8')
• JS-provided initiator stack parsing and forwarding
• malformed initiator stack error handling

Verification

Verified locally with:

python3 tools/test.py \
  parallel/test-diagnostics-channel-http \
  parallel/test-inspector-network-http \
  parallel/test-inspector-network-arbitrary-data \
  parallel/test-inspector-emit-protocol-event-errors

Refs

• Tracking issue: Stabilizing network inspection in Node.js #53946

[nodejs-github-bot]

Review requested:

• @nodejs/http
• @nodejs/inspector
• @nodejs/net

[GrinZero]

E2E Test:

const http = require("http");

const DEFAULT_TARGET_URL = "http://jsonplaceholder.typicode.com/posts";
const DEFAULT_PORT = Number(process.env.PORT || 3000);
const DEFAULT_HOST = "127.0.0.1";
const targetUrl = process.argv[2] || process.env.TARGET_URL || DEFAULT_TARGET_URL;
const defaultPayload = {
  title: "node inspect demo",
  body: "post payload from local trigger service",
  userId: 123,
};

if (!targetUrl.startsWith("http://")) {
  console.error(
    `[config] This script only uses the http module for outbound requests. Use an http:// URL, got: ${targetUrl}`
  );
  process.exit(1);
}

function readRequestBody(req) {
  return new Promise((resolve, reject) => {
    let body = "";

    req.setEncoding("utf8");
    req.on("data", (chunk) => {
      body += chunk;
    });
    req.on("end", () => {
      resolve(body);
    });
    req.on("error", reject);
  });
}

function buildPayload(rawBody) {
  if (!rawBody || !rawBody.trim()) {
    return defaultPayload;
  }

  const parsed = JSON.parse(rawBody);
  if (!parsed || Array.isArray(parsed) || typeof parsed !== "object") {
    throw new Error("payload must be a JSON object");
  }

  return {
    ...defaultPayload,
    ...parsed,
  };
}

function sendOutboundPost(url, payload) {
  return new Promise((resolve, reject) => {
    const body = JSON.stringify(payload);

    console.log(`\n[outbound] request -> POST ${url}`);
    console.log(`[outbound] request payload: ${body}`);

    const outboundReq = http.request(
      url,
      {
        method: "POST",
        headers: {
          "user-agent": "node-inspect-http-demo",
          accept: "application/json,text/plain,*/*",
          "content-type": "application/json; charset=utf-8",
          "content-length": Buffer.byteLength(body),
        },
      },
      (outboundRes) => {
        let responseBody = "";
        outboundRes.setEncoding("utf8");

        console.log(
          `[outbound] response <- ${outboundRes.statusCode} ${outboundRes.statusMessage || ""}`.trim()
        );
        console.log("[outbound] response headers:", outboundRes.headers);

        outboundRes.on("data", (chunk) => {
          responseBody += chunk;
        });

        outboundRes.on("end", () => {
          console.log("[outbound] body preview:");
          console.log(responseBody.slice(0, 300) || "<empty>");

          resolve({
            statusCode: outboundRes.statusCode,
            statusMessage: outboundRes.statusMessage,
            headers: outboundRes.headers,
            bodyPreview: responseBody.slice(0, 300),
          });
        });
      }
    );

    outboundReq.on("socket", (socket) => {
      socket.on("connect", () => {
        console.log(
          `[outbound] socket connected -> ${socket.remoteAddress}:${socket.remotePort}`
        );
      });
    });

    outboundReq.on("finish", () => {
      console.log("[outbound] request finished");
    });

    outboundReq.on("error", (error) => {
      console.error("[outbound] request error:", error.message);
      reject(error);
    });

    outboundReq.write(body);
    outboundReq.end();
  });
}

function writeJson(res, statusCode, payload) {
  const body = JSON.stringify(payload, null, 2);
  res.writeHead(statusCode, {
    "content-type": "application/json; charset=utf-8",
    "content-length": Buffer.byteLength(body),
  });
  res.end(body);
}

const server = http.createServer(async (req, res) => {
  const url = new URL(req.url || "/", `http://${req.headers.host || "127.0.0.1"}`);

  if (req.method === "GET" && url.pathname === "/") {
    return writeJson(res, 200, {
      ok: true,
      message: "local trigger service is running",
      endpoints: {
        trigger: "POST /trigger",
      },
      targetUrl,
      defaultPayload,
    });
  }

  if (req.method === "POST" && url.pathname === "/trigger") {
    console.log(`\n[inbound] trigger <- ${req.method} ${url.pathname}`);

    try {
      const rawBody = await readRequestBody(req);
      const payload = buildPayload(rawBody);
      const outbound = await sendOutboundPost(targetUrl, payload);

      return writeJson(res, 200, {
        ok: true,
        targetUrl,
        payload,
        outbound,
      });
    } catch (error) {
      return writeJson(res, 500, {
        ok: false,
        error: error.message,
      });
    }
  }

  return writeJson(res, 404, {
    ok: false,
    error: "not found",
  });
});

server.listen(DEFAULT_PORT, DEFAULT_HOST, () => {
  console.log(`[server] listening on http://${DEFAULT_HOST}:${DEFAULT_PORT}`);
  console.log(`[server] outbound target: ${targetUrl}`);
  console.log("[usage] node --inspect-wait --experimental-network-inspection inspect-http.js");
  console.log(
    `[usage] curl -X POST http://${DEFAULT_HOST}:${DEFAULT_PORT}/trigger -H 'content-type: application/json' -d '{"title":"manual trigger"}'`
  );
});

[metcoder95]

lgtm, it just seems that is gonna need a documentation for the new dcs

[GrinZero]

Thanks for the feedback. I’ve updated the documentation in doc/api/diagnostics_channel.md to address the review comments and include the new HTTP diagnostics channel events.

Specifically, I added/updated entries for:

• http.client.request.bodyChunkSent
• http.client.request.bodySent
• http.client.response.bodyChunkReceived

I also aligned the wording and structure with the existing Node.js docs style for built-in channels.

[GrinZero]

A slight push, hoping that a maintainer will notice.

[GrinZero]

Sorry for the ping, @legendecas

If you have some time, could you please help review this PR and trigger CI?

This PR fixes an issue where the network inspector does not show postData for http / https requests.

Thanks a lot!

[GrinZero]

push, request-ci plz

[marco-ippolito]

Ill try to review the PR later, I think it's important we make sure it doesn't leak memory or keep the body buffer reference alive, which can exhaust resource quickly @ShogunPanda ptal

[GrinZero]

Ill try to review the PR later, I think it's important we make sure it doesn't leak memory or keep the body buffer reference alive, which can exhaust resource quickly @ShogunPanda ptal

I'll go explore the issue you mentioned

[GrinZero]

Thanks for raising this concern.

I did a focused memory/lifetime check for this PR, mainly around two questions:

1. whether the inspector keeps the original userland request/response body buffers alive;
2. whether the new body buffering path can grow without bound.

Based on the current implementation and local testing, I do not see evidence that this PR keeps the original JS buffers alive.

For both request and response bodies, the payload bytes are copied into inspector-owned storage before being retained by NetworkAgent. I also verified this with a WeakRef/GC test: the original 4 MiB request body buffer and the observed response chunk both became collectible after the request completed, while Network.getRequestPostData() / Network.getResponseBody() could still return the payloads.

So the inspector is retaining copied payload bytes, not the original userland Buffer objects.

I also ran a repeated-request memory test with explicit Network.enable() limits. Memory increased initially, as expected from inspector-side buffering, but then reached a plateau and older requests were evicted. That behavior looks consistent with bounded retention by the existing request buffer manager, not an unbounded leak.

[GrinZero]

The test plan above was proposed by AI, and I reviewed it.

For the first test, it uses WeakRef, manual GC, and process.memoryUsage() to verify object lifetime. I think the underlying testing principle is reasonably credible.

The second test is a stress test. It lowers maxTotalBufferSize and maxResourceBufferSize, then sends multiple requests with relatively large bodies. I’m less familiar with this approach.

[GrinZero]

I merged the main branch code to avoid CI errors.

[ShogunPanda]

LGTM!

[GrinZero]

LGTM!

Thanks 🙏 I think we should add the request-ci tag to trigger the next steps. I have synced the code of the main branch

[codecov]

Codecov Report

❌ Patch coverage is 99.21875% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.68%. Comparing base (b2f6aa3) to head (8d24bf6).
⚠️ Report is 27 commits behind head on main.

Files with missing lines	Patch %	Lines
src/inspector/network_agent.cc	98.00%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #62915      +/-   ##
==========================================
+ Coverage   89.65%   89.68%   +0.03%     
==========================================
  Files         712      712              
  Lines      220512   221491     +979     
  Branches    42289    42453     +164     
==========================================
+ Hits       197690   198637     +947     
- Misses      14663    14692      +29     
- Partials     8159     8162       +3     

Files with missing lines	Coverage Δ
lib/_http_client.js	97.44% <100.00%> (+<0.01%)	⬆️
lib/_http_common.js	100.00% <100.00%> (ø)
lib/_http_outgoing.js	95.88% <100.00%> (+0.08%)	⬆️
lib/internal/http.js	92.14% <100.00%> (+0.02%)	⬆️
lib/internal/inspector/network_http.js	97.34% <100.00%> (+1.66%)	⬆️
src/inspector/network_agent.cc	89.77% <98.00%> (+1.74%)	⬆️

... and 83 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.