Fix decode HTTP content-encoding

[FreuMi]

Hi everyone!

Problem:
When consuming endpoints that return compressed payloads, the HTTP binding currently forwards the compressed bytes directly to the codecs. This causes parsing/validation errors since the codecs expect plain data.

For example, this public datetime API:

https://aisenseapi.com/services/v1/datetime

responds with:

Content-Type: application/json
Content-Encoding: gzip, br

With the current HTTP binding, the JSON codec receives compressed bytes and fails with errors such as:

Invalid value according to DataSchema

Here is the TD I used to reproduce the issue:

{
  "@context": "https://www.w3.org/2022/wot/td/v1.1",
  "title": "dateTimeAPI",
  "securityDefinitions": { "nosec_sc": { "scheme": "nosec" } },
  "security": ["nosec_sc"],
  "properties": {
    "currentDateTime": {
      "description": "Get current datetime",
      "readOnly": true,
      "type": "object",
      "properties": {
        "datetime": { "type": "string", "format": "date-time" }
      },
      "forms": [
        {
          "href": "https://aisenseapi.com/services/v1/datetime",
          "contentType": "application/json"
        }
      ]
    }
  }
}

Fix:
The HTTP client now checks the content-encoding response header and transparently decompresses the payload before passing it to the codecs. I added a helper in http-client-impl.ts to handle this logic when a response is compressed.

I thought this fix could be useful for others too.

[FreuMi]

So I found out that node-fetch is supposed to handle gzip. Therefore i am not sure why it does not work with my example API.

[danielpeintner]

General comment: IF we need to tackle compression separately, I wonder whether such a generic method like decodeByContentEncoding(...) can be placed as a util function...

https://github.com/FreuMi/node-wot/blob/beeff0407aff6624eb2d6848727c7be95e217214/packages/binding-http/src/http-client-impl.ts#L481

[danielpeintner]

So I found out that node-fetch is supposed to handle gzip. Therefore i am not sure why it does not work with my example API.

@relu91 do you know more?

[relu91]

I reviewed the issue. Here are my findings:

• It should definitely be handled by node-fetch, but we have a quite old dependency
• Even with the latest node-fetch, there is a bug where they basically don't handle multiple content-encodings (even though it is allowed by the RFC)
• The now stable fetch node implementation works correctly

Therefore, I'd keep the issue, but I would fix it by upgrading the HTTP client implementation with the new Node.js native fetch api (which is also cross-platform). However, for node 20, we should set the flag --experimental-fetch since it was marked as experimental in that release. Given that node 20 is sunsetting (is going to be around till April 2026), I think this might be a good comprise.

[FreuMi]

Thanks @relu91 for the clarifications!
I agree it’s better to upgrade in the future rather than adding a custom workaround, since that would only duplicate logic that should already be handled by the HTTP client.

If you want, I can close this PR and open an issue instead.

[relu91]

@danielpeintner are you ok to close this PR and open an issue?

[danielpeintner]

@danielpeintner are you ok to close this PR and open an issue?

If I understand the proposal correctly, we wait for May 2026 (Node.js no longer maintained) and switch to native fetch
OR
we change already now to native fetch and use the experimental flag?

Either way is fine by me.
Closing this PR and creating the issue is definitely fine!

[relu91]

If I understand the proposal correctly, we wait for May 2026 (Node.js no longer maintained) and switch to native fetch
OR
we change already now to native fetch and use the experimental flag?

Yes, it depends on user pressure. I tend to prefer fixing it now, but waiting is also ok.

Either way is fine by me.
Closing this PR and creating the issue is definitely fine!

Ok then, let's close this.

[relu91]

If you want, I can close this PR and open an issue instead.

@FreuMi if you don't mind, can you create the issue?