Skip to content

[fix] Upgrade to AsyncHTTP client 3.0.4#25023

Open
merlimat wants to merge 12 commits intoapache:masterfrom
merlimat:async-http-3
Open

[fix] Upgrade to AsyncHTTP client 3.0.4#25023
merlimat wants to merge 12 commits intoapache:masterfrom
merlimat:async-http-3

Conversation

@merlimat
Copy link
Copy Markdown
Contributor

@merlimat merlimat commented Nov 26, 2025 •
edited by lhotari
Loading

Motivation

Fix for CVE-2024-53990

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@merlimat merlimat added this to the 4.2.0 milestone Nov 26, 2025
@merlimat merlimat requested a review from lhotari November 26, 2025 20:03
@merlimat merlimat self-assigned this Nov 26, 2025
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Nov 26, 2025
@lhotari
Copy link
Copy Markdown
Member

lhotari commented Nov 27, 2025

Regarding CVE-2024-53990, it's also fixed in 2.12.4.

@lhotari lhotari mentioned this pull request Mar 23, 2026
Closed
@nodece
Copy link
Copy Markdown
Member

nodece commented Apr 7, 2026

Please use version 3.0.8.

merlimat added a commit to merlimat/pulsar that referenced this pull request Apr 14, 2026
@merlimat
[improve][build] Upgrade Netty from 4.1.132.Final to 4.2.12.Final
d92305b 
This upgrade brings Pulsar onto the Netty 4.2 line in preparation for
the async-http-client 3.x upgrade (apache#25023), which transitively depends
on Netty 4.2 and cannot be landed while Pulsar force-pins Netty 4.1.

Netty 4.1 and 4.2 cannot co-exist on the classpath (same io.netty.*
package namespace), so the upgrade has to be done in a single step.
The Netty team asserts source/binary forward compatibility from 4.1
to 4.2 for regular API users:

  https://netty.io/news/2025/04/03/4-2-0.html
  https://github.com/netty/netty/wiki/Netty-4.2-Migration-Guide

Changes in this PR:

* gradle/libs.versions.toml:
  - Bump netty from 4.1.132.Final to 4.2.12.Final.
  - Drop the separate netty-iouring version (0.0.26.Final).
    io_uring has graduated from incubator
    (io.netty.incubator:netty-incubator-transport-*-io_uring) to a
    first-class Netty artifact
    (io.netty:netty-transport-{classes,native}-io_uring), now pinned
    to the same Netty version.

* pulsar-common/build.gradle.kts: Point the io_uring consumer at the
  renamed aliases.

* pulsar-common/.../EventLoopUtil.java: Netty 4.2 removed the
  dedicated IOUringEventLoopGroup class. io_uring now uses the generic
  MultiThreadIoEventLoopGroup + IoUringIoHandler factory pattern,
  which makes io_uring groups indistinguishable from any other
  MultiThreadIoEventLoopGroup by type, breaking the existing
  instanceof-based channel class dispatch. Fix: introduce a private
  marker subclass IoUringMultiThreadIoEventLoopGroup used at
  construction. Also repoint the incubator imports
  (io.netty.incubator.channel.uring.*) to the core package
  (io.netty.channel.uring.*) and adjust class names (IOUring ->
  IoUring).

* build-logic/conventions/.../pulsar.java-conventions.gradle.kts:
  Exclude io.netty.incubator from all configurations. BookKeeper 4.17.3
  (bookkeeper-common and stream-storage-java-client) still declares a
  transitive dependency on the 0.0.26.Final incubator io_uring jars,
  which are compiled against Netty 4.1 internals and are not safe to
  leave on the 4.2 classpath. Pulsar uses the core io_uring API via
  EventLoopUtil; BK stream-storage is an optional feature that Pulsar
  does not expose in its default surface.

* distribution/{server,shell}/src/assemble/LICENSE.bin.txt: Reflect
  the actual Netty jar set shipped after the upgrade:
  - Bump all 4.1.132.Final entries to 4.2.12.Final.
  - Replace the monolithic netty-codec-*.jar with its 4.2 split-out
    sub-modules netty-codec-base and netty-codec-compression
    (netty-codec is now an aggregator POM that ships no classes).
  - Rename the incubator io_uring entries
    (io.netty.incubator-netty-incubator-transport-*-io_uring-0.0.26.Final)
    to the core io_uring artifacts
    (io.netty-netty-transport-{classes,native}-io_uring-4.2.12.Final).
  The jar set was cross-checked against the output of
  :distribution:pulsar-server-distribution:serverDistTar and
  :distribution:pulsar-shell-distribution:shellDistTar.

* pulsar-common/.../BitSetRecyclableRecyclableTest and
  ConcurrentBitSetRecyclableTest: Relax the testRecycle assertion.
  Netty 4.2's io.netty.util.Recycler (which is itself deprecated in
  4.2) no longer guarantees same-thread immediate reuse, so we only
  assert functional behavior: any recycled instance must come back
  cleared, and distinct create() calls must return distinct objects.

Verification:

* ./gradlew compileJava compileTestJava: clean across the entire
  project, only deprecation warnings (NioEventLoopGroup,
  EpollEventLoopGroup, DefaultEventLoopGroup,
  ChannelOption.RCVBUF_ALLOCATOR, EpollMode, Recycler,
  PlatformDependent.threadLocalRandom). These are compat shims that
  still function in 4.2; cleanup can follow in a separate PR.

* :pulsar-common:test: passes (678 tests).
* :pulsar-broker:test --tests BrokerServiceTest: passes (broker
  startup, producer/consumer flow, Netty transport end-to-end).
* :pulsar-proxy:test --tests ProxyServiceTlsStarterTest: passes
  (proxy, TLS handshake, tcnative-boringssl integration).
* :distribution:pulsar-server-distribution:serverDistTar and
  :distribution:pulsar-shell-distribution:shellDistTar both build,
  and the Netty jar set inside each tarball matches the
  LICENSE.bin.txt files exactly.

Known Netty 4.2 behavior changes that this PR does NOT address:

* The default SslContextBuilder.endpointIdentificationAlgorithm
  changed from null to HTTPS in 4.2. Pulsar's TLS client sites need
  to be audited and explicitly configured. This is intentionally out
  of scope here because the audit touches many modules
  (pulsar-client, pulsar-broker, pulsar-proxy,
  pulsar-broker-auth-oidc, admin) and should be its own PR.

* The default ByteBufAllocator changed from pooled to adaptive in
  4.2. Pulsar is not setting io.netty.allocator.type=pooled in this
  PR; if CI soak tests show regressions, the pooled override can be
  added to the launch scripts as a follow-up.
@merlimat merlimat mentioned this pull request Apr 14, 2026
Draft
@merlimat
Copy link
Copy Markdown
Contributor Author

merlimat commented Apr 23, 2026

Deferring this upgrade until BK 4.18, which will also be using Netty 4.2.x

@dao-jun
Copy link
Copy Markdown
Member

dao-jun commented Apr 24, 2026

Deferring this upgrade until BK 4.18, which will also be using Netty 4.2.x

@merlimat
apache/bookkeeper#4739
apache/bookkeeper#4741

I want the 2 PRs can be released in BK 4.18, could you please take a look?

@merlimat
Copy link
Copy Markdown
Contributor Author

merlimat commented Apr 24, 2026

I want the 2 PRs can be released in BK 4.18, could you please take a look?

Yes, no worries, we're still a few weeks away from BK 4.18. A discussion will be started before starting.

@lhotari lhotari modified the milestones: 4.2.0, 4.3.0 Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nodece nodece nodece approved these changes

@dao-jun dao-jun dao-jun approved these changes

@lhotari lhotari Awaiting requested review from lhotari

Assignees

@merlimat merlimat

Labels

doc-not-needed Your PR changes do not impact docs ready-to-test release/4.0.11 release/4.1.4 release/4.2.2

Projects

None yet

Milestone

4.3.0

Development

Successfully merging this pull request may close these issues.

4 participants

@merlimat @lhotari @nodece @dao-jun