Skip to content

Commit 075421f

Browse files
chaokunyangchaokunyang
authored
feat(java): remove sun.misc.Unsafe for jdk25+ (#3702)
## Why? JDK25+ removes the remaining viable `sun.misc.Unsafe` paths Fory used for Java object creation, field access, string internals, and memory operations. This PR makes the Java runtime continue to support modern JDKs while preserving the existing JDK8-24, Android, GraalVM, native-mode, and xlang behavior where applicable. ## What does this PR do? - Builds `fory-core` as a multi-release jar with JDK25-specific classes and module metadata, while keeping the root implementation for JDK8-24. - Replaces the JDK25+ runtime paths for field access, object instantiation, string internals, memory access, and generated serializers with trusted lookup, VarHandle, ReflectionFactory, and other non-Unsafe owners. - Updates Java serializers, codegen, copy paths, collection/map serializers, object stream handling, and default-value support to use the new accessor and instantiator ownership model. - Extends CI and integration coverage for JDK25/JDK26, JPMS zero-Unsafe runs, multi-release jar verification, GraalVM packaging, and cross-JDK compatibility expectations. - Updates Java, Kotlin, Scala, and top-level docs for JDK25+ setup, including the required `java.base/java.lang.invoke` open for classpath and module-path users. - Adds Java25 benchmark modules and MR-JAR benchmark verification for direct memory and direct-to-heap copy paths. ## Related issues Closes #3729 Closes #2090 Closes #2606 ## AI Contribution Checklist - [ ] Substantial AI assistance was used in this PR: `yes` / `no` - [ ] If `yes`, I included a completed [AI Contribution Checklist](https://github.com/apache/fory/blob/main/AI_POLICY.md#9-contributor-checklist-for-ai-assisted-prs) in this PR description and the required `AI Usage Disclosure`. - [ ] If `yes`, my PR description includes the required `ai_review` summary and screenshot evidence of the final clean AI review results from both fresh reviewers on the current PR diff or current HEAD after the latest code changes. ## Does this PR introduce any user-facing change? Yes. Java users on JDK25+ must open `java.base/java.lang.invoke` to Fory: - Classpath: `--add-opens=java.base/java.lang.invoke=ALL-UNNAMED` - Module path: `--add-opens=java.base/java.lang.invoke=org.apache.fory.core` No binary protocol compatibility change is intended. - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark This PR adds Java25 benchmark coverage under `benchmarks/java25` and a benchmark MR-JAR check. No benchmark result table is included here.
1 parent 33b0d6e commit 075421f

215 files changed

Lines changed: 17706 additions & 4480 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/languages/java.md‎

Lines changed: 202 additions & 1 deletion
Large diffs are not rendered by default.

‎.agents/languages/kotlin.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,14 @@ Load this file when changing `kotlin/`.
66

77
- Run Kotlin Maven commands from within `kotlin/`.
88
- Kotlin serializers build on the Java implementation. If Java changed and the updated Java artifacts are not installed yet, run `cd ../java && mvn -T16 install -DskipTests` first.
9+
- KSP `@ForyStruct` serializers that use a primary constructor map constructor parameters to
10+
same-named source properties at generation time and call the constructor directly. Do not restore
11+
`@ForyConstructor`, runtime constructor registration, or Kotlin `javaParameters` dependencies;
12+
mutable no-argument structs should use `var` properties with `@ForyField`.
13+
- Preserve serializer-family selection for Kotlin standard-library types already registered by
14+
Fory. Do not auto-install a new serializer for an existing type-registered Kotlin class unless the
15+
wire format matches the previous serializer family and old-payload/new-runtime compatibility is
16+
tested.
917

1018
## Commands
1119

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ jobs:
169169
MY_VAR: "PATH"
170170
strategy:
171171
matrix:
172-
java-version: ${{ fromJSON(needs.changes.outputs.java_code == 'true' && '["8","11","17","21","25"]' || '["8"]') }}
172+
java-version: ${{ fromJSON(needs.changes.outputs.java_code == 'true' && '["8","11","17","21","25","26"]' || '["8"]') }}
173173
steps:
174174
- uses: actions/checkout@v5
175175
- name: Set up JDK ${{ matrix.java-version }}

‎.github/workflows/release-java-snapshot.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,15 +32,15 @@ jobs:
3232
- name: Set up Maven Central Repository
3333
uses: actions/setup-java@v4
3434
with:
35-
java-version: "11"
36-
distribution: "adopt"
35+
java-version: "25"
36+
distribution: "temurin"
3737
architecture: x64
3838
cache: maven
3939
server-id: apache.snapshots.https
4040
server-username: NEXUS_USERNAME
4141
server-password: NEXUS_PASSWORD
4242
- name: Publish Fory Java Snapshot
43-
run: python ./ci/run_ci.py java --version 11 --release
43+
run: python ./ci/run_ci.py java --version 25 --release
4444
env:
4545
NEXUS_USERNAME: ${{ secrets.NEXUS_USER }}
4646
NEXUS_PASSWORD: ${{ secrets.NEXUS_PW }}

‎README.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,19 @@ Gradle:
142142
implementation "org.apache.fory:fory-core:1.1.0"
143143
```
144144

145+
On JDK25+, open `java.lang.invoke` to Fory. Use `ALL-UNNAMED` when Fory is on
146+
the classpath:
147+
148+
```bash
149+
--add-opens=java.base/java.lang.invoke=ALL-UNNAMED
150+
```
151+
152+
Use the Fory core module name when Fory is on the module path:
153+
154+
```bash
155+
--add-opens=java.base/java.lang.invoke=org.apache.fory.core
156+
```
157+
145158
**Scala**
146159

147160
sbt:

‎benchmarks/java/pom.xml‎

Lines changed: 147 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -212,13 +212,6 @@
212212
</plugin>
213213
</plugins>
214214
</build>
215-
<dependencies>
216-
<dependency>
217-
<groupId>org.apache.fory</groupId>
218-
<artifactId>fory-simd</artifactId>
219-
<version>${project.version}</version>
220-
</dependency>
221-
</dependencies>
222215
</profile>
223216
<profile>
224217
<id>jmh</id>
@@ -236,6 +229,136 @@
236229
</dependency>
237230
</dependencies>
238231
</profile>
232+
<profile>
233+
<id>jdk25-benchmark-mrjar-check</id>
234+
<activation>
235+
<jdk>[25,)</jdk>
236+
</activation>
237+
<build>
238+
<plugins>
239+
<plugin>
240+
<groupId>org.apache.maven.plugins</groupId>
241+
<artifactId>maven-antrun-plugin</artifactId>
242+
<executions>
243+
<execution>
244+
<id>verify-benchmark-mrjar</id>
245+
<phase>package</phase>
246+
<goals>
247+
<goal>run</goal>
248+
</goals>
249+
<configuration>
250+
<target>
251+
<property
252+
name="jdk25.benchmark.check.dir"
253+
value="${project.build.directory}/jdk25-benchmark-check"/>
254+
<delete dir="${jdk25.benchmark.check.dir}"/>
255+
<mkdir dir="${jdk25.benchmark.check.dir}"/>
256+
<unzip
257+
src="${project.build.directory}/${uberjar.name}.jar"
258+
dest="${jdk25.benchmark.check.dir}">
259+
<patternset>
260+
<include
261+
name="org/apache/fory/platform/UnsafeOps.class"/>
262+
<include
263+
name="META-INF/versions/25/org/apache/fory/platform/UnsafeOps.class"/>
264+
<include
265+
name="META-INF/versions/25/org/apache/fory/memory/LittleEndian.class"/>
266+
<include
267+
name="META-INF/versions/25/org/apache/fory/memory/MemoryBuffer.class"/>
268+
<include
269+
name="META-INF/versions/25/org/apache/fory/platform/internal/_UnsafeUtils.class"/>
270+
<include
271+
name="META-INF/versions/25/org/apache/fory/reflect/InstanceFieldAccessors.class"/>
272+
<include
273+
name="META-INF/versions/25/org/apache/fory/serializer/PlatformStringUtils.class"/>
274+
</patternset>
275+
</unzip>
276+
<available
277+
file="${jdk25.benchmark.check.dir}/org/apache/fory/platform/UnsafeOps.class"
278+
property="jdk25.benchmark.rootunsafeops.present"/>
279+
<available
280+
file="${jdk25.benchmark.check.dir}/META-INF/versions/25/org/apache/fory/platform/UnsafeOps.class"
281+
property="jdk25.benchmark.unsafeops.present"/>
282+
<available
283+
file="${jdk25.benchmark.check.dir}/META-INF/versions/25/org/apache/fory/memory/LittleEndian.class"
284+
property="jdk25.benchmark.littleendian.present"/>
285+
<available
286+
file="${jdk25.benchmark.check.dir}/META-INF/versions/25/org/apache/fory/memory/MemoryBuffer.class"
287+
property="jdk25.benchmark.memorybuffer.present"/>
288+
<available
289+
file="${jdk25.benchmark.check.dir}/META-INF/versions/25/org/apache/fory/platform/internal/_UnsafeUtils.class"
290+
property="jdk25.benchmark.unsafeutils.present"/>
291+
<available
292+
file="${jdk25.benchmark.check.dir}/META-INF/versions/25/org/apache/fory/reflect/InstanceFieldAccessors.class"
293+
property="jdk25.benchmark.instancefieldaccessors.present"/>
294+
<available
295+
file="${jdk25.benchmark.check.dir}/META-INF/versions/25/org/apache/fory/serializer/PlatformStringUtils.class"
296+
property="jdk25.benchmark.platformstring.present"/>
297+
<fail
298+
if="jdk25.benchmark.rootunsafeops.present"
299+
message="JDK25 benchmark jar must not contain root UnsafeOps class."/>
300+
<fail
301+
if="jdk25.benchmark.unsafeops.present"
302+
message="JDK25 benchmark jar must not contain versioned UnsafeOps class."/>
303+
<fail
304+
unless="jdk25.benchmark.littleendian.present"
305+
message="JDK25 benchmark jar is missing the versioned LittleEndian class."/>
306+
<fail
307+
unless="jdk25.benchmark.memorybuffer.present"
308+
message="JDK25 benchmark jar is missing the versioned MemoryBuffer class."/>
309+
<fail
310+
unless="jdk25.benchmark.unsafeutils.present"
311+
message="JDK25 benchmark jar is missing the versioned _UnsafeUtils class."/>
312+
<fail
313+
unless="jdk25.benchmark.instancefieldaccessors.present"
314+
message="JDK25 benchmark jar is missing the versioned InstanceFieldAccessors class."/>
315+
<fail
316+
unless="jdk25.benchmark.platformstring.present"
317+
message="JDK25 benchmark jar is missing the versioned PlatformStringUtils class."/>
318+
<exec
319+
executable="${java.home}/bin/jdeps"
320+
failonerror="true"
321+
outputproperty="jdk25.benchmark.jdeps.output">
322+
<arg value="--multi-release"/>
323+
<arg value="25"/>
324+
<arg value="--ignore-missing-deps"/>
325+
<arg value="-include"/>
326+
<arg value="org\.apache\.fory\..*"/>
327+
<arg value="-verbose:class"/>
328+
<arg value="${project.build.directory}/${uberjar.name}.jar"/>
329+
</exec>
330+
<condition property="jdk25.benchmark.jdeps.sunmisc.present">
331+
<contains string="${jdk25.benchmark.jdeps.output}" substring="-&gt; sun.misc"/>
332+
</condition>
333+
<condition property="jdk25.benchmark.jdeps.jdkinternalreflect.present">
334+
<contains
335+
string="${jdk25.benchmark.jdeps.output}"
336+
substring="-&gt; jdk.internal.reflect"/>
337+
</condition>
338+
<fail
339+
if="jdk25.benchmark.jdeps.sunmisc.present"
340+
message="JDK25 benchmark jar must not expose sun.misc dependencies in the resolved class graph."/>
341+
<fail
342+
if="jdk25.benchmark.jdeps.jdkinternalreflect.present"
343+
message="JDK25 benchmark jar must not expose jdk.internal.reflect dependencies in the resolved class graph."/>
344+
<java
345+
classname="org.apache.fory.benchmark.Jdk25MrJarCheck"
346+
fork="true"
347+
failonerror="true">
348+
<classpath>
349+
<pathelement location="${project.build.directory}/${uberjar.name}.jar"/>
350+
</classpath>
351+
<jvmarg value="--add-opens=java.base/java.lang.invoke=ALL-UNNAMED"/>
352+
<jvmarg value="--sun-misc-unsafe-memory-access=deny"/>
353+
</java>
354+
</target>
355+
</configuration>
356+
</execution>
357+
</executions>
358+
</plugin>
359+
</plugins>
360+
</build>
361+
</profile>
239362
</profiles>
240363

241364
<build>
@@ -266,6 +389,7 @@
266389
<configuration>
267390
<archive>
268391
<manifestEntries>
392+
<Multi-Release>true</Multi-Release>
269393
<Automatic-Module-Name>org.apache.fory.benchmark</Automatic-Module-Name>
270394
</manifestEntries>
271395
</archive>
@@ -287,6 +411,10 @@
287411
<transformer
288412
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
289413
<mainClass>org.openjdk.jmh.Main</mainClass>
414+
<manifestEntries>
415+
<Multi-Release>true</Multi-Release>
416+
<Automatic-Module-Name>org.apache.fory.benchmark</Automatic-Module-Name>
417+
</manifestEntries>
290418
</transformer>
291419
</transformers>
292420
<filters>
@@ -298,6 +426,18 @@
298426
<exclude>META-INF/*.RSA</exclude>
299427
</excludes>
300428
</filter>
429+
<filter>
430+
<artifact>org.apache.logging.log4j:*</artifact>
431+
<excludes>
432+
<exclude>META-INF/versions/**</exclude>
433+
</excludes>
434+
</filter>
435+
<filter>
436+
<artifact>com.fasterxml.jackson.core:jackson-core</artifact>
437+
<excludes>
438+
<exclude>META-INF/versions/**</exclude>
439+
</excludes>
440+
</filter>
301441
</filters>
302442
</configuration>
303443
</execution>

‎benchmarks/java/src/main/java/org/apache/fory/benchmark/CompressStringSuite.java‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121

2222
import java.nio.ByteBuffer;
2323
import org.apache.fory.memory.MemoryBuffer;
24+
import org.apache.fory.serializer.StringEncodingUtils;
2425
import org.apache.fory.util.StringUtils;
2526
import org.openjdk.jmh.Main;
2627
import org.openjdk.jmh.annotations.Benchmark;
@@ -99,7 +100,7 @@ public Object latinScalarCheck() {
99100

100101
@Benchmark
101102
public Object latinSuperWordCheck() {
102-
return StringUtils.isLatin(latinStrChars);
103+
return StringEncodingUtils.isLatin(latinStrChars);
103104
}
104105

105106
public static void main(String[] args) throws Exception {

‎benchmarks/java/src/main/java/org/apache/fory/benchmark/Identity2IdMap.java‎

Lines changed: 2 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@
1818

1919
import java.util.ArrayList;
2020
import java.util.List;
21-
import org.apache.fory.platform.UnsafeOps;
2221

2322
// Derived from
2423
// https://github.com/RuedigerMoeller/fast-serialization/blob/e8da5591daa09452791dcd992ea4f83b20937be7/src/main/java/org/nustaq/serialization/util/FSTIdentity2IdMap.java.
@@ -405,20 +404,10 @@ public static void clear(int[] arr, int len) {
405404
int count = 0;
406405
final int emptyArrayLength = EMPTY_INT_ARRAY.length;
407406
while (len - count > emptyArrayLength) {
408-
UnsafeOps.copyMemory(
409-
EMPTY_INT_ARRAY,
410-
UnsafeOps.INT_ARRAY_OFFSET,
411-
arr,
412-
UnsafeOps.INT_ARRAY_OFFSET + count,
413-
emptyArrayLength);
407+
System.arraycopy(EMPTY_INT_ARRAY, 0, arr, count, emptyArrayLength);
414408
count += emptyArrayLength;
415409
}
416-
UnsafeOps.copyMemory(
417-
EMPTY_INT_ARRAY,
418-
UnsafeOps.INT_ARRAY_OFFSET,
419-
arr,
420-
UnsafeOps.INT_ARRAY_OFFSET + count,
421-
len - count);
410+
System.arraycopy(EMPTY_INT_ARRAY, 0, arr, count, len - count);
422411
}
423412

424413
public static void clear(Object[] arr, int len) {

0 commit comments

Comments
 (0)