Skip to content

Commit 158d3c2

Browse files
cowtowncodercowtowncoder
committed
Merge branch '3.2' into 3.x
2 parents 1a9f0ce + 7c4694d commit 158d3c2

2 files changed

Lines changed: 247 additions & 0 deletions

File tree

‎CLAUDE.md‎

Lines changed: 243 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,243 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository
4+
on branches of the 3.x line (`3.1`, `3.2`, `3.x`).
5+
6+
## Project
7+
8+
`jackson-core` is the streaming (incremental) parser/generator layer of Jackson, plus the reference
9+
JSON implementation of those abstractions. Maven coordinates are `tools.jackson.core:jackson-core`
10+
and all packages live under `tools.jackson.core` (2.x used `com.fasterxml.jackson.core`) — never
11+
introduce `com.fasterxml.*` into 3.x code.
12+
13+
Its one compile dependency, `fastdoubleparser`, is shaded and relocated into
14+
`tools.jackson.core.internal.shaded.fdp` at package time and stripped from the published POM — so
15+
**consumers see zero external dependencies**. (Unlike 2.x, the relocation path is *not*
16+
version-suffixed: JPMS module encapsulation already prevents reuse by downstream deps.)
17+
18+
Everything else in Jackson (`jackson-databind`, and every data format backend: Smile, CBOR, XML,
19+
CSV, YAML, Protobuf...) builds on the abstractions here. That means **public API changes here ripple
20+
across the whole ecosystem** — treat `JsonParser`, `JsonGenerator`, `TokenStreamFactory`,
21+
`JsonFactory`, and the `*Feature` enums as frozen in patch releases and additive-only in minor ones.
22+
23+
## Build & test
24+
25+
Use the Maven wrapper (`./mvnw`), not a system `mvn`.
26+
27+
```bash
28+
./mvnw verify # full build: compile, test, JaCoCo agent + enforcer checks
29+
./mvnw test # tests only
30+
./mvnw -B -ff -ntp verify # exactly what CI runs (batch, fail-fast, no transfer log)
31+
32+
./mvnw test -Dtest=UTF8StreamJsonParserTest # single test class
33+
./mvnw test -Dtest=UTF8StreamJsonParserTest#testFoo # single test method
34+
./mvnw test -Dtest='*Filtering*' # pattern
35+
36+
./mvnw test jacoco:report # coverage report (separate goal; not part of `verify`)
37+
./mvnw clean package animal-sniffer:check # verify Android SDK 26 API compatibility
38+
```
39+
40+
Note `verify` binds the JaCoCo *agent* and an enforcer rule that requires `jacoco.exec` to exist —
41+
it does not produce the coverage report. CI generates that in a separate `test jacoco:report` step,
42+
and runs `animal-sniffer:check` only on the release-build matrix entry.
43+
44+
Baseline is **JDK 17** (source/target); CI builds on 17, 21, and 25 (plus 17 on Windows). Normal
45+
development works on JDK 17, but *releases* must be built with **JDK 21+** — the enforcer plugin
46+
fails otherwise, because JDK 17 produces an incomplete `-javadoc.jar` (see #1561 / #1625).
47+
`animal-sniffer` additionally restricts you to the Android SDK 26 API subset.
48+
49+
Tests are JUnit 5 (`org.junit.jupiter`) with AssertJ available. `src/test/java/perf/` holds manual
50+
benchmark drivers, not unit tests.
51+
52+
## Branching and release notes
53+
54+
This repo maintains many live branches. Fixes go to the **oldest branch that should receive them**,
55+
then get merged forward:
56+
57+
```
58+
2.21 → 2.22 → 2.x → 3.1 → 3.2 → 3.x
59+
```
60+
61+
`2.x` merges into `3.1`, so 2.x fixes flow into the 3.x line. `3.1` is the current patch branch
62+
(3.1.6-SNAPSHOT), `3.2` the current minor branch (3.2.1-SNAPSHOT), `3.x` the dev branch
63+
(3.3.0-SNAPSHOT). The `3.0` branch still exists (3.0.5-SNAPSHOT) but is dormant — fully merged into
64+
`3.1`, no commits since 3.0.4 shipped, and no unreleased section in `release-notes/VERSION`. Start
65+
from `3.1`, not `3.0`, unless told otherwise.
66+
67+
Don't commit a fix only to `3.x` if it belongs in a patch branch. Propagate with forward merges
68+
(`git merge 3.1` into `3.2`, then `3.2` into `3.x`) rather than cherry-picks — that is the pattern
69+
throughout the history.
70+
71+
Every user-visible change gets an entry in `release-notes/VERSION` under the unreleased version,
72+
formatted as `#<issue>: <summary>` with `(reported by @user)` / `(contributed by @user)` lines, and
73+
a matching name in `release-notes/CREDITS`. (`VERSION-2.x` / `CREDITS-2.x` are the frozen 2.x
74+
history; do not add to them.)
75+
76+
## What changed from 2.x (things to unlearn)
77+
78+
- **`JacksonException` extends `RuntimeException`.** Nothing in the streaming API declares
79+
`throws IOException`; parser/generator methods declare `throws JacksonException`. Low-level I/O
80+
failures are wrapped as `exc/JacksonIOException`.
81+
- **Factories are immutable.** There is no `factory.enable(...)` / `factory.configure(...)`.
82+
Configure through `JsonFactory.builder()` (or `factory.rebuild()` to derive a modified copy).
83+
`JsonFactory.builderWithJackson2Defaults()` moves defaults back toward 2.x, but its own Javadoc
84+
says it is a work in progress and does not yet fully replicate them — don't treat it as exact.
85+
- **`JsonParser.Feature` and `JsonGenerator.Feature` are gone.** Their members were split into
86+
`StreamReadFeature`/`StreamWriteFeature` (format-neutral) and `JsonReadFeature`/`JsonWriteFeature`
87+
(JSON-only). `JsonFactory.Feature` moved up to `TokenStreamFactory.Feature`.
88+
- **Renames**: `JsonLocation` → `TokenStreamLocation`, `JsonStreamContext` → `TokenStreamContext`,
89+
`JsonGeneratorImpl` → `json/JsonGeneratorBase`, `NonBlockingJsonParser` →
90+
`json/async/NonBlockingByteArrayJsonParser`. `nextFieldName()` and friends are `nextName()`.
91+
- **`ObjectCodec` is gone**, replaced by `ObjectReadContext` / `ObjectWriteContext`, which are passed
92+
explicitly to the `createParser(ObjectReadContext, ...)` / `createGenerator(ObjectWriteContext, ...)`
93+
overloads. Tree access goes through `TreeCodec` / `TreeNode` (`tree/` holds minimal
94+
`ArrayTreeNode` / `ObjectTreeNode` impls).
95+
- **`module-info.java` is a real source file** at `src/main/java/module-info.java` (2.x generated one
96+
via moditect). Its `module-info.class` lands at the jar root, not under `META-INF/versions/`.
97+
98+
## Architecture
99+
100+
### The layered type hierarchy
101+
102+
Each of parser, generator, and factory has the same shape: **format-neutral abstract API** →
103+
**shared partial implementation** → **JSON-specific partial implementation** → **concrete class**.
104+
105+
| Layer | Parser | Generator | Factory |
106+
|---|---|---|---|
107+
| Neutral API | `JsonParser` | `JsonGenerator` | `TokenStreamFactory` |
108+
| Partial impl | `base/ParserMinimalBase` → `base/ParserBase` | `base/GeneratorBase` | `base/DecorableTSFactory` → `base/TextualTSFactory` / `base/BinaryTSFactory` |
109+
| JSON partial | `json/JsonParserBase` | `json/JsonGeneratorBase` | — |
110+
| JSON impl | `json/ReaderBasedJsonParser`, `json/UTF8StreamJsonParser`, `json/UTF8DataInputJsonParser` | `json/WriterBasedJsonGenerator`, `json/UTF8JsonGenerator` | `json/JsonFactory` |
111+
112+
`base/TextualTSFactory` and `base/BinaryTSFactory` are the intended superclasses for backends in
113+
other repos (textual formats like YAML/CSV vs binary ones like Smile/CBOR); `JsonFactory` extends
114+
`TextualTSFactory`.
115+
116+
The non-blocking parsers sit under the same `json/JsonParserBase` node:
117+
`NonBlockingJsonParserBase` → `NonBlockingUtf8JsonParserBase` → `NonBlockingByteArrayJsonParser` /
118+
`NonBlockingByteBufferJsonParser`.
119+
120+
Despite the `Json` prefix, only classes in packages containing `json` are JSON-specific. Everything
121+
else is format-neutral and is subclassed by the binary/text format backends in other repos.
122+
123+
`JsonFactory` decides which concrete parser to build based on the input source:
124+
125+
- `byte[]` / `InputStream` → `ByteSourceJsonBootstrapper` sniffs the encoding. UTF-8 gets the
126+
dedicated `UTF8StreamJsonParser`; **other encodings are wrapped in a `Reader`** (`UTF32Reader`, or
127+
an `InputStreamReader`) and handed to `ReaderBasedJsonParser`.
128+
- `Reader` / `String` / `char[]` → `ReaderBasedJsonParser`.
129+
- `DataInput` → `UTF8DataInputJsonParser`.
130+
131+
These are largely parallel implementations of the same state machine over different input
132+
representations — **a bug fixed in one usually needs fixing in all of them**, plus the async parsers.
133+
134+
### Non-blocking parsing
135+
136+
`json/async/NonBlockingByteArrayJsonParser` (and the `ByteBuffer` variant) implement the same token
137+
stream as a fully resumable state machine driven by `async/ByteArrayFeeder` / `ByteBufferFeeder`. It
138+
shares no scanning code with `UTF8StreamJsonParser`, so parser fixes typically must be applied here
139+
separately. `NonBlockingJsonParserBase` holds the shared state/token-id machinery.
140+
141+
### Performance-critical infrastructure
142+
143+
- **`sym/` — symbol tables.** Object property names are canonicalized so repeated names in a document
144+
become the same `String`. `ByteQuadsCanonicalizer` (UTF-8 byte input) and
145+
`CharsToNameCanonicalizer` (char input) live as *root* instances on `JsonFactory`; each parser gets
146+
a **child** via `makeChild()`/`makeChildOrPlaceholder()`, and `release()` merges learned names back
147+
into the root (via `mergeChild()`) when the parser closes. This is where hash-collision DoS
148+
protection lives. Note: `INTERN_PROPERTY_NAMES` is **disabled** by default in 3.x (it was on in 2.x);
149+
`CANONICALIZE_PROPERTY_NAMES` remains on.
150+
- **`util/BufferRecycler`, `util/RecyclerPool`, `util/JsonRecyclerPools`.** Parsers and generators
151+
lease their I/O and text buffers from a pooled `BufferRecycler`, held via `io/IOContext`. Pool
152+
strategy is pluggable per-factory. Failing to release a buffer on an error path is a real leak.
153+
- **`util/TextBuffer`.** Grow-on-demand character accumulator used for decoded string values;
154+
`ReadConstrainedTextBuffer` is the variant that enforces `maxStringLength`.
155+
- **`io/NumberInput`, `io/BigDecimalParser`, `io/BigIntegerParser`, `io/schubfach/`.** Number
156+
decoding/encoding, delegating to shaded `fastdoubleparser` for `double`/`float` and to Schubfach
157+
for shortest-repr output.
158+
159+
### Configuration model
160+
161+
Feature *state* is held as plain `int` bitmask fields on `TokenStreamFactory` — `_factoryFeatures`,
162+
`_streamReadFeatures`, `_streamWriteFeatures`, `_formatReadFeatures`, `_formatWriteFeatures` — all
163+
`final`. `util/JacksonFeatureSet` is a separate, immutable holder used for the **capability** enums,
164+
reached via `streamReadCapabilities()` / `streamWriteCapabilities()`.
165+
166+
There are several distinct axes, and putting a feature on the wrong one is an API mistake that can't
167+
be undone:
168+
169+
- `TokenStreamFactory.Feature` — factory-level, affecting how parsers/generators get constructed
170+
(symbol table interning, canonicalization, hash-collision handling). Implements `JacksonFeature`.
171+
- `StreamReadFeature` / `StreamWriteFeature` — format-neutral, per parser/generator. Implement
172+
`JacksonFeature`.
173+
- `JsonReadFeature` / `JsonWriteFeature` — JSON-only. Unlike 2.x, these are first-class:
174+
they implement `FormatFeature` and own the `_formatReadFeatures` / `_formatWriteFeatures` bitmasks
175+
rather than mapping onto legacy `JsonParser.Feature` constants.
176+
- `StreamReadCapability` / `StreamWriteCapability` — what a backend *can* do, not what it's told to do.
177+
178+
All configuration flows through `TSFBuilder` / `JsonFactoryBuilder`. There is no mutable-setter
179+
fallback.
180+
181+
### Processing limits (security-relevant)
182+
183+
Three per-factory config objects, with quite different scopes — don't conflate them:
184+
185+
- **`StreamReadConstraints`** does the heavy lifting. `validateNestingDepth`, `validateDocumentLength`,
186+
`validateTokenCount`, `validateFPLength`, `validateIntegerLength`, `validateStringLength`
187+
(plus `validateStringLengthLong`), `validateNameLength`, `validateBigIntegerScale`.
188+
- **`StreamWriteConstraints`** bounds exactly one thing: output nesting depth.
189+
- **`ErrorReportConfiguration`** is *not* an input limit — it caps how much content
190+
(`maxErrorTokenLength`, `maxRawContentLength`) gets embedded in exception messages.
191+
192+
Constraint violations throw `exc/StreamConstraintsException`. This module is continuously fuzzed by
193+
OSS-Fuzz; `src/test/java/tools/jackson/core/unittest/fuzz/` and `.../dos/` hold regression tests from
194+
those findings, and `.../constraints/` tests the limits themselves. New parsing code paths that can
195+
accumulate unbounded input must consult the relevant constraint.
196+
197+
### Filtering
198+
199+
`filter/FilteringParserDelegate` and `filter/FilteringGeneratorDelegate` wrap a parser/generator and
200+
apply a `TokenFilter` (commonly `JsonPointerBasedFilter`) to include/exclude subtrees while keeping
201+
the surrounding token stream well-formed. `TokenFilterContext` tracks the deferred "has the parent
202+
START_OBJECT been emitted yet?" bookkeeping that makes this work.
203+
204+
## Code conventions
205+
206+
- Non-public instance/static fields are prefixed with `_` (`_currToken`, `_inputBuffer`). Public API
207+
never exposes fields.
208+
- Every new public method/class carries an `@since 3.NN` Javadoc tag for the version it lands in.
209+
- Long-lived comments are dated and attributed — `// 11-May-2020, tatu: ...` — with a
210+
`[core#1264]` issue reference when one applies. Follow this format when leaving a non-obvious note.
211+
- Tests for a specific GitHub issue are named after it: `GeneratorFiltering890Test`,
212+
`Base64Padding912Test`, `Fuzz34435ParseTest`.
213+
- Tests live under `src/test/java/tools/jackson/core/unittest/`, grouped by feature area (`base64/`,
214+
`filter/`, `json/`, `sym/`...), not by read/write side.
215+
- Most tests extend `JacksonCoreTestBase`, which supplies the `MODE_*` constants
216+
(`MODE_INPUT_STREAM`, `MODE_READER`, `MODE_DATA_INPUT`, throttled variants...). Parser tests should
217+
loop over `ALL_MODES` (or a relevant subset like `ALL_BINARY_MODES` / `ALL_TEXT_MODES`) so every
218+
input backend gets exercised — this is how the parallel-implementation problem above is caught.
219+
Async tests use `AsyncTestBase` (`asyncForBytes(factory, bytesPerRead, data, padding)`).
220+
- `src/test/.../unittest/testutil/` holds fakes (`ThrottledInputStream`, `ThrottledReader`,
221+
`MockDataInput`) that force buffer-boundary splits; use them when a fix concerns content spanning a
222+
buffer edge.
223+
- A known-broken test goes in the `unittest/tofix` package annotated `@JacksonTestFailureExpected`,
224+
which *fails the build if the test starts passing*. Use it to lock in a reproduction before the fix
225+
exists; remove the annotation and move the test when fixing.
226+
227+
## Things that are easy to get wrong
228+
229+
- **New public package** → must be added by hand to `src/main/java/module-info.java` (and the test
230+
counterpart at `src/test/java/module-info.java` if tests need it). The `osgi.export` property in
231+
`pom.xml` uses a `tools.jackson.core.*` wildcard and needs no change, but the module-info lists
232+
every package explicitly, so a missing `exports` fails silently at build time and loudly for users.
233+
- **Shading**: `dependency-reduced-pom.xml` at the repo root is generated by the shade plugin; don't
234+
hand-edit it.
235+
- **Java `\u` in comments and strings**: javac processes `\uXXXX` escapes *before* lexing, even inside
236+
comments. Write `\\u`, or avoid it (e.g. `%04X` in a `String.format`).
237+
- `pom.xml` pins `project.build.outputTimestamp` for reproducible builds.
238+
239+
## Misc
240+
241+
- Always ask for permission for "git commit" and "git push".
242+
- Make commit messages, issue titles compact; avoid unnecessary verbosity.
243+
- Same with comments: use accurate, concise descriptions.

‎release-notes/VERSION-2.x‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,10 @@ No changes since 2.19.1
157157
(requested by Ilenia S)
158158
(fixed by @pjfanning)
159159

160+
2.18.9 (07-Jul-2026)
161+
162+
No changes since 2.18.8
163+
160164
2.18.8 (28-May-2026)
161165

162166
#1611: Apply number-length validator on streaming integer path of async parser

0 commit comments

Comments
 (0)