From 9d139e1cbb0fd3fd6329b03d46a0b5e833bc5ad3 Mon Sep 17 00:00:00 2001 From: Harvy Kim Date: Tue, 12 May 2026 22:59:19 +0900 Subject: [PATCH] Document chunk scanning behavior in fault-tolerant steps The Retry section did not describe what users observe when an ItemWriter fails in a fault-tolerant step: the writer is invoked again, once per item, during recovery, even with a "never retry" policy. Add a "Chunk Scanning" section to retry.adoc covering the trigger condition, the per-item-per-transaction sequence, the non-skippable failure mode, and the two practical opt-outs. Cross-link from chunk-oriented-processing.adoc and from the existing mention in scalability.adoc. Resolves #3946 Signed-off-by: Harvy Kim --- .../modules/ROOT/pages/retry.adoc | 41 ++++++++++++++++++- .../modules/ROOT/pages/scalability.adoc | 2 +- .../pages/step/chunk-oriented-processing.adoc | 5 +++ 3 files changed, 46 insertions(+), 2 deletions(-) diff --git a/spring-batch-docs/modules/ROOT/pages/retry.adoc b/spring-batch-docs/modules/ROOT/pages/retry.adoc index 2b0541523b..fbdea27694 100644 --- a/spring-batch-docs/modules/ROOT/pages/retry.adoc +++ b/spring-batch-docs/modules/ROOT/pages/retry.adoc @@ -3,7 +3,6 @@ [[retry]] = Retry -:page-section-summary-toc: 1 To make processing more robust and less prone to failure, it sometimes helps to @@ -17,3 +16,43 @@ Examples include remote calls to a web service that fails because of a network g As of v6.0, Spring Batch does *not* use https://github.com/spring-projects/spring-retry[Spring Retry] to automate retry operations within the framework, and is now based on the https://docs.spring.io/spring-framework/reference/7.0/core/resilience.html[core retry feature] provided by Spring Framework 7.0. ==== + +[[chunk-scanning]] +== Chunk Scanning + +In a fault-tolerant chunk-oriented step, when an exception thrown by the +`ItemWriter` is classified as skippable, Spring Batch does not skip the whole +chunk. Instead, after the configured retries are exhausted, the step enters +_chunk scanning_ mode and replays the chunk one item at a time in order to +isolate the failing item. Each replayed item is written in its own transaction, +so the items that succeed are committed and the failing item is reported as a +write-skip (and to any registered `SkipListener`), while the rest of the chunk +is preserved. + +This is also why the `ItemWriter` can be invoked again after a failure even when +no retry policy is configured. With a "never retry" policy (for example, +`throwable -> false`), retries are simply considered exhausted on the first +attempt, so scanning starts immediately if the exception is classified as +skippable. If the exception is not skippable, the step fails with a +`NonSkippableWriteException` without scanning. + +Concretely, for a chunk of size `N` where the writer fails on one of the items: + +. The initial transaction wrapping the chunk is rolled back. +. The following `N` transactions each invoke the `ItemWriter` with a single item + from the buffered output of that chunk. +. A successful single-item write is committed; a failure that the `SkipPolicy` + accepts increments the write-skip count and is rolled back; a failure that the + `SkipPolicy` rejects fails the step with a `NonSkippableWriteException`. + +When this behavior is undesirable, the two practical options are to size the +chunk to one (so there is nothing to scan) or to move the failing condition into +the `ItemProcessor`, which is called per item, so failures raised there are +handled directly without scanning. Some failures, however, can only be detected +on the writer side (for example, constraint violations surfaced by a JDBC batch +insert or errors returned by a remote service) and therefore cannot be moved to +the processor. + +The behavior is exercised by `ChunkOrientedStepFaultToleranceIntegrationTests` +in `spring-batch-core`, in particular +`testNoDuplicateWritesAfterScanModeInSequentialMode`. diff --git a/spring-batch-docs/modules/ROOT/pages/scalability.adoc b/spring-batch-docs/modules/ROOT/pages/scalability.adoc index a28bfdb9bb..ca26519c3d 100644 --- a/spring-batch-docs/modules/ROOT/pages/scalability.adoc +++ b/spring-batch-docs/modules/ROOT/pages/scalability.adoc @@ -250,7 +250,7 @@ public ChunkProcessor chunkProcessor(DataSource dataSource, TransactionTemp [IMPORTANT] ==== -It should be noted that transaction management of the chunk as well as fault tolerance features (like retry, skip, chunk scanning, etc) +It should be noted that transaction management of the chunk as well as fault tolerance features (like retry, skip, xref:retry.adoc#chunk-scanning[chunk scanning], etc) are not handled by the `ChunkTaskExecutorItemWriter` nor the driving step, and are the responsibility of the delegate chunk processor. Moreover, the lifecycle of the task executor is not handled by the `ChunkTaskExecutorItemWriter`. ==== diff --git a/spring-batch-docs/modules/ROOT/pages/step/chunk-oriented-processing.adoc b/spring-batch-docs/modules/ROOT/pages/step/chunk-oriented-processing.adoc index 180c6a755c..ee1b6a4b84 100644 --- a/spring-batch-docs/modules/ROOT/pages/step/chunk-oriented-processing.adoc +++ b/spring-batch-docs/modules/ROOT/pages/step/chunk-oriented-processing.adoc @@ -58,3 +58,8 @@ itemWriter.write(processedItems); For more details about item processors and their use cases, see the xref:processor.adoc[Item processing] section. +When a chunk-oriented step is configured as fault-tolerant, an exception +thrown during writing may trigger _chunk scanning_, in which the items of the +failing chunk are replayed one at a time so the failing item can be isolated and +skipped. See xref:retry.adoc#chunk-scanning[Chunk Scanning] for details. +