Missing documentation about transaction management in multi-threaded steps

[pkernevez]

Bug description
When a ChunkOrientedStep is configured with a TaskExecutor, the item processor is executed outside any transaction. This is a regression from Spring Batch 5.2.5 (where the processor ran inside the chunk transaction) and contradicts the documentation, which states:

chunk: The Java-specific name of the dependency that indicates that this is an item-based step and 
the number of items to be processed before the transaction is committed.

Environment
Migrating from Spring Boot 3.5.13 / Spring Batch 5.2.5 (working) to Spring Boot 4.0.5 / Spring Batch 6.0.3 (broken)
Java 25, PostgreSQL

Steps to reproduce
Clone https://github.com/pkernevez/pb-hibernate-proxy/tree/springbatch-transaction-issue (/!\ branch: springbatch-transaction-issue) and run IssueSpringBatchTest.
The test fails because the processor is annotated with @transactional(TxType.MANDATORY) and no transaction is active when it runs.
Commenting out .taskExecutor(executor) in BatchConfiguration makes the test pass.
To reproduce from scratch: build a ChunkOrientedStep with a chunk size, a TransactionManager, and a TaskExecutor, then annotate the processor with @transactional(TxType.MANDATORY).

        return new StepBuilder("currencyStep", jobRepository)
                .<Long, CurrencyEntity>chunk(10)
                .transactionManager(transactionManager)
//                .taskExecutor(executor)
                .reader(currencyReader)
                .processor(processor)
                .writer(writer)
                .build();

Expected behavior

• In normal mode: one transaction per chunk, wrapping reader, processor, and writer for all items in that chunk.
• In faultTolerant() mode after a rollback: one transaction per item (scan mode).

Investigation
I can't understand the current implementation regarding SpringBatch 5, the documentation and my experience.
This is my understanding without scanMode.

Without a TaskExecutor:

1. doExecute iterate on the chunk starts a transaction before each
2. For each chunk we process processNextChunk that delegates to processChunkSequentially
3. We iterate on the item and process them in the transaction started in 1
   All is ok.

With a TaskExecutor:

1. doExecute iterate on the chunk starts a transaction before each in the main thread
2. For each chunk we process processNextChunk that delegates to processChunkConcurrently
3. We iterate on the item and push them in the task executor.
4. They are all executed in other threads without transations ❌
5. Wait for all the item execution before returning to 1 for the next chunk

Consequences

• Correctness: the processor runs without a transaction and items in a chunk no longer share a Hibernate session — breaking consistency guarantees.
• Performance (caching): reference entities loaded by item N in the Session are not cached in the session for items N+1..N+chunkSize.
• Performance (parallelism): previously, each worker processed a full chunk sequentially in its own transaction; a slow item only stalled one worker.
  Now a slow item blocks the main thread (and the whole batch) until all futures of the current chunk finish, even when the executor pool has spare capacity.

Proposed implementation

1. doExecute iterate on the chunk starts a transaction before each in the main thread
2. For each chunk we process processNextChunk that delegates to processChunkConcurrently
3. We prepare the chunk to be executed in another thread, and push them in the task executor.
4. In the main thread we directly return to 1 to prepare the next chunk to push in the executor
   4bis. In the thread of the task executor, we start a new transaction then iterate on the item of the chunk
5. Wait in 'main-thread` for the completion of all the chunk/Future

Minimal Complete Reproducible example
Clone the repo https://github.com/pkernevez/pb-hibernate-proxy/tree/springbatch-transaction-issue
/!\ the right branch is springbatch-transaction-issue

[pkernevez]

Seems related to #5361

[Chandu-Putta]

Hi team, I'd like to investigate and work on this issue. I successfully reproduced the regression using the linked repo (springbatch-transaction-issue branch).

Running IssueSpringBatchTest fails with:
IllegalTransactionStateException: No existing transaction found for transaction marked with propagation 'mandatory'
Commenting out .taskExecutor(executor) makes the test pass, confirming the bug is isolated to concurrent chunk execution.

I also reviewed #5361 which appears to be a related regression — both issues point to broken transaction management in ChunkOrientedTasklet introduced in Spring Batch 6.x. While #5361 affects scan mode transaction isolation, this issue affects normal chunk processing with a TaskExecutor. They may share a common root cause in how transaction boundaries were refactored between 5.x and 6.x.

My proposed approach aligns with the implementation described in this issue — moving transaction management into the worker threads inside processChunkConcurrently using TransactionTemplate, rather than relying on the main thread's thread-local transaction context to propagate across threads.

Would the team accept a PR along these lines? Happy to wait for triage confirmation before submitting. I'm also happy to adjust the approach based on maintainer feedback.

[fmbenhassine]

Thank you for raising this and for providing an example. The sample does not work OOTB, but I managed to use it to understand your intent. This is not a regression from v5, in fact the concurrency model has changed in v6 (see #4955 and #3950) and you might need to update a couple of things. In your case, changing the transaction propagation to REQUIRED fixes the problem. I will explain with some low-level details.

In v5, multi-threaded steps typically require synchronizing readers and writers (most readers and writers provided by Spring Batch are not thread-safe). The problem with this is that we end up using multiple threads that are serialized on reads and writes (What's the point of going parallel if we end up being serialized on both ends?).

In v6, reading and writing of items is done in serial by the main thread executing the step, and only the item processor is executed asynchronously (this is documented here). And since transactions in Spring are thread-bound, the transactional context is not propagated to worker threads (Spring Batch does not manage threads from the pool). But this is not a problem: if an item processor needs a DB read to enrich items for example, it can do it in a separate transaction (it won't see uncommitted changes even with a low isolation level like READ_UNCOMMITTED since the item writer is not called yet at this point). And for DB writes, there is no problem neither as an item processor is not expected to perform write operations anyway (those should be moved to the item writer and will be part of the chunk's transaction).

This new model requires no synchronisation on the user's side (less work to configure things properly with synchronized reader/writer decorators) and no state synchronization at different levels on the framework's side. The constraint of having a thread-safe item processor still applies though (as in v5), and this is documented as well in the same section mentioned above:

Therefore, the ItemProcessor will be called from multiple threads at the same time. This means that the ItemProcessor must be thread-safe. If you are using stateful components in the processing, you must ensure that they are properly synchronized for concurrent access.

However, I see that the transactional behaviour in that case is not documented, so I will update the docs accordingly with these details. Related thread-safety, you mentioned that items in a chunk no longer share a Hibernate session. A hibernate session is not thread safe (see its Javadoc) and is not intended to be used by multiple threads.

I also wanted to mention the introduction of local chunking in v6, where you can provide a ChunkProcessor (vs ItemProcessor) implementation that handles both processing and writing in the same transaction (each thread with be handed an entire chunk to process in a separate transaction. This time, the transactional behaviour is documented (see the important note at the end of the section).

Seems related to #5361

This issue is actually different, there is no scanning involved here.

PS: the documentation section you mentioned is for serial executions (and is correct, see how the corresponding code sample does not use a task executor and the text does not mention concurrent executions), not concurrent executions which is documented here.

[Chandu-Putta]

Thank you for the detailed explanation — this clarifies the intentional design shift in v6 around transaction boundaries in multi-threaded steps.
I successfully reproduced the issue locally and now understand why changing MANDATORY to REQUIRED resolves it for the processor use case.
Since you mentioned the transactional behavior in concurrent mode is not yet documented, I'd be happy to help contribute to the documentation update if that would be useful. I can draft the changes based on your explanation above — covering:

Processors running in worker threads without the chunk transaction context
Recommended propagation (REQUIRED vs MANDATORY) for processors in multi-threaded steps
Pointer to local chunking as an alternative when full chunk-level transaction control is needed

Happy to submit a PR for the docs if that helps. Let me know if you'd prefer to handle it internally.

[pkernevez]

Hi @fmbenhassine, and thank you for your answer.

Let me give you more information about our use case, so you can better understand our needs and how to solve them with v6.

Context: This is a banking application. The daily job creates new position snapshots (in fact, we have several of them, following more or less the same pattern).

To compute a position, we need:

• An account
• The previous position
• All the movements between the previous and the new position
• A lot of reference data: instruments, prices, FX rates, accrued interest, etc. This data is shared across all accounts, but we only load what is needed for the current account and the days being recomputed.

A position is a complex graph of objects: for each one, we keep the list of instruments, their quantities, cost data, custodian information, etc. It is often a graph of 100+ objects.

Processing a position is IO-bound (against the DB), not CPU-bound. Loading the reference data is a significant part of the computation cost. We try to strike a balance, through chunking, between sharing the loading of most of the reference data and keeping the session reasonably small.

Our v5 implementation

We addressed the trade-off above by processing chunks in parallel within a shared Hibernate session. We only used multithreading, but the implementation was already capable of handling distributed processing (Remote Chunking) for when we need to scale.

To do this, we have:

1. A first step that loads the list of accounts to process and saves it in a dedicated work table with (accountNumber, positionStatus). The position status lets us manage the status of each account on our side.
2. A job that reads the accounts (RepositoryItemReaderBuilder<>) from the work table and processes them with an ItemProcessor (which saves them into the Hibernate session), plus a mock writer that does nothing, because the processor has already saved them.

This may have been a misuse, but it works. The reader was configured to read the entire work table, and the processor skips already-processed items in case of a restart or when the fault-tolerance mechanism kicks in. The fault-tolerance mechanism documents the transaction behavior around the chunk in the event of an error.

2nd step configuration

return new StepBuilder("cobSnapshotAccountPositionStep", jobRepository)
        .<CobAccountWorkEntity, String>chunk(chunkSize, transactionManager)
        .faultTolerant()
        .skip(JobErrorStepListener.JobProcessingException.class)
        .skipLimit(skipLimit)
        .taskExecutor(taskExecutor)
        .reader(cobWorkAccountReader(null, null, null))
        .processor(processor)
        .listener(someListener())
        .writer(cobAccountWriter())
        .listener(errorListener)
        .build();

Reader configuration

return new RepositoryItemReaderBuilder<CobAccountWorkEntity>()
        .name("cobWorkAccountReader")
        .saveState(false) // Multi-thread requirement
        .repository(repository)
        .methodName("findByStatusInAndBusinessDate")
        .arguments(List.of(CobAccountWorkEntity.CobAccountStatus.TO_DO, CobAccountWorkEntity.CobAccountStatus.DONE), businessDate)
        .sorts(Collections.singletonMap(AbstractEntity_.ID, Sort.Direction.ASC))
        .pageSize(readerPageSize)
        .build();

Processor extract

@Override
public @Nullable String process(CobAccountWorkEntity item) {
    try {
        var manageItem = accountWorkService.getById(item.getId());
        if (manageItem.getStatus() == CobAccountWorkEntity.CobAccountStatus.DONE) {
            logger.info("Account {} already processed", manageItem.getId());
            return null; // null means the item is filtered out of the batch by the ItemProcessor
        }
        // all the processing
    }
}

Because of the v5 threading issue you mentioned, the item received as a parameter in the processor was detached or not, depending on the chunk. However, all items of a given chunk were processed with the same Hibernate session. Our processor is stateless (@StepScope & @Component("cobAccountProcessor")), but it was executed by Spring Batch in a thread/transaction context that was shared across the chunk.

What is the better way to have the same think with v6 ?
You mentioned LocalChunk that may work, but I think it's not the right way to move next on distributed chunks.

Thanks for your help,