Fix skip policy with JPA ItemWriter in scan mode#5379
Closed
MinChul-Son wants to merge 2 commits into
Closed
Conversation
Process one item per transaction in scan mode to prevent JPA rollback-only contamination across items. Also explicitly set local rollback-only in doExecute() to avoid UnexpectedRollbackException from JpaTransactionManager. Closes spring-projectsgh-5377 Signed-off-by: Minchul-Son <smc5236@naver.com>
Signed-off-by: Minchul-Son <smc5236@naver.com>
Contributor
|
Thank you for the PR! LGTM 👍 Rebased and merged as 05b2484. I introduced two changes:
Thank you for your contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Closes #5377
When using a JPA-based
ItemWriterwith a skip policy, the step fails with:This only happens with JPA; JDBC-based writers work correctly.
Root Cause
When a chunk write fails and the step enters scan mode,
ChunkOrientedStepwas processing all pending items in a single transaction. JPA/Hibernate marks the entire transaction as rollback-only after a flush failure. This causes the next item write in the same transaction to throw"Transaction silently rolled back", which is not in the skip list and gets promoted to aNonSkippableWriteException, failing the step.JDBC does not have this problem because a caught SQL exception does not poison the surrounding transaction.
A secondary issue also existed: when
scan()catches a skippable exception and returns normally,doExecute()detectstransactionStatus.isRollbackOnly()and returns from the lambda without callingtransactionStatus.setRollbackOnly()explicitly. Since JPA sets the transaction as globally rollback-only (viaConnectionHolder) but not locally,AbstractPlatformTransactionManager.commit()callsprocessRollback(defStatus, unexpected=true)and throwsUnexpectedRollbackException.Changes
ChunkOrientedStepOne item per scan transaction
Replaced
ChunkTracker.pendingChunk: Chunk<O>withpendingScanItems: LinkedList<O>. Instead of passing the entire pending chunk toscan()at once, thedoExecute()while loop now dequeues one item at a time viapollNextScanItem(), so each scan item runs in its own independent transaction. A JPA rollback-only marker from a failed item no longer affects subsequent items.Prevent
UnexpectedRollbackExceptionIn
doExecute(), when a rollback-only state is detected,transactionStatus.setRollbackOnly()is now called explicitly before returning from the lambda. This converts the global rollback-only (set by JPA) to a local one, soAbstractPlatformTransactionManagerperforms a clean rollback instead of throwingUnexpectedRollbackException.beforeChunk/afterChunkconsistency (concurrent mode)The concurrent scan path was calling
compositeChunkListener.beforeChunk(new Chunk<>())with an empty chunk. This is now aligned with the sequential path — both pass the actual single-item chunk, andafterChunkis guarded by!status.isRollbackOnly().Tests (
ChunkOrientedStepScanModeIntegrationTests)Four new test cases are added:
testSkipPolicyWithJpaLikeRollbackOnlyBehaviorInSequentialModetestSkipPolicyWithJpaLikeRollbackOnlyBehaviorInConcurrentModetestUnexpectedRollbackExceptionPreventedInSequentialScanModeUnexpectedRollbackExceptionwhenConnectionHolderis globally rollback-only (directConnectionHolder.setRollbackOnly()simulation of JPA behavior)testUnexpectedRollbackExceptionPreventedInConcurrentScanModeThe
globalRollbackOnlyWriterhelper directly callsConnectionHolder.setRollbackOnly()on failure, which is exactly whatJpaTransactionManager/Hibernate does internally, and is the minimal reproduction of theUnexpectedRollbackExceptionpath.