fix: preserve export progress statistics on batch failure#313
fix: preserve export progress statistics on batch failure#313YunchuWang wants to merge 1 commit into
Conversation
… batch failure When a batch of export activities fails after exhausting all retry attempts, the orchestrator commits a checkpoint with scannedInstances=0 and exportedInstances=0. This discards the actual work done: the number of instances that were scanned and the number that were successfully exported before the batch was marked as failed. The entity accumulates these counters via addition, so reporting 0 means the job's progress statistics undercount the real work, giving operators inaccurate progress data. Fix: Use the actual scannedCount and batchResult.exportedCount values in the failure-path checkpoint, matching the success-path behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR fixes progress reporting in the export-history job orchestrator so that, when an export batch ultimately fails (after exhausting the batch-level retry loop), the job’s checkpoint still records the actual scanned/exported counts and preserves failure details. This aligns failure-path checkpointing behavior with the success path and addresses the incorrect “0/0 progress” reporting described in issue #213.
Changes:
- Update failure-path checkpoint commits to use
scannedCountandbatchResult.exportedCountinstead of hardcoded zeros. - Add regression tests that validate checkpoint counts for partial-batch failures, full-batch failures, and successful batches.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| packages/durabletask-js-export-history/src/orchestrators/export-job-orchestrator.ts | Fixes failure-path checkpointing to persist actual scanned/exported counts alongside failure details. |
| packages/durabletask-js-export-history/test/export-job-orchestrator.spec.ts | Adds regression coverage to ensure progress statistics are preserved on batch failure and correct on success. |
|
Closing after a first-principles cross-SDK review. The issue's premise ("copy-paste oversight") is not supported. The .NET sibling Why Net: in the current code failure is terminal (throws → |
Summary
Fixes #213
Testing
npm test -w @microsoft/durabletask-js-export-history -- --runTestsByPath test/export-job-orchestrator.spec.tsnpm run build -w @microsoft/durabletask-js-export-history