Confluence and CloudWatch and multiple other failing tests fix by san81 · Pull Request #6348 · opensearch-project/data-prepper

san81 · 2025-12-10T00:37:38Z

Description

Fixing two failing tests:

Jira and Confluence host name change to fix the failing tests
We need to explicitly build the RetryPolicy object instead of using the consumer pattern when using "ADAPTIVE_V2". This api is also marked as depecreated and expected to be replaced by RetryStrategy but thats a bigger exercise we will take up clean up all RetryPolicy occurrences together.
Fix OutOfMemoryError in CloudWatchLogsServiceTest by replacing String.repeat() with RandomStringUtils.randomAlphabetic() to generate test data of correct size (25MB)
instead of excessive repetitions (~288GB)
Date format validation fix randomly failing when UUID string generates a string that actually passes Date format validation. Removed UUID usage here. Made the test more reliable
Added Await-ability for more consistent validation in the DDB source coordination test

Check List

New functionality includes testing.
New functionality has a documentation issue. Please link to it in this PR.
- New functionality has javadoc added
Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

…_V2 retry mode and requires using RetryStrategy instead. Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

….repeat() with RandomStringUtils.randomAlphabetic() to generate test data of correct size (25MB) instead of excessive repetitions (~288GB) Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

…or instances to be registered: Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

graytaylor0

Thanks for all these fixes!

dlvenable

Thanks for these changes!

dlvenable · 2025-12-10T16:58:42Z

-        final Optional<SourcePartitionStoreItem> maybeAcquired =
-                objectUnderTest.tryAcquireAvailablePartition(sourceIdentifier, ownerId, Duration.ofSeconds(20));
+        // Wait for partition to be available in DynamoDB Local before attempting to acquire
+        final Optional<SourcePartitionStoreItem>[] maybeAcquiredHolder = new Optional[]{Optional.empty()};


I'm not sure this will actually fix the issue. I've tried this a few times and ended up disabling this test in #6328.

There is still some sleep in that PR, which could add uncertainty. I am hoping, we can give it a try with this approach and see.

dlvenable · 2025-12-10T16:59:58Z

        void testValidAndInvalidOutputFormats() throws NoSuchFieldException, IllegalAccessException {
-            setField(DateProcessorConfig.class, dateProcessorConfig, "outputFormat", random);
+            // Use a string with invalid pattern characters (] and [ are reserved and will always fail)
+            setField(DateProcessorConfig.class, dateProcessorConfig, "outputFormat", "invalid[pattern]format");


This should probably an @ParameterizedTest with a few different invalid formats provided.

Yes, converted it now.
Thought of minimizing the changes but as I am touching this, I could as well do it 👍

dlvenable · 2025-12-10T17:00:59Z

+
        final ClientOverrideConfiguration.Builder configBuilder = ClientOverrideConfiguration.builder()
-                .retryPolicy(r -> r.numRetries(AwsConfig.DEFAULT_CONNECTION_ATTEMPTS));
+                .retryPolicy(retryPolicy);


Do you know why this improves the tests?

I think, it started with AWS SDK behavior/validation changes that made the consumer pattern incompatible with the default retry mode. They now changed the default retry mode to be ADAPTIVE_V2

https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/retry-strategy.html

Ideal fix is to move to using RetryStrategy but thats a bigger change so minimized the changes here by avoiding using the consumer pattern. https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/retries/api/RetryStrategy.html

dlvenable · 2025-12-10T17:42:28Z

        for (int i = 0; i < thresholdConfig.getBatchSize(); i++) {
-            JacksonEvent mockJacksonEvent = (JacksonEvent) JacksonEvent.fromMessage("testMessage".repeat((int) thresholdConfig.getMaxEventSizeBytes()));
+            JacksonEvent mockJacksonEvent =
+                    (JacksonEvent) JacksonEvent.fromMessage(RandomStringUtils.insecure().nextAlphabetic(messageSize));


Do you know why the repeat used so much memory?

I see. It was because the original string was already long.

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

dlvenable

Thanks @san81 !

…earch-project#6348) Making the tests less flaky. More reliable. Avoiding possible Out of memory issue with large pay load generation.

…earch-project#6348) Making the tests less flaky. More reliable. Avoiding possible Out of memory issue with large pay load generation. Signed-off-by: Nathan Wand <wandna@amazon.com>

opensearch-trigger-bot · 2026-02-05T18:37:24Z

The backport to 2.13 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.13 2.13
# Navigate to the new working tree
cd .worktrees/backport-2.13
# Create a new branch
git switch --create backport/backport-6348-to-2.13
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 442cd7058c5546ea6337f48c55d180fa3ac8c590
# Push it to GitHub
git push --set-upstream origin backport/backport-6348-to-2.13
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.13

Then, create a pull request where the base branch is 2.13 and the compare/head branch is backport/backport-6348-to-2.13.

opensearch-trigger-bot · 2026-02-05T18:37:27Z

The backport to 2.13 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.13 2.13
# Navigate to the new working tree
cd .worktrees/backport-2.13
# Create a new branch
git switch --create backport/backport-6348-to-2.13
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 442cd7058c5546ea6337f48c55d180fa3ac8c590
# Push it to GitHub
git push --set-upstream origin backport/backport-6348-to-2.13
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.13

Then, create a pull request where the base branch is 2.13 and the compare/head branch is backport/backport-6348-to-2.13.

Making the tests less flaky. More reliable. Avoiding possible Out of memory issue with large pay load generation. (cherry picked from commit 442cd70)

…roject#6348. Taken from 442cd70 Co-authored-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com> Signed-off-by: David Venable <dlv@amazon.com>

Taken from 442cd70 Signed-off-by: David Venable <dlv@amazon.com> Co-authored-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

confluence host name change to fix the failing tests

98f4456

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

san81 requested review from KarstenSchnitter, chenqi0805, dinujoh, dlvenable, engechas, graytaylor0, kkondaka, oeyh, sb2k16 and srikanthjg as code owners December 10, 2025 00:37

The AWS SDK v2 has deprecated the retryPolicy API when using ADAPTIVE…

d06269b

…_V2 retry mode and requires using RetryStrategy instead. Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

san81 changed the title ~~Confluence host name change to fix the failing tests~~ Confluence and CloudWatch failing tests fix Dec 10, 2025

san81 added 8 commits December 9, 2025 17:27

Fix OutOfMemoryError in CloudWatchLogsServiceTest by replacing String…

a766e90

….repeat() with RandomStringUtils.randomAlphabetic() to generate test data of correct size (25MB) instead of excessive repetitions (~288GB) Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

Fix OutOfMemoryError in CloudWatchLogsServiceTest by replacing String…

b176372

….repeat() with RandomStringUtils.randomAlphabetic() to generate test data of correct size (25MB) instead of excessive repetitions (~288GB) Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

domain name fix

62b5ebd

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

unused imports

7c2de36

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

checkstyle issues

1d3d31b

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

Added an await() statement before assertions to wait for all process…

17dfb37

…or instances to be registered: Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

fixing the validation logic to make the tests behave consistently

ba129b3

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

Making the test more reliable with using awaitability

0749d35

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

san81 changed the title ~~Confluence and CloudWatch failing tests fix~~ Confluence and CloudWatch and multiple other failing tests fix Dec 10, 2025

graytaylor0 previously approved these changes Dec 10, 2025

View reviewed changes

dlvenable requested changes Dec 10, 2025

View reviewed changes

dlvenable reviewed Dec 10, 2025

View reviewed changes

converted to become a parameterized test

abdec4e

Signed-off-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

san81 dismissed graytaylor0’s stale review via abdec4e December 10, 2025 17:52

dlvenable approved these changes Dec 10, 2025

View reviewed changes

graytaylor0 approved these changes Dec 10, 2025

View reviewed changes

san81 merged commit 442cd70 into opensearch-project:main Dec 10, 2025
45 of 47 checks passed

dlvenable added backport 2.13 labels Feb 5, 2026

dlvenable mentioned this pull request Feb 5, 2026

[Backport 2.13] Confluence and CloudWatch and multiple other failing tests fix #6492

Closed

4 tasks

dlvenable mentioned this pull request Feb 5, 2026

Testing stability changes. This is a partial commit from #6348. #6493

Merged

4 tasks

dlvenable added a commit that referenced this pull request Feb 9, 2026

Testing stability changes. This is a partial commit from #6348. (#6493)

a74108f

Taken from 442cd70 Signed-off-by: David Venable <dlv@amazon.com> Co-authored-by: Santhosh Gandhe <1909520+san81@users.noreply.github.com>

Conversation

san81 commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Check List

Uh oh!

graytaylor0 left a comment

Choose a reason for hiding this comment

Uh oh!

dlvenable left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dlvenable left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

opensearch-trigger-bot Bot commented Feb 5, 2026

Uh oh!

opensearch-trigger-bot Bot commented Feb 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

san81 commented Dec 10, 2025 •

edited

Loading