Skip to content

HDDS-13466. Intermittent timeout in TestOmSnapshot#9471

Merged
adoroszlai merged 3 commits intoapache:masterfrom
swamirishi:HDDS-13466
Dec 12, 2025
Merged

HDDS-13466. Intermittent timeout in TestOmSnapshot#9471
adoroszlai merged 3 commits intoapache:masterfrom
swamirishi:HDDS-13466

Conversation

@swamirishi
Copy link
Copy Markdown
Contributor

@swamirishi swamirishi commented Dec 10, 2025

What changes were proposed in this pull request?

Fix OmSnapshotTest to close snapshots in snapshot cache every 100 ms to ensure too many files are not open. Fix individual test to also close all snapshots opened in the test to ensure miniozone cluster shuts down correctly

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13466

How was this patch tested?

Fixing existing tests

Change-Id: Ia1bb3f9bdd199bf7c652b25f158556c8b0b24c10
@adoroszlai
Copy link
Copy Markdown
Contributor

Please run flaky-test-check in your fork for TestOmSnapshot*.

@swamirishi
Copy link
Copy Markdown
Contributor Author

@swamirishi swamirishi marked this pull request as ready for review December 10, 2025 15:42
@adoroszlai
Copy link
Copy Markdown
Contributor

https://github.com/swamirishi/ozone/actions/runs/20092503951

Thanks, this looks promising.

However, it looks like flaky-test-check does not build/test with the native lib (see Skipped: 72 and Tests run: 0 in many cases).

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.62 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotObjectStore
[INFO] Tests run: 72, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 368.0 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotFsoWithoutNativeLibWithLinkedBuckets
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.99 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotWithoutBucketLinkingLegacy
[INFO] Tests run: 72, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 369.2 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotFsoWithoutNativeLib
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.79 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotWithBucketLinkingLegacy
[WARNING] Tests run: 72, Failures: 0, Errors: 0, Skipped: 72, Time elapsed: 0.017 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotFsoWithNativeLib
[WARNING] Tests run: 72, Failures: 0, Errors: 0, Skipped: 72, Time elapsed: 0.023 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotFsoWithNativeLibWithLinkedBuckets
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.93 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotDisabled
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.67 s -- in org.apache.hadoop.ozone.om.snapshot.TestOmSnapshotObjectStoreWithLinkedBuckets

@swamirishi
Copy link
Copy Markdown
Contributor Author

Any idea how I can trigger the native build in the framework?

@adoroszlai
Copy link
Copy Markdown
Contributor

Any idea how I can trigger the native build in the framework?

Adding -Drocks_tools_native in these places should help:

args="-DskipRecon -DskipShade -Dmaven.javadoc.skip=true"

args="-DexcludedGroups=native|slow|unhealthy -DskipShade"

Change-Id: I712916db194af2f906fad8bbfc130093cbc6a928
@swamirishi
Copy link
Copy Markdown
Contributor Author

@swamirishi
Copy link
Copy Markdown
Contributor Author

The previous run timed out after 6 hours https://github.com/swamirishi/ozone/actions/runs/20133765218/job/57781800484

@swamirishi
Copy link
Copy Markdown
Contributor Author

@adoroszlai the splits passed can this be merged?

@adoroszlai adoroszlai changed the title HDDS-13466. Fix Unhealthy Snapshot Test HDDS-13466. Intermittent timeout in TestOmSnapshot Dec 11, 2025
Copy link
Copy Markdown
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @swamirishi for the fix.

- name: Build (most) of Ozone
run: |
args="-DskipRecon -DskipShade -Dmaven.javadoc.skip=true"
args="-DskipRecon -DskipShade -Dmaven.javadoc.skip=true ${{ github.event.inputs.maven_additional_args }}"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of the generic "additional args" I'd prefer adding -Drocks_tools_native to match normal integration check.

Suggested change
args="-DskipRecon -DskipShade -Dmaven.javadoc.skip=true ${{ github.event.inputs.maven_additional_args }}"
args="-DskipRecon -DskipShade -Dmaven.javadoc.skip=true -Drocks_tools_native"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

fi

args="-DexcludedGroups=native|slow|unhealthy -DskipShade"
args="-DexcludedGroups=native|slow|unhealthy -DskipShade ${{ github.event.inputs.maven_additional_args }}"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
args="-DexcludedGroups=native|slow|unhealthy -DskipShade ${{ github.event.inputs.maven_additional_args }}"
args="-DexcludedGroups=slow|unhealthy -DskipShade -Drocks_tools_native"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +23 to +26
maven_additional_args:
description: Additional Maven build arguments
default: ''
required: false
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
maven_additional_args:
description: Additional Maven build arguments
default: ''
required: false

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Change-Id: Iee168bb4eeb16351d52062f027ddeaf51888c633
@swamirishi
Copy link
Copy Markdown
Contributor Author

@adoroszlai can I merge this?

@adoroszlai adoroszlai merged commit f490ece into apache:master Dec 12, 2025
29 checks passed
@adoroszlai
Copy link
Copy Markdown
Contributor

Thanks @swamirishi for the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants