Skip to content

Make shape an optional attribute for constant components#1661

Merged
ax3l merged 21 commits intoopenPMD:devfrom
franzpoeschel:dont-require-shape-for-constant-components
Dec 15, 2025
Merged

Make shape an optional attribute for constant components#1661
ax3l merged 21 commits intoopenPMD:devfrom
franzpoeschel:dont-require-shape-for-constant-components

Conversation

@franzpoeschel
Copy link
Copy Markdown
Contributor

@franzpoeschel franzpoeschel commented Aug 7, 2024 •
edited
Loading

Fix #1663

TODO:

  • Actually test this
  • Link to standard update: Const Records: Relax shape for Particles openPMD-standard#289
  • Maybe only do this for standard version 2.0 - actually ok as a fallback to expose in 1.X files, too UPDATE: This comment explains a corner case that makes me hesitate adding this for version 1.0 since it might break datasets in singular cases. UPDATE: Corner case not relevant enough, would imply misuse of reserved value attribute, so not strictly standard-breaking.
  • For particles only or for meshes too?
  • Test that the shape can be queried from every component by the reader, even if shape attribute is missing
  • Document compatibility option

@ax3l ax3l self-requested a review August 27, 2024 20:53
@ax3l ax3l added this to the 0.17.0 milestone Aug 27, 2024
@ax3l ax3l requested a review from guj August 27, 2024 20:53
@ax3l ax3l mentioned this pull request Aug 27, 2024
Merged
6 tasks
@franzpoeschel franzpoeschel mentioned this pull request Sep 6, 2024
Closed
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 0c031dd to 11425ce Compare October 11, 2024 15:08
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 3764e01 to 9f2474b Compare November 15, 2024 14:12
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch 2 times, most recently from 96a8dce to 1b835b2 Compare December 17, 2024 10:49
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 1b835b2 to ded204d Compare February 21, 2025 10:19
@guj
Copy link
Copy Markdown
Contributor

guj commented Feb 21, 2025

Thanks I will give it a try!

guj
guj approved these changes Mar 5, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@guj guj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried with joined array, it worked!

@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from ded204d to 5e0fcbd Compare March 26, 2025 10:17
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch 2 times, most recently from 6dfb00e to d63f5ad Compare April 4, 2025 08:31
github-advanced-security[bot]
github-advanced-security bot found potential problems Apr 4, 2025
View reviewed changes
test/SerialIOTest.cpp
#endif
}

TEST_CASE("components_without_extent", "[serial]")

Check notice

Code scanning / CodeQL

Unused static function Note test

Static function CATCH2_INTERNAL_TEST_18 is unreachable (
autoRegistrar19
Loading
must be removed at the same time)
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 0a30b07 to 329d77e Compare April 7, 2025 16:10
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 07c7f2e to d37c7a2 Compare April 22, 2025 08:36
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch 2 times, most recently from f4117bb to e1b2c6c Compare July 17, 2025 08:33
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from e1b2c6c to 0387d88 Compare July 22, 2025 16:03
github-advanced-security[bot]
github-advanced-security bot found potential problems Jul 22, 2025
View reviewed changes
test/Files_ParallelIO/read_variablebased_randomaccess.cpp
}
MPI_Barrier(MPI_COMM_WORLD);
read_file_in_parallel(true);
// read_file_in_parallel(std::nullopt);

Check notice

Code scanning / CodeQL

Commented-out code Note test

This comment appears to contain commented-out code.
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch 2 times, most recently from 751760c to 396f0de Compare August 11, 2025 15:07
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch 2 times, most recently from 72532f8 to 52aae35 Compare September 3, 2025 09:19
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 31948e9 to 8353bcc Compare September 5, 2025 12:42
ax3l
ax3l reviewed Sep 16, 2025
View reviewed changes
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 05bbff0 to 5d828d4 Compare September 17, 2025 09:56
@franzpoeschel franzpoeschel mentioned this pull request Oct 20, 2025
Merged
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 5d828d4 to 6257ac7 Compare November 21, 2025 11:52
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from 6257ac7 to d6fc7ec Compare December 3, 2025 15:57
@franzpoeschel
Make shape an optional attribute for constant components
c7dc258
@franzpoeschel
Adhere to the standard changes more closely
1e23ac3
@franzpoeschel
Remove is_scalar distinction from flushing again
489803c 
Both reasons for making the distinction now no longer valid
@franzpoeschel
Cleanup
77c79bb
@franzpoeschel
Homogenize extents
249043a
@franzpoeschel
Check for extent consistency, fill in undefined extents
00c7d6e
@franzpoeschel
Make homogenization configurable
d2aa11c
@franzpoeschel
Fix tests with this
50e7cd4
@franzpoeschel
Add note on how to deactivate this verification
15ea735
@franzpoeschel
WIP: Testing
98918ad
@franzpoeschel
CI fixes
04e8fd8
@franzpoeschel
Extend test
84b8d0a
@franzpoeschel
Use JSON backend for this test
478de74
@franzpoeschel
Fix unfinished_iteration_test
3358051
@franzpoeschel
Bugfix: skip skipped compoments
6a80a7d
@franzpoeschel
Use UNDEFINED_EXTENT for this instead of empty extent
c02dab4
@franzpoeschel
Document verify_homogeneous_extents option
6deed76
@franzpoeschel
Fix typo
3a65f58
@franzpoeschel
Adapt test to stricter parsing
83dd352
@franzpoeschel
Relax JSON schema
02f801f
@franzpoeschel franzpoeschel force-pushed the dont-require-shape-for-constant-components branch from d6fc7ec to 02f801f Compare December 5, 2025 15:05
ax3l
ax3l approved these changes Dec 15, 2025
View reviewed changes
test/SerialIOTest.cpp
// ADIOS2 does not support changing the dimensionality
// (older versions used to somewhat support it, but not really)
auto E_y = iteration.meshes["E"]["y"];
auto B_y = iteration.meshes["B"]["y"];
Copy link
Copy Markdown
Member

@ax3l ax3l Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting example. I was thinking primarily about particle records, but yes fields also could be used to shapes of some components depend on others...

@ax3l ax3l merged commit 57dba13 into openPMD:dev Dec 15, 2025
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ax3l ax3l ax3l approved these changes

+1 more reviewer

@guj guj guj approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@ax3l ax3l

Labels

backend: ADIOS2 discussion frontend: C++17

Projects

None yet

Milestone

0.17.0

Development

Successfully merging this pull request may close these issues.

Constant Records for JoinedArrays: Skip shape for Particles

4 participants

@franzpoeschel @guj @ax3l @github-advanced-security