Skip to content

Commit edaa266

Browse files
committed
Print special warning is HDF5 dataset is chunked
openPMD API does not yet write HDF5 datasets in a chunked manner, so let's not go through hoops to read them back in a special manner.
1 parent 1d43d22 commit edaa266

File tree

1 file changed

+34
-7
lines changed

1 file changed

+34
-7
lines changed

src/IO/HDF5/HDF5IOHandler.cpp

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -384,11 +384,6 @@ HDF5IOHandlerImpl::availableChunks(
384384
Writable * writable,
385385
Parameter< Operation::AVAILABLE_CHUNKS > & parameters )
386386
{
387-
std::cerr << R"END(
388-
[HDF5] Warning: HDF5 does not carry information on available chunks.
389-
Returning instead the whole dataset as one chunk.
390-
Some positions in the chunk may hold undefined data.
391-
)END" << std::endl;
392387
auto res = m_fileIDs.find( writable );
393388
hid_t dataset_id = H5Dopen(
394389
res->second,
@@ -399,12 +394,44 @@ HDF5IOHandlerImpl::availableChunks(
399394
"[HDF5] Internal error: Failed to open HDF5 dataset during dataset "
400395
"read" );
401396
hid_t dataset_space = H5Dget_space( dataset_id );
402-
403397
int ndims = H5Sget_simple_extent_ndims( dataset_space );
404398
VERIFY(
405399
ndims >= 0,
406-
"[HDF5]: Internal error: Failed to retrieve dimensionality of dataset "
400+
"[HDF5]: Internal error: Failed to retrieve dimensionality of "
401+
"dataset "
407402
"during dataset read." );
403+
404+
// now let's figure out whether this one has chunks
405+
hid_t propertyList = H5Dget_create_plist( dataset_id );
406+
std::vector< hsize_t > chunkExtent( ndims, 0 );
407+
int chunkDimensionality =
408+
H5Pget_chunk( propertyList, ndims, chunkExtent.data() );
409+
410+
if( chunkDimensionality < 0 )
411+
{
412+
std::cerr << R"END(
413+
[HDF5] Warning: HDF5 does not carry information on available chunks.
414+
Returning instead the whole dataset as one chunk.
415+
Some positions in the chunk may hold undefined data.
416+
)END" << std::endl;
417+
}
418+
else
419+
{
420+
// so, the dataset indeed has chunks
421+
// alas, this backend doesn't write chunks, so for now, reading them is
422+
// unimplemented
423+
std::cerr << R"END(
424+
[HDF5] Warning: HDF5 dataset has chunked layout. Since the openPMD API does not
425+
support writing chunked HDF5 datasets, the whole dataset will be
426+
returned as one chunk.
427+
Some positions in the chunk may hold undefined data.
428+
)END" << std::endl;
429+
/*
430+
* https://hdf5.io/develop/group___h5_d.html#gaccff213d3e0765b86f66d08dd9959807
431+
* May or may not be helpful if implementing this properly one day.
432+
*/
433+
}
434+
408435
std::vector< hsize_t > dims( ndims, 0 );
409436
// return value is equal to ndims
410437
H5Sget_simple_extent_dims( dataset_space, dims.data(), nullptr );

0 commit comments

Comments
 (0)