Limit libhdf5 versions?

[davidhassell]

Should we limit libhdf5 versions? This can be done with the --L, --low and --high options to h5repack:

   -L, --latest            Use latest version of file format
                           This option will take precedence over the options
                           --low and --high
   --low=BOUND             The low bound for library release versions to use
                           when creating objects in the file
                           (default is H5F_LIBVER_EARLIEST)
   --high=BOUND            The high bound for library release versions to use
                           when creating objects in the file
                           (default is H5F_LIBVER_LATEST)

I think we should only do this if we know that there is a version of libhdf5 that doesn't work for CMIP7 input files. We want to reduce the number of barriers to use wherever possible.

[davidhassell]

@zequihg50 - any thoughts? It's not urgent - we can always add in a restriction at any stage.

[zequihg50]

We might copy the behavior of netCDF in here, although it requires some programming and it is not straightforward. The details are discussed in this issue Unidata/netcdf-c#951 and the code of the C library is available here. It basically sets bounds in function of the version of HDF5 installed.

[davidhassell]

Hi @zequihg50, I think I don't understand this! We ask for "a working h5repack". Given that, is the exact version of libhdf5 of concern to us?

[zequihg50]

My concern is about what h5repack will do to some internal data structures and how these will be compatible with netCDF clients. Thus, I was considering if using the same exact protocol that netCDF-C library uses is the correct way to go. It seems netCDF-C always uses high = H5F_LIBVER_LATEST but low bound may vary in function of the installed HDF5 library version. Thus, I'm not sure if it would be better to use low = H5F_LIBVER_V18 or low = H5F_LIBVER_EARLIEST.

[zequihg50]

After some reading, I think we’re fine with the defaults. We just need to be aware of unusual combinations of HDF5 and NetCDF that could cause issues Unidata/netcdf4-python#549.