Migrating from HDF5 1.12 to HDF5 1.14

Migrating to HDF5 1.14 from Previous Versions of HDF5

API Changes

There are new API calls that require API Compatibility Macros in HDF5 1.14.0. There are, however, two API calls which have had their signature change.

H5Iregister_type / H5I_free_t

The signature of H5Iregister_type did not change, but the H5I_free_t callback it accepts did have its signature change to add an asynchronous request parameter. There is no API compatibility macro to paper over this change. The request parameter can be ignored in the callback if you are not concerned with asynchronous operations and future IDs. A description of how the request parameter should be used will be found in the (soon to be released) HDF5 Asynchronous Programming Guide.

Old: - typedef herr_t (*H5I_free_t)(void *obj);
New: - typedef herr_t (*H5I_free_t)(void *obj, void **request);

H5VLquery_optional

The way optional operations are handled in the virtual object layer (VOL) changed significantly in HDF5 1.14.0. To better support this, the H5VLquery_optional API call now takes an output flags parameter instead of a Boolean. Since the entire 1.12 VOL API has been deprecated, no API compatibility macro exists for this API call.

Old: - herr_t H5VLquery_optional(hid_t obj_id, H5VL_subclass_t subcls, int opt_type, hbool_t *supported);
New: - herr_t H5VLquery_optional(hid_t obj_id, H5VL_subclass_t subcls, int opt_type, uint64_t *flags);

Virtual File Layer (VFL) Changes

The virtual file layer has changed in HDF5 1.14.0. Existing virtual file drivers (VFDs) will have to be updated to work with this version of the library.

Virtual Object Layer (VOL) Changes

The virtual object layer has changed significantly in HDF5 1.14.0 and the 1.12 VOL API is now considered deprecated and unsupported. Existing virtual object layer connectors should be updated to work with this version of the library.

New Features in HDF5 Release 1.14

The new features in the HDF5 1.14 series include:

Adding support for 16-bit floating point and Complex number datatypes to HDF
Support for the 16-bit floating-point _Float16 C type has been added to HDF5. On platforms where this type is available, this can enable more efficient storage of floating-point data when an application doesn't need the precision of larger floating-point datatypes. It can also allow for improved performance when converting between 16-bit floating-point data and data of another HDF5 datatype.

The HDF5 Event Set Interface
HDF5 provides asynchronous APIs for the HDF5 VOL connectors that support asynchronous HDF5 operations using the HDF5 Event Set (H5ES) API. This allows I/O to proceed in the background while the application is performing other tasks.

VFD Sub-filing
The basic idea behind sub-filing is to find the middle ground between single shared file and one file per process - thereby avoiding some of the complexity of one file per process, and minimizing the locking issues of a single shared file on a parallel file system.

Onion VFD
There is a desire to introduce and track modifications to an HDF5 file while preserving or having access to the file as it existed prior to a particular set of modifications. To this end, this RFC proposes an Onion Virtual File Driver (VFD) as an effectively in-file revision management facility. Users will be able to open a particular revision of the file, read from and make modifications to the file, and write to file as a new revision. The name "Onion" derives from a mnemonic: the original file exists with data layered atop one another from an original file to the most recent revision

New HDF5 API Routines for HPC Applications - Read/Write Multiple Datasets in an HDF5 file
The HDF5 library allows a data access operation to access one dataset at a time, whether access is collective or independent. However, accessing multiple datasets will require the user to issue an I/O call for each dataset. This release provides a set of new routines that allow users to access multiple datasets with a single I/O call.

New tools h5dwalk and h5delete
The new tool h5dwalk provides parallelism for improved performance while also including critical logging capabilities to capture outputs from applying the serial tools over large collections of HDF5 files.

Note that the HDF5 Release 1.14.0 is the final released version of all the features that were released in 1.13.0-1.13.3.

RFCs

Software Changes from Release to Release in HDF5 1.14