VOL Mapping (H5M)

Detailed Description

The interface can only be used with the HDF5 VOL connectors that implement map objects. The native HDF5 library does not support this feature.

While the HDF5 data model is a flexible way to store data, some applications require a more general way to index information. HDF5 effectively uses key-value stores internally for a variety of purposes, but it does not expose a generic key-value store to the API. The Map APIs provide this capability to the HDF5 applications in the form of HDF5 map objects. These Map objects contain application-defined key-value stores, to which key-value pairs can be added, and from which values can be retrieved by key.

HDF5 VOL connectors with support for map objects:

• DAOS

Example:

hid_t file_id, fapl_id, map_id, vls_type_id;
const char *names[2] = ["Alice", "Bob"];
uint64_t IDs[2] = [25385486, 34873275];
uint64_t val_out;
 
<HDF5 VOL setup code ....>
 
vls_type_id = H5Tcopy(H5T_C_S1);
H5Tset_size(vls_type_id, H5T_VARIABLE);
file_id = H5Fcreate("file.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapl_id);
map_id = H5Mcreate(file_id, "map", vls_type_id, H5T_NATIVE_UINT64, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
H5Mput(map_id, vls_type_id, &names[0], H5T_NATIVE_UINT64, &IDs[0], H5P_DEFAULT);
H5Mput(map_id, vls_type_id, &names[1], H5T_NATIVE_UINT64, &IDs[1], H5P_DEFAULT);
H5Mget(map_id, vls_type_id, &names[0], H5T_NATIVE_UINT64, &val_out, H5P_DEFAULT);
if(val_out != IDs[0])
  ERROR;
H5Mclose(map_id);
H5Tclose(vls_type_id);
H5Fclose(file_id);

Functions
hid_t	H5Mcreate (hid_t loc_id, const char *name, hid_t key_type_id, hid_t val_type_id, hid_t lcpl_id, hid_t mcpl_id, hid_t mapl_id)
	Creates a map object.
hid_t	H5Mcreate_anon (hid_t loc_id, hid_t key_type_id, hid_t val_type_id, hid_t mcpl_id, hid_t mapl_id)
	Creates a map object without linking it into a file.
hid_t	H5Mopen (hid_t loc_id, const char *name, hid_t mapl_id)
	Opens a map object.
herr_t	H5Mclose (hid_t map_id)
	Terminates access to a map object.
hid_t	H5Mget_key_type (hid_t map_id)
	Gets key datatype for a map object.
hid_t	H5Mget_val_type (hid_t map_id)
	Gets value datatype for a map object.
hid_t	H5Mget_create_plist (hid_t map_id)
	Gets creation property list for a map object.
hid_t	H5Mget_access_plist (hid_t map_id)
	Gets access property list for a map object.
herr_t	H5Mget_count (hid_t map_id, hsize_t *count, hid_t dxpl_id)
	Retrieves the number of key-value pairs in a map object.
herr_t	H5Mput (hid_t map_id, hid_t key_mem_type_id, const void *key, hid_t val_mem_type_id, const void *value, hid_t dxpl_id)
	Adds a key-value pair to a map object.
herr_t	H5Mget (hid_t map_id, hid_t key_mem_type_id, const void *key, hid_t val_mem_type_id, void *value, hid_t dxpl_id)
	Retrieves a key-value pair from a map object.
herr_t	H5Mexists (hid_t map_id, hid_t key_mem_type_id, const void *key, hbool_t *exists, hid_t dxpl_id)
	Checks if provided key exists in a map object.
herr_t	H5Miterate (hid_t map_id, hsize_t *idx, hid_t key_mem_type_id, H5M_iterate_t op, void *op_data, hid_t dxpl_id)
	Iterates over all key-value pairs in a map object.
herr_t	H5Miterate_by_name (hid_t loc_id, const char *map_name, hsize_t *idx, hid_t key_mem_type_id, H5M_iterate_t op, void *op_data, hid_t dxpl_id, hid_t lapl_id)
	Iterates over all key-value pairs in a map object.
herr_t	H5Mdelete (hid_t map_id, hid_t key_mem_type_id, const void *key, hid_t dxpl_id)
	Deletes a key-value pair from a map object.

Function Documentation

H5Mclose()

herr_t H5Mclose

(

hid_t

map_id

)

Terminates access to a map object.

Parameters

[in]

map_id

Map identifier

Returns

Returns a non-negative value if successful; otherwise, returns a negative value.

H5Mclose() closes access to a map object specified by map_id and releases resources used by it.

It is illegal to subsequently use that same map identifier in calls to other map functions.

Since

1.12.0

H5Mcreate()

hid_t H5Mcreate	(	hid_t	loc_id,
		const char *	name,
		hid_t	key_type_id,
		hid_t	val_type_id,
		hid_t	lcpl_id,
		hid_t	mcpl_id,
		hid_t	mapl_id
	)

Creates a map object.

Parameters

[in]	loc_id	Location identifier. The identifier may be that of a file, group, dataset, named datatype, or attribute.
[in]	name	Map object name
[in]	key_type_id	Datatype identifier
[in]	val_type_id	Datatype identifier
[in]	lcpl_id	Link creation property list identifier
[in]	mcpl_id	Map creation property list identifier
[in]	mapl_id	Map access property list identifier

Returns

Returns a map object identifier if successful; otherwise returns H5I_INVALID_HID.

H5Mcreate() creates a new map object for storing key-value pairs. The in-file datatype for keys is defined by key_type_id and the in-file datatype for values is defined by val_type_id. loc_id specifies the location to create the map object and name specifies the name of the link to the map object relative to loc_id.

Since

1.12.0

H5Mcreate_anon()

hid_t H5Mcreate_anon	(	hid_t	loc_id,
		hid_t	key_type_id,
		hid_t	val_type_id,
		hid_t	mcpl_id,
		hid_t	mapl_id
	)

Creates a map object without linking it into a file.

Parameters

[in]	loc_id	Location identifier. The identifier may be that of a file, group, dataset, named datatype, or attribute.
[in]	key_type_id	Datatype identifier
[in]	val_type_id	Datatype identifier
[in]	mcpl_id	Map creation property list identifier
[in]	mapl_id	Map access property list identifier

Returns

Returns a map object identifier if successful; otherwise returns H5I_INVALID_HID. The resulting ID should be linked into the file with H5Olink or it will be deleted when closed.

H5Mcreate_anon() creates a new map object for storing key-value pairs. The in-file datatype for keys is defined by key_type_id and the in-file datatype for values is defined by val_type_id. loc_id specifies the file to create the map object, but no link to the object is created. Other options can be specified through the property lists mcpl_id and mapl_id.

The new map should be linked into the group hierarchy before being closed or it will be deleted. The map should be closed when the caller no longer requires it.

Since

1.12.0

H5Mdelete()

herr_t H5Mdelete	(	hid_t	map_id,
		hid_t	key_mem_type_id,
		const void *	key,
		hid_t	dxpl_id
	)

Deletes a key-value pair from a map object.

Parameters

[in]	map_id	Map identifier
[in]	key_mem_type_id	Datatype identifier
[in]	key	Pointer to key buffer
[in]	dxpl_id	Dataset transfer property list identifier

Returns

Returns a non-negative value if successful; otherwise, returns a negative value.

H5Mdelete() deletes a key-value pair from the map object specified by map_id. key_mem_type_id specifies the datatype for the provided key buffer key, and if different from that used to create the map object, the key will be internally converted to the datatype for the map object.

Any further options can be specified through the property list dxpl_id.

Since

1.12.0

H5Mexists()

herr_t H5Mexists	(	hid_t	map_id,
		hid_t	key_mem_type_id,
		const void *	key,
		hbool_t *	exists,
		hid_t	dxpl_id
	)

Checks if provided key exists in a map object.

Parameters

[in]	map_id	Map identifier
[in]	key_mem_type_id	Datatype identifier
[in]	key	Pointer to key buffer
[out]	exists	Pointer to a buffer to return the existence status
[in]	dxpl_id	Dataset transfer property list identifier

Returns

Returns a non-negative value if successful; otherwise, returns a negative value.

H5Mexists() checks if the provided key is stored in the map object specified by map_id. If key_mem_type_id is different from that used to create the map object the key will be internally converted to the datatype for the map object for the query.

Any further options can be specified through the property list dxpl_id.

Since

1.12.0

H5Mget()

herr_t H5Mget	(	hid_t	map_id,
		hid_t	key_mem_type_id,
		const void *	key,
		hid_t	val_mem_type_id,
		void *	value,
		hid_t	dxpl_id
	)

Retrieves a key-value pair from a map object.

Parameters

[in]	map_id	Map identifier
[in]	key_mem_type_id	Datatype identifier
[in]	key	Pointer to key buffer
[in]	val_mem_type_id	Datatype identifier
[out]	value	Pointer to value buffer
[in]	dxpl_id	Dataset transfer property list identifier

Returns

Returns a non-negative value if successful; otherwise, returns a negative value.

H5Mget() retrieves from a map object specified by map_id, the value associated with the provided key key. key_mem_type_id and val_mem_type_id specify the datatypes for the provided key and value buffers. If if the datatype specified by key_mem_type_id is different from that used to create the map object the key will be internally converted to the datatype for the map object for the query, and if the datatype specified by val_mem_type_id is different from that used to create the map object the returned value will be converted to have a datatype as specified by val_mem_type_id before the function returns.

Any further options can be specified through the property list dxpl_id.

Since

1.12.0

H5Mget_access_plist()

hid_t H5Mget_access_plist

(

hid_t

map_id

)

Gets access property list for a map object.

Parameters

[in]

map_id

Map identifier

Returns

Returns a map access property list identifier if successful; otherwise returns H5I_INVALID_HID.

H5Mget_access_plist() returns an identifier for a copy of the access property list for a map object specified by map_id.

Since

1.12.0

H5Mget_count()

herr_t H5Mget_count	(	hid_t	map_id,
		hsize_t *	count,
		hid_t	dxpl_id
	)

Retrieves the number of key-value pairs in a map object.

Parameters

[in]	map_id	Map identifier
[out]	count	The number of key-value pairs stored in the map object
[in]	dxpl_id	Dataset transfer property list identifier

Returns

Returns a non-negative value if successful; otherwise, returns a negative value.

H5Mget_count() retrieves the number of key-value pairs stored in a map specified by map_id.

Since

1.12.0

H5Mget_create_plist()

hid_t H5Mget_create_plist

(

hid_t

map_id

)

Gets creation property list for a map object.

Parameters

[in]

map_id

Map identifier

Returns

Returns a map creation property list identifier if successful; otherwise returns H5I_INVALID_HID.

H5Mget_create_plist() returns an identifier for a copy of the creation property list for a map object specified by map_id.

The creation property list identifier should be released with H5Pclose() to prevent resource leaks.

Since

1.12.0

H5Mget_key_type()

hid_t H5Mget_key_type

(

hid_t

map_id

)

Gets key datatype for a map object.

Parameters

[in]

map_id

Map identifier

Returns

Returns a datatype identifier if successful; otherwise returns H5I_INVALID_HID.

H5Mget_key_type() retrieves key datatype as stored in the file for a map object specified by map_id and returns identifier for the datatype.

Since

1.12.0

H5Mget_val_type()

hid_t H5Mget_val_type

(

hid_t

map_id

)

Gets value datatype for a map object.

Parameters

[in]

map_id

Map identifier

Returns

Returns a datatype identifier if successful; otherwise returns H5I_INVALID_HID.

H5Mget_val_type() retrieves value datatype as stored in the file for a map object specified by map_id and returns identifier for the datatype .

Since

1.12.0

H5Miterate()

herr_t H5Miterate	(	hid_t	map_id,
		hsize_t *	idx,
		hid_t	key_mem_type_id,
		H5M_iterate_t	op,
		void *	op_data,
		hid_t	dxpl_id
	)

Iterates over all key-value pairs in a map object.

Parameters

[in]	map_id	Map identifier
[in,out]	idx	iteration index
[in]	key_mem_type_id	Datatype identifier
[in]	op	User-defined iterator function
[in,out]	op_data	User-defined callback function context
[in]	dxpl_id	Dataset transfer property list identifier

Returns

Returns a non-negative value if successful; otherwise, returns a negative value.

H5Miterate() iterates over all key-value pairs stored in the map object specified by map_id, making the callback specified by op for each. The idx parameter is an in/out parameter that may be used to restart a previously interrupted iteration. At the start of iteration idx should be set to 0, and to restart iteration at the same location on a subsequent call to H5Miterate(), idx should be the same value as returned by the previous call. Iterate callback is defined as:

 
typedef herr_t (*H5M_iterate_t)(hid_t map_id, const void *key, void *op_data);

The key parameter is the buffer for the key for this iteration, converted to the datatype specified by key_mem_type_id. The op_data parameter is a simple pass through of the value passed to H5Miterate(), which can be used to store application-defined data for iteration. A negative return value from this function will cause H5Miterate() to issue an error, while a positive return value will cause H5Miterate() to stop iterating and return this value without issuing an error. A return value of zero allows iteration to continue.

Any further options can be specified through the property list dxpl_id.

Warning

Adding or removing key-value pairs to the map during iteration will lead to undefined behavior.

Attention

Leaving callback functions:
The callback function must return normally, even in the case of error. Returning with H5_ITER_ERROR, instead of leaving by means of exceptions, exit() function, etc... will allow the HDF5 library to manage its resources and maintain a consistent state. See C++ Developers using HDF5 C-API functions warning for detail.

Since

1.12.0

H5Miterate_by_name()

herr_t H5Miterate_by_name	(	hid_t	loc_id,
		const char *	map_name,
		hsize_t *	idx,
		hid_t	key_mem_type_id,
		H5M_iterate_t	op,
		void *	op_data,
		hid_t	dxpl_id,
		hid_t	lapl_id
	)

Iterates over all key-value pairs in a map object.

Parameters

[in]	loc_id	Location identifier
[in]	map_name	Map object name relative to the location specified by loc_id
[in,out]	idx	Iteration index
[in]	key_mem_type_id	Datatype identifier
[in]	op	User-defined iterator function
[in,out]	op_data	User-defined callback function context
[in]	dxpl_id	Dataset transfer property list identifier
[in]	lapl_id	Link access property list identifier

Returns

Returns a non-negative value if successful; otherwise, returns a negative value.

H5Miterate_by_name() iterates over all key-value pairs stored in the map object specified by map_id, making the callback specified by op for each. The idx parameter is an in/out parameter that may be used to restart a previously interrupted iteration. At the start of iteration idx should be set to 0, and to restart iteration at the same location on a subsequent call to H5Miterate(), idx should be the same value as returned by the previous call. Iterate callback is defined as:

 
typedef herr_t (*H5M_iterate_t)(hid_t map_id, const void *key, void *op_data);

Thekey parameter is the buffer for the key for this iteration, converted to the datatype specified by key_mem_type_id. The op_data parameter is a simple pass through of the value passed to H5Miterate(), which can be used to store application-defined data for iteration. A negative return value from this function will cause H5Miterate() to issue an error, while a positive return value will cause H5Miterate() to stop iterating and return this value without issuing an error. A return value of zero allows iteration to continue.

Any further options can be specified through the property list dxpl_id.

Warning

Adding or removing key-value pairs to the map during iteration will lead to undefined behavior.

Attention

Leaving callback functions:
The callback function must return normally, even in the case of error. Returning with H5_ITER_ERROR, instead of leaving by means of exceptions, exit() function, etc... will allow the HDF5 library to manage its resources and maintain a consistent state. See C++ Developers using HDF5 C-API functions warning for detail.

Since

1.12.0

H5Mopen()

hid_t H5Mopen	(	hid_t	loc_id,
		const char *	name,
		hid_t	mapl_id
	)

Opens a map object.

Parameters

[in]	loc_id	Location identifier. The identifier may be that of a file, group, dataset, named datatype, or attribute.
[in]	name	Map object name relative to loc_id
[in]	mapl_id	Map access property list identifier

Returns

Returns a map object identifier if successful; otherwise returns H5I_INVALID_HID.

H5Mopen() finds a map object specified by name under the location specified by loc_id. The map object should be close with H5Mclose() when the application is not longer interested in accessing it.

Since

1.12.0

H5Mput()

herr_t H5Mput	(	hid_t	map_id,
		hid_t	key_mem_type_id,
		const void *	key,
		hid_t	val_mem_type_id,
		const void *	value,
		hid_t	dxpl_id
	)

Adds a key-value pair to a map object.

Parameters

[in]	map_id	Map identifier
[in]	key_mem_type_id	Datatype identifier
[in]	key	Pointer to key buffer
[in]	val_mem_type_id	Datatype identifier
[in]	value	Pointer to value buffer
[in]	dxpl_id	Dataset transfer property list identifier

Returns

Returns a non-negative value if successful; otherwise, returns a negative value.

H5Mput() adds a key-value pair to a map object specified by map_id, or updates the value for the specified key if one was set previously.

key_mem_type_id and val_mem_type_id specify the datatypes for the provided key and value buffers, and if different from those used to create the map object, the key and value will be internally converted to the datatypes for the map object.

Any further options can be specified through the property list dxpl_id.

Since

1.12.0