- Document's Audience:
Current H5 library designers and knowledgeable external developers.
- Background Reading:
- HDF5 File Format Specification Version 3.0
This describes the current HDF5 file format.
Introduction
What is this document about?
This document attempts to explain the HDF5 file format specification with a few examples and describes some potential improvements to the format specification.
File Format Examples
This section has several small programs and describes the format of a file created with each of them.
Example program one - Create an empty file:
#include "hdf5.h"
#include <assert.h>
int main()
{
assert(fid>=0);
assert(ret>=0);
return(0);
}
#define H5F_ACC_TRUNC
Definition H5Fpublic.h:30
int64_t hid_t
Definition H5Ipublic.h:60
#define H5P_DEFAULT
Definition H5Ppublic.h:220
int herr_t
Definition H5public.h:239
herr_t H5Fclose(hid_t file_id)
Terminates access to an HDF5 file.
hid_t H5Fcreate(const char *filename, unsigned flags, hid_t fcpl_id, hid_t fapl_id)
Creates an HDF5 file.
Super Block
| byte | byte | byte | byte |
| \211 | 'H' | 'D' | 'F' |
| \r | \n | \032 | \n |
| 0 | 0 | 0 | 0 |
| 0 | 8 | 8 | 0 |
| 4 | 16 |
| 0x00000003 |
| 0 |
0xffffffffffffffff
|
| ? |
0xffffffffffffffff
|
| 0 |
928
|
| H5G_CACHED_STAB (1) |
| 0 |
|
|
|
%h5debug example1.h5
Reading signature at address 0 (rel)
File Super Block...
File name: example1.h5
File access flags 0x00000000
File open reference count: 1
Address of super block: 0 (abs)
Size of user block: 0 bytes
Super block version number: 0
Free list version number: 0
Root group symbol table entry version number: 0
Shared header version number: 0
Size of file offsets (
haddr_t type): 8 bytes
Size of file lengths (
hsize_t type): 8 bytes
Symbol table leaf node 1/2 rank: 4
Symbol table internal node 1/2 rank: 16
File consistency flags: 0x00000003
Base address: 0 (abs)
Free list address: UNDEF (rel)
Address of driver information block: UNDEF (rel)
Root group symbol table entry:
Name offset into private heap: 0
Object header address: 928
Dirty: Yes
Cache info type: Symbol Table
Cached information:
B-tree address: 384
Heap address: 96
uint64_t hsize_t
Definition H5public.h:301
uint64_t haddr_t
Definition H5public.h:352
Root Group Object Header
| byte | byte | byte | byte |
| 1 | 0 | 2 |
| 1 |
| 32 |
| 0x0011 | 16 |
| 0x01 | 0 |
|
|
| 0 | 0 |
| 0x00 | 0 |
%h5debug example1.h5 928
New address: 928
Reading signature at address 928 (rel)
Object Header...
Dirty: 0
Version: 1
Header size (in bytes): 16
Number of links: 1
Number of messages (allocated): 2 (32)
Number of chunks (allocated): 1 (8)
Chunk 0...
Dirty: 0
Address: 944
Size in bytes: 32
Message 0...
Message ID (sequence number): 0x0011 stab(0)
Shared message: No
Constant: Yes
Raw size in obj header: 16 bytes
Chunk number: 0
Message Information:
B-tree address: 384
Name heap address: 96
Message 1...
Message ID (sequence number): 0x0000 null(0)
Shared message: No
Constant: No
Raw size in obj header: 0 bytes
Chunk number: 0
Message Information:
<no info="" for="" this="" message="">
Root Group Local Heap
| byte | byte | byte | byte |
| 'H' | 'E' | 'A' | 'P' |
| 0 |
| 256 |
| 8 |
| 128 |
%h5debug example1.h5 96
New address: 96
Reading signature at address 96 (rel)
Local Heap...
Dirty: 0
Header size (in bytes): 32
Address of heap data: 128
Data bytes allocated on disk: 256
Data bytes allocated in core: 256
Free Blocks (offset, size):
Block #0: 8, 248
Percent of heap used: 3.12%
Data follows (`__' indicates free region)...
0: 00 00 00 00 00 00 00 00 __ __ __ __ __ __ __ __ ........
16: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
32: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
48: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
64: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
80: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
96: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
112: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
128: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
144: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
160: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
176: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
192: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
208: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
224: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
240: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
Root Group B-tree
| byte | byte | byte | byte |
| 'T' | 'R' | 'E' | 'E' |
| 0 | 0 | 0 |
0xffffffffffffffff
|
0xffffffffffffffff
|
%h5debug example1.h5 384 96
New address: 384
Reading signature at address 384 (rel)
Tree type ID: H5B_SNODE_ID
Size of node: 544
Size of raw (disk) key: 8
Dirty flag: False
Number of initial dirty children: 0
Level: 0
Address of left sibling: UNDEF
Address of right sibling: UNDEF
Number of children (max): 0 (32)
Example program two - Create a file with a single dataset in it:
#include "hdf5.h"
#include <assert.h>
int main()
{
assert(fid>=0);
assert(sid>=0);
assert(did>=0);
assert(ret>=0);
assert(ret>=0);
assert(ret>=0);
return(0);
}
@ H5S_SCALAR
Definition H5Spublic.h:81
#define H5Dcreate
Definition H5version.h:1088
herr_t H5Dclose(hid_t dset_id)
Closes the specified dataset.
herr_t H5Sclose(hid_t space_id)
Releases and terminates access to a dataspace.
hid_t H5Screate(H5S_class_t type)
Creates a new dataspace of a specified type.
#define H5T_NATIVE_INT
Definition H5Tpublic.h:813
Super Block
| byte | byte | byte | byte |
| \211 | 'H' | 'D' | 'F' |
| \r | \n | \032 | \n |
| 0 | 0 | 0 | 0 |
| 0 | 8 | 8 | 0 |
| 4 | 16 |
| 0x00000003 |
| 0 |
0xffffffffffffffff
|
| ? |
0xffffffffffffffff
|
| 0 |
928
|
| H5G_CACHED_STAB (1) |
| 0 |
|
|
|
%h5debug example2.h5
Reading signature at address 0 (rel)
File Super Block...
File name: example2.h5
File access flags 0x00000000
File open reference count: 1
Address of super block: 0 (abs)
Size of user block: 0 bytes
Super block version number: 0
Free list version number: 0
Root group symbol table entry version number: 0
Shared header version number: 0
Size of file offsets (
haddr_t type): 8 bytes
Size of file lengths (
hsize_t type): 8 bytes
Symbol table leaf node 1/2 rank: 4
Symbol table internal node 1/2 rank: 16
File consistency flags: 0x00000003
Base address: 0 (abs)
Free list address: UNDEF (rel)
Address of driver information block: UNDEF (rel)
Root group symbol table entry:
Name offset into private heap: 0
Object header address: 928
Dirty: Yes
Cache info type: Symbol Table
Cached entry information:
B-tree address: 384
Heap address: 96
Root Group Object Header
| byte | byte | byte | byte |
| 1 | 0 | 2 |
| 1 |
| 32 |
| 0x0011 | 16 |
| 0x01 | 0 |
|
|
| 0 | 0 |
| 0x00 | 0 |
%h5debug example2.h5 928
New address: 928
Reading signature at address 928 (rel)
Object Header...
Dirty: 0
Version: 1
Header size (in bytes): 16
Number of links: 1
Number of messages (allocated): 2 (32)
Number of chunks (allocated): 1 (8)
Chunk 0...
Dirty: 0
Address: 944
Size in bytes: 32
Message 0...
Message ID: 0x0011 stab(0)
Shared message: No
Constant: Yes
Raw size in obj header: 16 bytes
Chunk number: 0
Message Information:
B-tree address: 384
Name heap address: 96
Message 1...
Message ID: 0x0000 null(0)
Shared message: No
Constant: No
Raw size in obj header: 0 bytes
Chunk number: 0
Message Information:
<no info="" for="" this="" message="">
Root Group Local Heap
| byte | byte | byte | byte |
| 'H' | 'E' | 'A' | 'P' |
| 0 |
| 256 |
| 16 |
| 128 |
%h5debug example2.h5 96
New address: 96
Reading signature at address 96 (rel)
Local Heap...
Dirty: 0
Header size (in bytes): 32
Address of heap data: 128
Data bytes allocated on disk: 256
Data bytes allocated in core: 256
Free Blocks (offset, size):
Block #0: 16, 240
Percent of heap used: 6.25%
Data follows (`__' indicates free region)...
0: 00 00 00 00 00 00 00 00 44 61 74 61 73 65 74 00 ........Dataset.
16: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
32: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
48: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
64: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
80: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
96: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
112: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
128: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
144: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
160: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
176: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
192: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
208: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
224: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
240: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
Root Group B-tree
| byte | byte | byte | byte |
| 'T' | 'R' | 'E' | 'E' |
| 0 | 0 | 1 |
0xffffffffffffffff
|
0xffffffffffffffff
|
0
|
1248
|
8
|
%h5debug example2.h5 384 96
New address: 384
Reading signature at address 384 (rel)
Tree type ID: H5B_SNODE_ID
Size of node: 544
Size of raw (disk) key: 8
Dirty flag: False
Number of initial dirty children: 0
Level: 0
Address of left sibling: UNDEF
Address of right sibling: UNDEF
Number of children (max): 1 (32)
Child 0...
Address: 1248
Left Key:
Heap offset: 0
Name :
Right Key:
Heap offset: 8
Name : Dataset
Root Group B-tree Symbol Table Node
| byte | byte | byte | byte |
| 'S' | 'N' | 'O' | 'D' |
| 1 | 0 | 1 |
|
|
%h5debug example2.h5 1248 96
New address: 1248
Reading signature at address 1248 (rel)
Symbol Table Node...
Dirty: No
Size of Node (in bytes): 328
Number of Symbols: 1 of 8
Symbol 0:
Name: `Dataset'
Name offset into private heap: 8
Object header address: 976
Dirty: No
Cache info type: Nothing Cached
'/Dataset' Object Header
| byte | byte | byte | byte |
| Version: 1 | Reserved: 0 | Number of Header Messages: 6 |
| Object Reference Count: 1 |
Total Object Header Size: 256
|
Fill Value Header Message
| | | |
Message Type: 0x0005
| Message Data Size: 8 |
Flags: 0x01
| Reserved: 0 |
Version: 1
| Space Allocation Time: 2 (Late)
| Fill Value Writing Time: 0 (At allocation)
| Fill Value Defined: 0 (Undefined) |
| Fill Value Datatype Size: 0 (Use dataset's datatype for fill-value datatype) |
|
Datatype Header Message
| | | |
Message Type: 0x0003
| Message Data Size: 16 |
Flags: 0x01
<tdcolspan="3">Reserved: 0
|
| Version: 0x1 | Class: 0x0 (Fixed-Point) |
| Fixed-Point Bit-Field: 0x08 (Little-endian, No padding, Signed) |
| Size: 4 |
Bit Offset: 0
| Bit Precision: 32 |
| Message Alignment Filler: - |
|
Dataspace Header Message
| | | |
Message Type: 0x0001
| Message Data Size: 8 |
Flags: 0x00
| Reserved: 0 |
Version: 1
| Rank: 0 (Scalar)
| Flags: 0x00 (No maximum dimensions, no permutation information)
| Reserved: 0 |
| Reserved: 0 |
|
Layout Header Message
| | | |
Message Type: 0x0008
| Message Data Size: 24 |
Flags: 0x00
| Reserved: 0 |
Version: 1
| Rank: 1 (Dataspace rank+1)
| Class: 1 (Contiguous)
| Reserved: 0 |
| Reserved: 0 |
Address: 0xffffffffffffffff (Undefined)
|
| Dimension 0 Size: 4 (Datatype size) |
| Message Alignment Filler: - |
|
Modification Date & Time Header Message
| | | |
Message Type: 0x0012
| Message Data Size: 8 |
Flags: 0x00
| Reserved: 0 |
Version: 1
| Reserved: 0 |
| Seconds Since Epoch: 1052401700 (2003-05-08 08:48:20 CDT) |
|
Null Header Message
| | | |
| Message Type: 0x0000 | Message Data Size: 144 |
| Flags: 0x00 | Reserved: 0 |
|
%h5debug example2.h5 976
New address: 976
Reading signature at address 976 (rel)
Object Header...
Dirty: 0
Version: 1
Header size (in bytes): 16
Number of links: 1
Number of messages (allocated): 6 (32)
Number of chunks (allocated): 1 (8)
Chunk 0...
Dirty: 0
Address: 992
Size in bytes: 256
Message 0...
Message ID (sequence number): 0x0005 `fill_new' (0)
Shared: No
Constant: Yes
Raw size in obj header: 8 bytes
Chunk number: 0
Message Information:
Version: 1
Space Allocation Time: Late
Fill Time: On Allocation
Fill Value Defined: Undefined
Size: 0
Data type: <dataset type="">
Message 1...
Message ID (sequence number): 0x0003 data_type(0)
Shared message: No
Constant: Yes
Raw size in obj header: 16 bytes
Chunk number: 0
Message Information:
Type class: integer
Size: 4 bytes
Byte order: little endian
Precision: 32 bits
Offset: 0 bits
Low pad type: zero
High pad type: zero
Sign scheme: 2's comp
Message 2...
Message ID (sequence number): 0x0001 simple_dspace(0)
Shared message: No
Constant: No
Raw size in obj header: 8 bytes
Chunk number: 0
Message Information:
Rank: 0
Message 3...
Message ID (sequence number): 0x0008 layout(0)
Shared message: No
Constant: No
Raw size in obj header: 24 bytes
Chunk number: 0
Message Information:
Data address: UNDEF
Number of dimensions: 1
Size: {4}
Message 4...
Message ID (sequence number): 0x0012 mtime_new(0)
Shared message: No
Constant: No
Raw size in obj header: 8 bytes
Chunk number: 0
Message Information:
Time: 2003-03-05 14:52:00 CST
Message 5...
Message ID (sequence number): 0x0000 null(0)
Shared message: No
Constant: No
Raw size in obj header: 144 bytes
Chunk number: 0
Message Information:
<no info="" for="" this="" message="">
1.9.7 