/** \page FMTDISC HDF5 File Format Discussion Navigate back: \ref index "Main" / \ref TN
Document's Audience:
Current H5 library designers and knowledgeable external developers.
Background Reading:
\ref FMT4
This describes the current HDF5 file format.
\section sec_fmtdisc_intro Introduction What is this document about?
This document attempts to explain the HDF5 file format specification with a few examples and describes some potential improvements to the format specification. \section sec_fmtdisc_exam File Format Examples This section has several small programs and describes the format of a file created with each of them. Example program one - Create an empty file: \code #include "hdf5.h" #include int main() { hid_t fid; /* File ID */ herr_t ret; /* Generic return value */ /* Create the file */ fid=H5Fcreate("example1.h5", H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT); assert(fid>=0); /* Close the file */ ret=H5Fclose(fid); assert(ret>=0); return(0); } \endcode
Super Block
byte byte byte byte
\211 'H' 'D' 'F'
\\r \\n \032 \\n
0 0 0 0
0 8 8 0
4 16
0x00000003
0

0xffffffffffffffff
?

0xffffffffffffffff
0

928
H5G_CACHED_STAB (1)
0

384

96
\code %h5debug example1.h5 Reading signature at address 0 (rel) File Super Block... File name: example1.h5 File access flags 0x00000000 File open reference count: 1 Address of super block: 0 (abs) Size of user block: 0 bytes Super block version number: 0 Free list version number: 0 Root group symbol table entry version number: 0 Shared header version number: 0 Size of file offsets (haddr_t type): 8 bytes Size of file lengths (hsize_t type): 8 bytes Symbol table leaf node 1/2 rank: 4 Symbol table internal node 1/2 rank: 16 File consistency flags: 0x00000003 Base address: 0 (abs) Free list address: UNDEF (rel) Address of driver information block: UNDEF (rel) Root group symbol table entry: Name offset into private heap: 0 Object header address: 928 Dirty: Yes Cache info type: Symbol Table Cached information: B-tree address: 384 Heap address: 96 \endcode
Root Group Object Header
byte byte byte byte
1 0 2
1
32
0x0011 16
0x01 0

384

96
0 0
0x00 0
\code %h5debug example1.h5 928 New address: 928 Reading signature at address 928 (rel) Object Header... Dirty: 0 Version: 1 Header size (in bytes): 16 Number of links: 1 Number of messages (allocated): 2 (32) Number of chunks (allocated): 1 (8) Chunk 0... Dirty: 0 Address: 944 Size in bytes: 32 Message 0... Message ID (sequence number): 0x0011 stab(0) Shared message: No Constant: Yes Raw size in obj header: 16 bytes Chunk number: 0 Message Information: B-tree address: 384 Name heap address: 96 Message 1... Message ID (sequence number): 0x0000 null(0) Shared message: No Constant: No Raw size in obj header: 0 bytes Chunk number: 0 Message Information: \endcode
Root Group Local Heap
byte byte byte byte
'H' 'E' 'A' 'P'
0
256
8
128
\code %h5debug example1.h5 96 New address: 96 Reading signature at address 96 (rel) Local Heap... Dirty: 0 Header size (in bytes): 32 Address of heap data: 128 Data bytes allocated on disk: 256 Data bytes allocated in core: 256 Free Blocks (offset, size): Block #0: 8, 248 Percent of heap used: 3.12% Data follows (`__' indicates free region)... 0: 00 00 00 00 00 00 00 00 __ __ __ __ __ __ __ __ ........ 16: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 32: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 48: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 64: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 80: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 96: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 112: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 128: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 144: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 160: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 176: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 192: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 208: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 224: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 240: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ \endcode
Root Group B-tree
byte byte byte byte
'T' 'R' 'E' 'E'
0 0 0

0xffffffffffffffff

0xffffffffffffffff
\code %h5debug example1.h5 384 96 New address: 384 Reading signature at address 384 (rel) Tree type ID: H5B_SNODE_ID Size of node: 544 Size of raw (disk) key: 8 Dirty flag: False Number of initial dirty children: 0 Level: 0 Address of left sibling: UNDEF Address of right sibling: UNDEF Number of children (max): 0 (32) \endcode Example program two - Create a file with a single dataset in it: \code #include "hdf5.h" #include int main() { hid_t fid; /* File ID */ hid_t sid; /* Dataspace ID */ hid_t did; /* Dataset ID */ herr_t ret; /* Generic return value */ /* Create the file */ fid=H5Fcreate("example2.h5", H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT); assert(fid>=0); /* Create a scalar dataspace for the dataset */ sid=H5Screate(H5S_SCALAR); assert(sid>=0); /* Create a trivial dataset */ did=H5Dcreate(fid, "Dataset", H5T_NATIVE_INT, sid, H5P_DEFAULT); assert(did>=0); /* Close the dataset */ ret=H5Dclose(did); assert(ret>=0); /* Close the dataspace */ ret=H5Sclose(sid); assert(ret>=0); /* Close the file */ ret=H5Fclose(fid); assert(ret>=0); return(0); } \endcode
Super Block
byte byte byte byte
\211 'H' 'D' 'F'
\\r \\n \032 \\n
0 0 0 0
0 8 8 0
4 16
0x00000003
0

0xffffffffffffffff
?

0xffffffffffffffff
0

928
H5G_CACHED_STAB (1)
0

384

96
\code %h5debug example2.h5 Reading signature at address 0 (rel) File Super Block... File name: example2.h5 File access flags 0x00000000 File open reference count: 1 Address of super block: 0 (abs) Size of user block: 0 bytes Super block version number: 0 Free list version number: 0 Root group symbol table entry version number: 0 Shared header version number: 0 Size of file offsets (haddr_t type): 8 bytes Size of file lengths (hsize_t type): 8 bytes Symbol table leaf node 1/2 rank: 4 Symbol table internal node 1/2 rank: 16 File consistency flags: 0x00000003 Base address: 0 (abs) Free list address: UNDEF (rel) Address of driver information block: UNDEF (rel) Root group symbol table entry: Name offset into private heap: 0 Object header address: 928 Dirty: Yes Cache info type: Symbol Table Cached entry information: B-tree address: 384 Heap address: 96 \endcode
Root Group Object Header
byte byte byte byte
1 0 2
1
32
0x0011 16
0x01 0

384

96
0 0
0x00 0
\code %h5debug example2.h5 928 New address: 928 Reading signature at address 928 (rel) Object Header... Dirty: 0 Version: 1 Header size (in bytes): 16 Number of links: 1 Number of messages (allocated): 2 (32) Number of chunks (allocated): 1 (8) Chunk 0... Dirty: 0 Address: 944 Size in bytes: 32 Message 0... Message ID: 0x0011 stab(0) Shared message: No Constant: Yes Raw size in obj header: 16 bytes Chunk number: 0 Message Information: B-tree address: 384 Name heap address: 96 Message 1... Message ID: 0x0000 null(0) Shared message: No Constant: No Raw size in obj header: 0 bytes Chunk number: 0 Message Information: \endcode
Root Group Local Heap
byte byte byte byte
'H' 'E' 'A' 'P'
0
256
16
128
\code %h5debug example2.h5 96 New address: 96 Reading signature at address 96 (rel) Local Heap... Dirty: 0 Header size (in bytes): 32 Address of heap data: 128 Data bytes allocated on disk: 256 Data bytes allocated in core: 256 Free Blocks (offset, size): Block #0: 16, 240 Percent of heap used: 6.25% Data follows (`__' indicates free region)... 0: 00 00 00 00 00 00 00 00 44 61 74 61 73 65 74 00 ........Dataset. 16: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 32: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 48: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 64: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 80: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 96: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 112: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 128: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 144: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 160: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 176: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 192: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 208: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 224: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ 240: __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ \endcode
Root Group B-tree
byte byte byte byte
'T' 'R' 'E' 'E'
0 0 1

0xffffffffffffffff

0xffffffffffffffff

0

1248

8
\code %h5debug example2.h5 384 96 New address: 384 Reading signature at address 384 (rel) Tree type ID: H5B_SNODE_ID Size of node: 544 Size of raw (disk) key: 8 Dirty flag: False Number of initial dirty children: 0 Level: 0 Address of left sibling: UNDEF Address of right sibling: UNDEF Number of children (max): 1 (32) Child 0... Address: 1248 Left Key: Heap offset: 0 Name : Right Key: Heap offset: 8 Name : Dataset \endcode
Root Group B-tree Symbol Table Node
byte byte byte byte
'S' 'N' 'O' 'D'
1 0 1
8

976
0
0


0
\code %h5debug example2.h5 1248 96 New address: 1248 Reading signature at address 1248 (rel) Symbol Table Node... Dirty: No Size of Node (in bytes): 328 Number of Symbols: 1 of 8 Symbol 0: Name: `Dataset' Name offset into private heap: 8 Object header address: 976 Dirty: No Cache info type: Nothing Cached \endcode
'/Dataset' Object Header
byte byte byte byte
Version: 1 Reserved: 0 Number of Header Messages: 6
Object Reference Count: 1
Total Object Header Size: 256
Fill Value Header Message
Message Type: 0x0005 Message Data Size: 8
Flags: 0x01 Reserved: 0
Version: 1 Space Allocation Time: 2 (Late) Fill Value Writing Time: 0 (At allocation) Fill Value Defined: 0 (Undefined)
Fill Value Datatype Size: 0 (Use dataset's datatype for fill-value datatype)
Datatype Header Message
Message Type: 0x0003 Message Data Size: 16
Flags: 0x01 Reserved: 0
Version: 0x1 Class: 0x0 (Fixed-Point)
Fixed-Point Bit-Field: 0x08 (Little-endian, No padding, Signed)
Size: 4
Bit Offset: 0 Bit Precision: 32
Message Alignment Filler: -
Dataspace Header Message
Message Type: 0x0001 Message Data Size: 8
Flags: 0x00 Reserved: 0
Version: 1 Rank: 0 (Scalar) Flags: 0x00 (No maximum dimensions, no permutation information) Reserved: 0
Reserved: 0
Layout Header Message
Message Type: 0x0008 Message Data Size: 24
Flags: 0x00 Reserved: 0
Version: 1 Rank: 1 (Dataspace rank+1) Class: 1 (Contiguous) Reserved: 0
Reserved: 0

Address: 0xffffffffffffffff (Undefined)
Dimension 0 Size: 4 (Datatype size)
Message Alignment Filler: -
Modification Date & Time Header Message
Message Type: 0x0012 Message Data Size: 8
Flags: 0x00 Reserved: 0
Version: 1 Reserved: 0
Seconds Since Epoch: 1052401700 (2003-05-08 08:48:20 CDT)
Null Header Message
Message Type: 0x0000 Message Data Size: 144
Flags: 0x00 Reserved: 0
\code %h5debug example2.h5 976 New address: 976 Reading signature at address 976 (rel) Object Header... Dirty: 0 Version: 1 Header size (in bytes): 16 Number of links: 1 Number of messages (allocated): 6 (32) Number of chunks (allocated): 1 (8) Chunk 0... Dirty: 0 Address: 992 Size in bytes: 256 Message 0... Message ID (sequence number): 0x0005 `fill_new' (0) Shared: No Constant: Yes Raw size in obj header: 8 bytes Chunk number: 0 Message Information: Version: 1 Space Allocation Time: Late Fill Time: On Allocation Fill Value Defined: Undefined Size: 0 Data type: Message 1... Message ID (sequence number): 0x0003 data_type(0) Shared message: No Constant: Yes Raw size in obj header: 16 bytes Chunk number: 0 Message Information: Type class: integer Size: 4 bytes Byte order: little endian Precision: 32 bits Offset: 0 bits Low pad type: zero High pad type: zero Sign scheme: 2's comp Message 2... Message ID (sequence number): 0x0001 simple_dspace(0) Shared message: No Constant: No Raw size in obj header: 8 bytes Chunk number: 0 Message Information: Rank: 0 Message 3... Message ID (sequence number): 0x0008 layout(0) Shared message: No Constant: No Raw size in obj header: 24 bytes Chunk number: 0 Message Information: Data address: UNDEF Number of dimensions: 1 Size: {4} Message 4... Message ID (sequence number): 0x0012 mtime_new(0) Shared message: No Constant: No Raw size in obj header: 8 bytes Chunk number: 0 Message Information: Time: 2003-03-05 14:52:00 CST Message 5... Message ID (sequence number): 0x0000 null(0) Shared message: No Constant: No Raw size in obj header: 144 bytes Chunk number: 0 Message Information: \endcode
Navigate back: \ref index "Main" / \ref TN */