mirror of
https://github.com/HDFGroup/hdf5.git
synced 2026-09-25 04:09:44 +03:00
1662 lines
117 KiB
Markdown
1662 lines
117 KiB
Markdown
# HDF5 History
|
|
============
|
|
|
|
This file contains development history of the HDF5 2.X.X releases
|
|
|
|
04. Release Information for hdf5-2.2.0
|
|
03. Release Information for hdf5-2.1.1
|
|
02. Release Information for hdf5-2.1.0
|
|
01. Release Information for hdf5-2.0.0
|
|
|
|
[Search on the string '%%%%' for section breaks of each release.]
|
|
|
|
%%%%2.2.0%%%%
|
|
|
|
HDF5 version 2.2.0 released on 2026-07-28
|
|
|
|
---
|
|
|
|
# 🔺 HDF5 Changelog
|
|
All notable changes to this project will be documented in this file. This document describes the differences between this release and the previous
|
|
HDF5 release, platforms tested, and known problems in this release.
|
|
|
|
For releases prior to version 2.0.0, please see the release.txt file and for more details check the HISTORY*.txt files in the HDF5 source.
|
|
|
|
# 🔗 Quick Links
|
|
* [HDF5 documentation](https://support.hdfgroup.org/documentation/hdf5/latest/)
|
|
* [Official HDF5 releases](https://support.hdfgroup.org/downloads/index.html)
|
|
* [Changes from Release to Release and New Features in the HDF5-2.x.y](https://support.hdfgroup.org/releases/hdf5/documentation/release_specific_info.md)
|
|
* [Getting help, questions, or comments](https://github.com/HDFGroup/hdf5#help-and-support)
|
|
|
|
## 📖 Contents
|
|
* [Executive Summary](CHANGELOG.md#execsummary)
|
|
* [Breaking Changes](CHANGELOG.md#%EF%B8%8F-breaking-changes)
|
|
* [Deprecations](CHANGELOG.md#-deprecations)
|
|
* [New Features & Improvements](CHANGELOG.md#-new-features--improvements)
|
|
* [Bug Fixes](CHANGELOG.md#-bug-fixes)
|
|
* [Support for new platforms and languages](CHANGELOG.md#-support-for-new-platforms-and-languages)
|
|
* [Platforms Tested](CHANGELOG.md#%EF%B8%8F-platforms-tested)
|
|
* [Known Problems](CHANGELOG.md#-known-problems)
|
|
|
|
# 🔆 Executive Summary: HDF5 Version 2.2.0
|
|
|
|
> [!IMPORTANT]
|
|
>
|
|
> - The format of the GitHub tag for HDF5 releases has been changed to Major.Minor.Patch, consistent with the versioning policy change to follow the Semantic Versioning Specification described in this [Wiki page](https://github.com/HDFGroup/hdf5/wiki/HDF5-Version-Numbers-and-Branch-Strategy). The previous tag format hdf5_Major_Minor_Patch that was created in addition for the 2.0.0 and 2.1.0 releases will not be continued.
|
|
> - An RPM package is not provided for this release of HDF5 as an issue with the package was found during testing. The HDF Group is investigating alternative packaging methods for future releases.
|
|
|
|
## Performance Enhancements:
|
|
|
|
- Added an I/O block cache to the ROS3 VFD to reduce the number of requests to S3 for files not using paged allocation
|
|
|
|
- Improved the performance of several tools (h5dump, h5ls, h5diff, h5repack, h5stat and h5format_convert) for specific file structures where many objects are linked to with multiple hard links
|
|
|
|
## Significant Advancements:
|
|
|
|
|
|
## Enhanced Features:
|
|
|
|
- Made several improvements to the CMake logic for handling filter libraries
|
|
|
|
## Java Enhancements:
|
|
|
|
- Java dependency JAR paths are now configurable CMake cache variables, allowing system-provided JARs to be used in place of the bundled copies.
|
|
|
|
|
|
## Acknowledgements:
|
|
|
|
We would like to thank the many HDF5 community members who contributed to this release of HDF5.
|
|
|
|
# ⚠️ Breaking Changes
|
|
|
|
# 🪦 Deprecations
|
|
|
|
- The CMake variable `ZLIB_GIT_BRANCH` has been deprecated in favor of `ZLIB_GIT_TAG`
|
|
- The CMake variable `ZLIBNG_GIT_BRANCH` has been deprecated in favor of `ZLIBNG_GIT_TAG`
|
|
- The CMake variable `LIBAEC_GIT_BRANCH` has been deprecated in favor of `LIBAEC_GIT_TAG`
|
|
- The CMake variable `PLUGIN_GIT_BRANCH` has been deprecated in favor of `HDF5_FILTER_PLUGINS_GIT_TAG`
|
|
- The CMake variable `PLUGIN_GIT_URL` has been deprecated in favor of `HDF5_FILTER_PLUGINS_GIT_URL`
|
|
- The CMake variable `PLUGIN_TGZ_NAME` has been deprecated in favor of `HDF5_FILTER_PLUGINS_TGZ_NAME`
|
|
- The CMake variable `PLUGIN_TGZ_ORIGPATH` has been deprecated in favor of `HDF5_FILTER_PLUGINS_TGZ_ORIGPATH`
|
|
- The CMake variable `PLUGIN_PACKAGE_NAME` has been deprecated in favor of `HDF5_FILTER_PLUGINS_PACKAGE_NAME`
|
|
|
|
# 🚀 New Features & Improvements
|
|
|
|
## Configuration
|
|
|
|
### Consolidated documentation under docs/ directory
|
|
|
|
User-facing guides (installation, build instructions, platform-specific docs) and
|
|
Doxygen API documentation have been consolidated under a new top-level `docs/`
|
|
directory. All internal references (CMakeLists.txt, README.md, workflow files,
|
|
Doxygen sources, scripts, etc.) have been updated accordingly.
|
|
|
|
### Updated external building of zlib, zlib-ng and libaec to not use a patching process
|
|
|
|
When building these libraries from external sources while building HDF5, the library previously used a patching process to adapt the libraries to its own build process. The sources for these libraries are no longer patched and build directly from the sources of the latest upstream releases (currently, zlib 1.3.2, zlib-ng 2.3.3 and libaec 1.1.6). This also fixed an issue with the build of zlib-ng failing due to updates that were made since the last version that HDF5 was patching the sources for.
|
|
|
|
Fixes GitHub issue #6204
|
|
|
|
### Fixed an issue where CMake-built installations of zlib libraries couldn't be located on a system
|
|
|
|
An incorrect package name was being supplied to CMake's find_package() function when attempting to locate zlib libraries on the system in Config mode. The package name has been corrected and CMake-built zlib libraries can now be located.
|
|
|
|
### Fixed an issue where static zlib libraries couldn't be found on the system
|
|
|
|
The value of the HDF5 CMake variable `HDF5_USE_ZLIB_STATIC` was previously used incorrectly when locating zlib libraries on the system with CMake's find_package() function, causing it to have no effect. This has been fixed and static zlib libraries can now be located.
|
|
|
|
### Added a CMake module to locate zlib-ng for zlib support
|
|
|
|
A new `FindZLIBNG.cmake` CMake module has been added. This module is intended to locate zlib-ng on the system for zlib support in HDF5 when zlib-ng was built with Autotools instead of CMake. When zlib-ng support is enabled in HDF5 with the `HDF5_ENABLE_ZLIB_SUPPORT` and `HDF5_USE_ZLIB_NG` options, this module will first check for an existing CMake-built zlib-ng and use that if it's available. Otherwise, the module will heuristically search for zlib-ng on the system. If necessary, the module can be hinted toward a particular zlib-ng installation by setting the CMake variable `ZLIBNG_ROOT` to point to a directory.
|
|
|
|
### Added a CMake module to locate libaec for SZIP support
|
|
|
|
A new `Findlibaec.cmake` CMake module has been added. This module is intended to locate libaec on the system for SZIP support in HDF5 when libaec was built with Autotools instead of CMake. When SZIP support is enabled in HDF5 with the `HDF5_ENABLE_SZIP_SUPPORT` option, this module will first check for an existing CMake-built libaec and use that if it's available. Otherwise, the module will heuristically search for libaec on the system. If necessary, the module can be hinted toward a particular libaec installation by setting the CMake variable `libaec_ROOT` to point to a directory. If it is known that a CMake-built libaec installation exists on the system in a non-standard location, the CMake variable `libaec_DIR` can instead be set to a directory containing a `libaec-config.cmake` file to cause the module to prefer that libaec installation.
|
|
|
|
## Library
|
|
|
|
### Added an I/O block cache to the ROS3 VFD
|
|
|
|
Added an I/O block cache to the ROS3 VFD to reduce the number of requests to S3 for files that don't use paged allocation. This is a simple LRU cache that performs I/O in fixed-size blocks and serves I/O requests from the in-memory cached buffers. By default, the ROS3 VFD now performs I/O in 16 MiB (see new macro `HDF5_ROS3_VFD_DEFAULT_BLOCK_SIZE`) blocks, caching up to a total of 128 MiB (see new macro `HDF5_ROS3_VFD_DEFAULT_BLOCK_CACHE_SIZE`) of data at a time. The new `H5Pset_fapl_ros3_block_caching()` / `H5Pget_fapl_ros3_block_caching()` API functions can be used to modify or retrieve the caching parameters set on a File Access Property List, respectively. Additionally, caching of the initial bytes of a file has been delayed from file open to the first read of a file instead to reduce the overhead of file opens.
|
|
|
|
### Added optional digital signature verification for dynamically loaded plugins
|
|
|
|
When built with `-DHDF5_REQUIRE_SIGNED_PLUGINS=ON` and OpenSSL, HDF5 will cryptographically verify each plugin before loading it. Plugins are signed with the new `h5sign` tool, which appends an RSA signature and a compact footer to the plugin binary. Verification uses a keystore directory of trusted public keys, configurable at compile time (`-DHDF5_PLUGIN_KEYSTORE_DIR=<path>`) or at runtime via the `HDF5_PLUGIN_KEYSTORE` environment variable. Individual signatures can be revoked without removing the entire public key by listing their SHA-256 hashes in a `revoked_signatures.txt` file in the keystore directory. Supported algorithms include SHA-256, SHA-384, and SHA-512 with both PKCS#1 v1.5 and PSS padding. See `docs/PLUGIN_SIGNATURE_README.md` for details.
|
|
|
|
### Improve performance of H5Ovisit() with deeply nested group structures
|
|
|
|
`H5Ovisit()` would previously internally traverse each object's path name from the iteration root group in order to retrieve information about that object, causing severe performance degradation with a deeply nested group structure. Modified the algorithm to instead retrieve information directly from the object. To get this benefit, users should use `H5Ovisit3()`, or use `H5Ovisit2()` with neither `H5O_INFO_HDR` nor `H5O_INFO_META_SIZE` selected in the `fields` parameter. Performance of `H5Ocopy()`, `H5Iget_name()`, and external links with a callback set should also improve in similar situations.
|
|
|
|
### Versioned API functions now default to earliest version for older API settings
|
|
|
|
When a global API compatibility version is set (e.g., `H5_USE_16_API`), functions introduced after that version previously defaulted to their latest version, which could break applications. For example, an application using `H5_USE_16_API` that called `H5Sencode()` (introduced in 1.8, versioned in 1.12) would get `H5Sencode2()` instead of `H5Sencode1()`, potentially causing compilation or runtime failures. Versioned functions now default to their earliest (version 1) variant when the configured API level predates the function's introduction, providing maximum compatibility. See issue [#6278](https://github.com/HDFGroup/hdf5/issues/6278).
|
|
|
|
## Parallel Library
|
|
|
|
## Fortran Library
|
|
|
|
## C++ Library
|
|
|
|
## Java Library
|
|
|
|
### Java dependency JAR paths are now user-configurable
|
|
|
|
The CMake variables `HDF5_JAVA_LOGGING_JAR`, `HDF5_JAVA_LOGGING_NOP_JAR`, `HDF5_JAVA_LOGGING_SIMPLE_JAR`, `HDF5_JAVA_JUNIT_JAR`, and `HDF5_JAVA_HAMCREST_JAR` are now CMake cache variables with the bundled JARs as defaults. Users can override these at configure time to use system-provided JARs. See `INSTALL_CMake_options.md` for details.
|
|
|
|
## Tools
|
|
|
|
### Default low and high library version bounds in `h5repack` now use the HDF5 library's default
|
|
|
|
The `h5repack` tool now obtains its default low and high library version bounds from the HDF5 library's default (`H5P_FILE_ACCESS_DEFAULT`). To revert to the previous behavior, apply the `--low=0` command option.
|
|
|
|
### Added `h5sign` tool for signing plugins with RSA digital signatures
|
|
|
|
The `h5sign` command-line tool signs HDF5 plugin shared libraries by appending an RSA signature and a 14-byte footer. It supports SHA-256, SHA-384, SHA-512, and their PSS variants, and accepts passphrase-protected private keys. Use `-f` / `--force` to strip an existing signature before re-signing. The tool is built automatically when `HDF5_REQUIRE_SIGNED_PLUGINS` is enabled.
|
|
|
|
## High-Level APIs
|
|
|
|
## C Packet Table API
|
|
|
|
## Internal header file
|
|
|
|
## Documentation
|
|
|
|
|
|
# 🪲 Bug Fixes
|
|
|
|
## Library
|
|
|
|
### Fixed a possible heap leak in a utility function
|
|
|
|
A couple of unnecessary allocations in h5str_convert were never freed, causing memory leaks. These are now removed.
|
|
|
|
Fixes GitHub issue #6511
|
|
|
|
### Fixed bug that prevented internal library filters from printing error messages
|
|
|
|
Previously the error stack would be cleared when exiting a data filter, even an internal library filter, so the user could not see what caused the filter to fail. This has been fixed by not treating internal data filters like a user callback. Note that user-defined or third-party filters that use the default error stack will need to print that stack before returning from their callbacks.
|
|
|
|
### Fix NULL pointer access when H5A_operator2_t is NULL
|
|
|
|
Passing NULL for the callback function pointer to H5Aiterate2 and H5Aiterate_by_name was not detected, leading to a subsequent access of a NULL pointer. This is now fixed.
|
|
|
|
Fixes CVE-2025-9274
|
|
|
|
### Fixed error when reading variable-length chunked datasets in read-only mode
|
|
|
|
When reading from a chunked dataset with a variable-length type, a non-default fill value, and unwritten chunks, the library would internally try to write data to the file and fail due to writing to a read-only file. Reworked the I/O code to avoid these writes in this case. This may also improve performance and file space usage in similar cases with files open with write access.
|
|
|
|
### Validate free space section type during decode
|
|
|
|
When loading a free space section info block, the per-section type byte read from the file was used directly to index the free space manager's section class array and to call the class `deserialize` callback, guarded only by an assertion that is removed in release builds. A corrupted or fuzzed file could supply a type beyond the number of registered classes, causing an out-of-bounds read of the class array and an indirect call through a bogus function pointer. `H5FS__cache_sinfo_deserialize()` now rejects a section type that is not less than the number of section classes.
|
|
|
|
### Fixed a heap buffer overflow when decoding a shared message list
|
|
|
|
When reading a shared object header message (SOHM) list from the metadata cache, `H5SM__cache_list_deserialize()` allocated the message array for `list_max` entries but drove the decode loop with the `num_messages` count read from the on-disk index header. A corrupted or malicious file whose `num_messages` exceeds `list_max` caused writes past the end of the array and reads past the end of the input buffer. The count is now validated against `list_max` before the loop runs.
|
|
|
|
Fixes CVE-2026-17572
|
|
|
|
### HTTP 403 errors in the ROS3 VFD for object keys with special characters
|
|
|
|
The ROS3 VFD did not URI-encode the S3 object key when building the HTTP request path, so keys containing characters that AWS Signature Version 4 requires to be percent-encoded — such as the '=' in Hive-style `key=value` partition prefixes, '+', or spaces — produced a signed request whose signature did not match S3's server-side recomputation. S3 rejects such requests with `SignatureDoesNotMatch`, which surfaces as an HTTP 403 error (indistinguishable from a permissions error on a HEAD request), even though tools like the AWS CLI could access the same object. The object key is now percent-encoded exactly once when the request path is built, matching the behavior of other S3 clients. Note that URLs must now be passed to the ROS3 VFD with their object keys unencoded; a key that was pre-encoded as a workaround for this issue will now be double-encoded and fail to resolve.
|
|
|
|
### Fixed file descriptor leaks in stdio VFD error paths
|
|
|
|
Fixed multiple resource leaks in the H5FDstdio driver where file descriptors were not properly closed on error paths. The error handling code was incorrectly attempting to close a local variable instead of the file pointer stored in the file structure, leading to file descriptor leaks. This issue affected 5 error paths in `H5FD_stdio_open()` and could cause file descriptor exhaustion in long-running applications.
|
|
|
|
### Added defensive NULL pointer checks in native VOL connector
|
|
|
|
Added assertion checks for NULL pointer parameters in `H5VL_native_get_file_struct()` to catch programming errors earlier and improve code robustness.
|
|
|
|
### Added checks for data filter behavior
|
|
|
|
The library now verifies that the returned data size from a data filter's filter callback function can fit inside the returned data buffer size. The library also checks that, when data is filtered then unfiltered (filtered in reverse), the returned data size is exactly the same as the original data size.
|
|
|
|
### Fixed bugs with chunk buffer handling
|
|
|
|
Fixed a bug in the deflate filter that caused it to report the wrong buffer size. Fixed a bug in the chunk copy code that could cause a background buffer overflow. Fixed a bug in the chunk copy code that could cause a double free if the filter realloced the data buffer.
|
|
|
|
### Fixed checking of data alignment requirements in direct I/O VFD
|
|
|
|
The direct I/O VFD attempts to determine data alignment requirements for a file on file open to try and avoid extra work when data alignment isn't required. Depending on the file access flags used when opening a file, the VFD could incorrectly determine these requirements for either writes or reads, eventually leading to a possible EINVAL return value on write or read. This has been fixed by separately determining the requirements for writes and reads and being more conservative about trying to avoid data alignment requirements.
|
|
|
|
### Fixed integer overflow in array datatype element count computation
|
|
|
|
Fixed a bug in H5O__dtype_decode_helper() where the loop computing the total number of elements in an array datatype had no per-step overflow check. On 64-bit systems, large dimension sizes could cause the element count to wrap around, bypassing the post-loop overflow check and producing silently incorrect results in downstream type conversion and size calculations.
|
|
|
|
### Validate VL datatype type during decode
|
|
|
|
When decoding a variable-length datatype message, `H5O__dtype_decode_helper()` set `vlen.type` directly from the flags byte read from the file, guarded only by an assertion in `H5T__vlen_set_loc()` that is removed in release builds. A corrupted or fuzzed file could supply a `vlen.type` that is neither `H5T_VLEN_SEQUENCE` nor `H5T_VLEN_STRING`, triggering the assertion in debug builds or a NULL pointer dereference in release builds. `H5O__dtype_decode_helper()` now rejects an invalid `vlen.type` as soon as it is read, and `H5T_set_loc()` now also checks for a NULL file pointer before calling `H5T__vlen_set_loc()` for disk-based VL types.
|
|
|
|
Fixes CVE-2026-17574
|
|
|
|
### Fixed an issue with chunked datasets using the wrong index type with parallel HDF5
|
|
|
|
Fixed a bug in parallel HDF5 that would cause chunked datasets with fixed dimensions and without filters applied to use the "none" index type instead of the "fixed array" index type.
|
|
|
|
### Fixed an issue with decoding metadata cache image superblock extension messages
|
|
|
|
Fixed a bug where loading of a metadata cache image superblock extension message would fail when the image had an undefined address and size of 0.
|
|
|
|
### Fixed an issue with an incorrect file format validation check when decoding metadata cache entries
|
|
|
|
Fixed a bug where a flag in H5Cimage.c wasn't getting set correctly for release builds of HDF5, leading to incorrect error checking when reconstructing metadata cache entries.
|
|
|
|
### Hardened decoding of serialized dataspace selections against malformed buffers
|
|
|
|
`H5S_select_deserialize()` and the per-selection-type deserialize callbacks (all, hyperslab, none, and point) previously computed the pointer to the last valid buffer byte as `buffer + size - 1` without first checking the buffer size. A buffer shorter than the 4-byte selection-type header, or a zero-length selection-info buffer, would underflow this computation and produce an out-of-bounds end pointer, defeating subsequent overflow checks. The deserialize routines now reject a buffer that is too small to hold the selection type, and they reject an empty selection-info buffer before deriving the end pointer. Hyperslab decoding additionally now rejects a serialized rank of 0 or greater than `H5S_MAX_RANK`. As a companion fix, `H5S__hyper_serialize()` now returns an error when asked to serialize a hyperslab selection on a rank-0 (scalar or null) dataspace, a state that can arise when a dataspace extent is collapsed to a scalar after a hyperslab selection has already been made.
|
|
|
|
## Java Library
|
|
|
|
### Fixed a datatype ID leak when reading or writing array/vlen datatypes
|
|
|
|
The JNI wrappers for `H5Dread`, `H5Dwrite`, `H5Aread`, and `H5Awrite` inspect the
|
|
memory datatype using an internal helper (`h5str_detect_vlen_str()`). For an array
|
|
or variable-length datatype whose base type is not a variable-length string, the
|
|
helper opened the base type with `H5Tget_super()` but failed to close it if
|
|
no variable-length string was found, leaking one datatype ID per
|
|
read/write call. The base type ID is now closed on all paths, so Java applications
|
|
that repeatedly access datasets or attributes with these datatypes no longer leak
|
|
HDF5 datatype IDs.
|
|
|
|
## Configuration
|
|
|
|
### Fixed INSTALL_CMAKE_DIR for GNU_DIRS and enabled it by default on Linux.
|
|
|
|
The previous `lib/cmake/hdf5-config.cmake` wasn't in CMake's `find_package`
|
|
search path since CMake expects that to be a common path with each
|
|
package having its own subdirectory. This changes it to
|
|
`lib/cmake/hdf5/hdf5-config.cmake` so the config is now found when the
|
|
install prefix is in `CMAKE_PREFIX_PATH`.
|
|
This also sets `HDF5_USE_GNU_DIRS=ON` by default for non-Windows builds.
|
|
|
|
## Tools
|
|
|
|
### Fixed h5repack silently dropping a declared cd_nelmts for user-defined filters
|
|
|
|
`h5repack -f UD=<filtn>,<flag>,<cd_nelmts>` with no values following `cd_nelmts` silently
|
|
treated the declared count as 0 instead of validating it, because the trailing token (with
|
|
no comma after it) was never committed to `cd_nelmts` inside the parser. `parse_filter()` now
|
|
commits the trailing token to whichever UD field is still pending, so a declared, unfulfilled
|
|
`cd_nelmts` is correctly rejected with "incorrect number of compression parameters" instead of
|
|
being silently coerced to 0.
|
|
|
|
## Performance
|
|
|
|
Fixed performance issues in several tools (h5dump, h5ls, h5diff, h5repack, h5stat and h5format_convert) for specific file structures where many objects are linked to with multiple hard links. While traversing a file's structure, these tools internally track already visited objects to avoid redundant processing on objects linked to multiple times. Checking if an object was already visited previously used a linear scan over an array of all the already visited objects that were multiply linked, resulting in behavior that was potentially quadratic with the number of objects visited and causing most of the application runtime to be spent checking this array. Additionally, h5repack had a separate array for hard link name aliases for objects that further contributed to performance issues in that tool. Replacing these arrays with hash tables greatly improved the performance of these tools on files with structures matching the structure mentioned previously.
|
|
|
|
## Fortran API
|
|
|
|
## High-Level Library
|
|
|
|
### Fixed critical buffer overflow vulnerability in H5TBget_field_info() (CWE-120)
|
|
|
|
`H5TBget_field_info()` copied field names into caller-provided buffers using unbounded `strcpy()`,
|
|
allowing a malicious HDF5 file with overly long field names to overflow those buffers. The copy
|
|
now uses bounds-checked `memcpy()`: names shorter than `HLTB_MAX_FIELD_LEN` (255) are copied
|
|
exactly (preserving backward compatibility); names at or above that limit are safely truncated to
|
|
254 characters plus a NUL terminator.
|
|
|
|
### Made HLTB_MAX_FIELD_LEN public
|
|
|
|
`HLTB_MAX_FIELD_LEN` (255) has been moved from the private header `H5TBprivate.h` to the public
|
|
header `H5TBpublic.h`. Applications can now use this constant to correctly size their
|
|
`field_names[]` buffers when calling `H5TBget_field_info()`.
|
|
|
|
### Fixed memory leaks and improved safety in H5LT functions
|
|
|
|
- Fixed memory leak in `H5LTtext_to_dtype()` by adding NULL check after `strdup()` call
|
|
- Added defensive NULL checks and pointer nullification after `free()` calls to prevent use-after-free bugs
|
|
- Improved documentation for `realloc_and_append()` internal function with detailed parameter contracts and preconditions
|
|
|
|
### Eliminated code duplication in H5LT datatype conversion
|
|
|
|
Refactored `H5LT_dtype_to_text()` by extracting common super-type handling logic into a new helper function `H5LT_append_dtype_super_text()`. This eliminates approximately 80 lines of duplicated code that was previously repeated across 4 datatype cases (ENUM, VLEN, ARRAY, COMPLEX), improving maintainability and reducing the risk of inconsistent behavior.
|
|
|
|
### Fixed H5TBread_fields_name/H5TBwrite_fields_name matching the wrong field when one field name is a prefix of another
|
|
|
|
H5TB_find_field() used strncmp() limited to strlen(field) when comparing the last entry of the supplied comma-separated field list against a table member name. This matched any user-supplied name whose leading characters equaled an existing field name (for example, requesting "PressureExtra" on a table containing "Pressure" would silently operate on the "Pressure" field). The comparison has been changed to strcmp() so full names must match exactly. In addition, H5TBwrite_fields_name() now returns an error when none of the requested field names are found (previously it silently performed a no-op write), matching the existing behavior of H5TBread_fields_name().
|
|
|
|
Fixes GitHub issue #5633
|
|
|
|
### Fixed prefix-based false matches when checking "CLASS" attribute strings in the High-Level API
|
|
|
|
`H5DSis_scale()`, `H5DS_is_reserved()`, `H5IMis_image()`, and `H5IMis_palette()` all compared a
|
|
dataset's "CLASS" attribute against an expected class name using
|
|
`strncmp(buf, CLASS, MIN(strlen(CLASS), strlen(buf)))`. Because the comparison was limited to the
|
|
shorter of the two strings, any non-empty value whose leading characters matched the expected class
|
|
name was accepted — for example, a CLASS of `"IMAGE_EXTRA"` was treated as an IMAGE dataset, and
|
|
`"DIMENSION_S"` (null-padded to 16 bytes) was treated as a DIMENSION_SCALE. (`H5DSis_scale()` already
|
|
required the attribute datatype to be exactly 16 bytes, which incidentally prevented false matches
|
|
against shorter class names such as `"IMAGE"` or `"PALETTE"`; the other three functions had no such
|
|
guard and were directly exposed.) These
|
|
comparisons now use `strcmp()` so only an exact class name is accepted.
|
|
|
|
Additional fixes applied to all four routines:
|
|
|
|
- **VLEN-string CLASS attributes are now handled correctly.** Previously, reading a VLEN-typed
|
|
attribute into a fixed `char *` buffer would overwrite it with a heap-allocated `char *` pointer
|
|
rather than the string content, which is undefined behaviour and could corrupt memory or produce
|
|
garbage comparison results.
|
|
All four routines now read VLEN CLASS attributes properly (via `H5Treclaim`) and compare the
|
|
string content: `H5DSis_scale()`, `H5IMis_image()`, and `H5IMis_palette()` return 1 when the
|
|
value matches exactly, and `H5DS_is_reserved()` correctly identifies reserved class names stored
|
|
as VLEN strings.
|
|
- **NUL-termination hardening.** The read buffer is now allocated one byte larger than the stored
|
|
attribute size, and a NUL terminator is explicitly written after the attribute data. This protects
|
|
`strcmp` from over-reading files where the CLASS attribute was written without strictly honouring
|
|
`H5T_STR_NULLTERM`.
|
|
- **Resource leak fix in `H5IMis_image()` and `H5IMis_palette()`.** The `out:` error-handling block
|
|
previously closed only the dataset ID, leaving the attribute ID (`aid`) and attribute datatype ID
|
|
(`atid`) open on every error path. Both IDs are now properly closed on error.
|
|
|
|
Related to GitHub issue #5633
|
|
|
|
## Fortran High-Level APIs
|
|
|
|
## Documentation
|
|
|
|
## F90 APIs
|
|
|
|
## C++ APIs
|
|
|
|
## Testing
|
|
|
|
# ✨ Support for new platforms and languages
|
|
|
|
# ☑️ Platforms Tested
|
|
|
|
A table of platforms tested can be seen on the [wiki](https://github.com/HDFGroup/hdf5/wiki/Platforms-Tested).
|
|
Current test results are available [here](https://my.cdash.org/index.php?project=HDF5).
|
|
|
|
# ⛔ Known Problems
|
|
|
|
- When performing implicit datatype conversion on specific non-IEEE floating-point format data, HDF5 may improperly convert some data values:
|
|
|
|
When performing I/O operations using a non-IEEE floating-point format datatype, HDF5 may improperly convert some data values due to incomplete handling of non-IEEE types. Such types include the following pre-defined datatypes:
|
|
|
|
H5T_FLOAT_F8E4M3
|
|
H5T_FLOAT_F8E5M2
|
|
H5T_FLOAT_F6E2M3
|
|
H5T_FLOAT_F6E3M2
|
|
H5T_FLOAT_F4E2M1
|
|
|
|
If possible, an application should perform I/O with these datatypes using an in-memory type that matches the specific floating-point format and perform explicit data conversion outside of HDF5, if necessary. Otherwise, read/written values should be verified to be correct.
|
|
|
|
- When the library detects and builds in support for the _Float16 datatype, an issue has been observed on at least one MacOS 14 system where the library fails to initialize due to not being able to detect the byte order of the _Float16 type [#4310](https://github.com/HDFGroup/hdf5/issues/4310):
|
|
|
|
#5: H5Tinit_float.c line 308 in H5T__fix_order(): failed to detect byte order
|
|
major: Datatype
|
|
minor: Unable to initialize object
|
|
|
|
If this issue is encountered, support for the _Float16 type can be disabled with a configuration option:
|
|
|
|
`CMake: HDF5_ENABLE_NONSTANDARD_FEATURE_FLOAT16=OFF`
|
|
|
|
- When HDF5 is compiled with NVHPC versions 23.5 - 23.9 (additional versions may also be applicable) and with -O2 (or higher) and -DNDEBUG, test failures occur in the following tests:
|
|
|
|
- H5PLUGIN-filter_plugin
|
|
- H5TEST-flush2
|
|
- H5TEST-testhdf5-base
|
|
- MPI_TEST_t_filters_parallel
|
|
|
|
Sporadic failures (even with lower -O levels):
|
|
|
|
- Java JUnit-TestH5Pfapl
|
|
- Java JUnit-TestH5D
|
|
|
|
Also, NVHPC will fail to compile the test/tselect.c test file with a compiler error of `use of undefined value` when the optimization level is -O2 or higher.
|
|
|
|
This is confirmed to be a [bug in the nvc compiler](https://forums.developer.nvidia.com/t/hdf5-no-longer-compiles-with-nv-23-9/269045) that has been fixed as of 23.11. If you are using an affected version of the NVidia compiler, the work-around is to set the optimization level to -O1.
|
|
|
|
- CMake files do not behave correctly with paths containing spaces
|
|
|
|
Do not use spaces in paths because the required escaping for handling spaces results in very complex and fragile build files.
|
|
|
|
- At present, metadata cache images may not be generated by parallel applications. Parallel applications can read files with metadata cache images, but since this is a collective operation, a deadlock is possible if one or more processes do not participate.
|
|
|
|
- The subsetting option in `ph5diff` currently will fail and should be avoided
|
|
|
|
The subsetting option works correctly in serial `h5diff`.
|
|
|
|
- Flang Fortran compilation will fail (last check version 17) due to not yet implemented: (1) derived type argument passed by value (H5VLff.F90), and (2) support for REAL with KIND = 2 in intrinsic SPACING used in testing.
|
|
|
|
- Fortran tests HDF5_1_8.F90 and HDF5_F03.F90 will fail with Cray compilers greater than version 16.0 due to a compiler bug. The latest version verified as failing was version 17.0.
|
|
|
|
- Several tests currently fail on certain platforms:
|
|
MPI_TEST-t_bigio fails with spectrum-mpi on ppc64le platforms.
|
|
|
|
MPI_TEST-t_subfiling_vfd and MPI_TEST_EXAMPLES-ph5_subfiling fail with
|
|
cray-mpich on theta and with XL compilers on ppc64le platforms.
|
|
|
|
- File space may not be released when overwriting or deleting certain nested variable length or reference types.
|
|
|
|
Known problems in previous releases can be found in the HISTORY*.txt files in the HDF5 source. Please report any new problems found to <a href="mailto:help@hdfgroup.org">help@hdfgroup.org</a>.
|
|
|
|
|
|
%%%%2.1.1%%%%
|
|
|
|
HDF5 version 2.1.1 released on 2026-03-23
|
|
|
|
---
|
|
|
|
> [!Note]
|
|
>
|
|
> - A typo in variable LT_HL_F_VERS_INTERFACE left H5_HL_F_SOVERS_INTERFACE empty, resulting in malformed macOS linker flags. The typo was fixed in this patch release.
|
|
> - A change in the directory structure of one of the compression filters resulted in undetected failure signing the hdf5-2.1.0-macos14_clang.dmg release binary. This was found and fixed in this patch release.
|
|
|
|
---
|
|
|
|
|
|
%%%%2.1.0%%%%
|
|
|
|
HDF5 version 2.1.0 released on 2026-03-02
|
|
|
|
# 🔺 HDF5 Changelog
|
|
All notable changes to this project will be documented in this file. This document describes the differences between this release and the previous
|
|
HDF5 release, platforms tested, and known problems in this release.
|
|
|
|
For releases prior to version 2.0.0, please see the release.txt file and for more details check the HISTORY*.txt files in the HDF5 source.
|
|
|
|
# 🔗 Quick Links
|
|
* [HDF5 documentation](https://support.hdfgroup.org/documentation/hdf5/latest/)
|
|
* [Official HDF5 releases](https://support.hdfgroup.org/downloads/index.html)
|
|
* [Changes from Release to Release and New Features in the HDF5-2.x.y](https://support.hdfgroup.org/releases/hdf5/documentation/release_specific_info.md)
|
|
* [Getting help, questions, or comments](https://github.com/HDFGroup/hdf5#help-and-support)
|
|
|
|
## 📖 Contents
|
|
* [Executive Summary](CHANGELOG.md#execsummary)
|
|
* [Breaking Changes](CHANGELOG.md#%EF%B8%8F-breaking-changes)
|
|
* [New Features & Improvements](CHANGELOG.md#-new-features--improvements)
|
|
* [Bug Fixes](CHANGELOG.md#-bug-fixes)
|
|
* [Support for new platforms and languages](CHANGELOG.md#-support-for-new-platforms-and-languages)
|
|
* [Platforms Tested](CHANGELOG.md#%EF%B8%8F-platforms-tested)
|
|
* [Known Problems](CHANGELOG.md#-known-problems)
|
|
|
|
# 🔆 Executive Summary: HDF5 Version 2.1.0
|
|
|
|
> [!IMPORTANT]
|
|
>
|
|
> - The format of the GitHub tag for HDF5 releases has been changed to Major.Minor.Patch, consistent with the versioning policy change to follow the Semantic Versioning Specification described in this [Wiki page](https://github.com/HDFGroup/hdf5/wiki/HDF5-Version-Numbers-and-Branch-Strategy). The previous tag format hdf5_Major_Minor_Patch that was created in addition for HDF5 2.0.0 and this release will not be continued.
|
|
|
|
## Performance Enhancements:
|
|
|
|
|
|
## Significant Advancements:
|
|
|
|
|
|
## Enhanced Features:
|
|
|
|
|
|
## Java Enhancements:
|
|
|
|
|
|
## Acknowledgements:
|
|
|
|
We would like to thank the many HDF5 community members who contributed to this release of HDF5.
|
|
|
|
# ⚠️ Breaking Changes
|
|
|
|
|
|
# 🚀 New Features & Improvements
|
|
|
|
## Configuration
|
|
|
|
### Added a CMake module to locate libaec for SZIP support
|
|
|
|
A new `Findlibaec.cmake` CMake module has been added. This module is intended to locate libaec on the system for SZIP support in HDF5 when libaec was built with Autotools instead of CMake. When SZIP support is enabled in HDF5 with the `HDF5_ENABLE_SZIP_SUPPORT` option, this module will first check for an existing CMake-built libaec and use that if it's available. Otherwise, the module will heuristically search for libaec on the system. If necessary, the module can be hinted toward a particular libaec installation by setting the CMake variable `libaec_ROOT` to point to a directory. If it is known that a CMake-built libaec installation exists on the system in a non-standard location, the CMake variable `libaec_DIR` can instead be set to a directory containing a `libaec-config.cmake` file to cause the module to prefer that libaec installation.
|
|
|
|
### Refactored CMake library export mechanism
|
|
|
|
The CMake build system now uses separate export targets for different library types: static libraries use `${HDF5_EXPORTED_TARGETS}_static`, Java libraries use `${HDF5_EXPORTED_TARGETS}_java`, and shared libraries continue using the main `${HDF5_EXPORTED_TARGETS}`. The generated `hdf5-config.cmake` file conditionally includes the appropriate target files (`hdf5-targets_static.cmake` and `hdf5-targets_java.cmake`), allowing downstream projects to selectively link against specific library variants. This refactoring improves build system maintainability and applies consistently across all HDF5 components (core, C++, Fortran, high-level APIs, tools, and Java JNI).
|
|
|
|
## Library
|
|
|
|
### Added predefined datatypes for FP6 data
|
|
|
|
Predefined datatypes have been added for FP6 data in [E2M3 and E3M2 formats](https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf).
|
|
|
|
The following new macros have been added:
|
|
|
|
- `H5T_FLOAT_F6E2M3`
|
|
- `H5T_FLOAT_F6E3M2`
|
|
|
|
These macros map to IDs of HDF5 datatypes representing a 6-bit floating-point datatype with 1 sign bit and either 2 exponent bits and 3 mantissa bits (E2M3 format) or 3 exponent bits and 2 mantissa bits (E3M2 format).
|
|
|
|
Note that support for a native FP6 datatype has not been added yet. This means that any datatype conversions to/from the new FP6 datatypes will be emulated in software rather than potentially using specialized hardware instructions. Until support for a native FP6 type is added, an application can avoid datatype conversion performance issues if it is sure that the datatype used for in-memory data buffers matches one of the above floating-point formats. In this case, the application can specify one of the above macros for both the file datatype when creating a dataset or attribute and the memory datatype when performing I/O on the dataset or attribute.
|
|
|
|
Also note that HDF5 currently has incomplete support for datatype conversions involving non-IEEE floating-point format datatypes. Refer to the 'Known Problems' section for information about datatype conversions with these new datatypes.
|
|
|
|
### Added predefined datatype for FP4 data
|
|
|
|
A predefined datatype has been added for FP4 data in [E2M1 format](https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf).
|
|
|
|
The following new macro has been added:
|
|
|
|
- `H5T_FLOAT_F4E2M1`
|
|
|
|
This macro maps to the ID of an HDF5 datatype representing a 4-bit floating-point datatype with 1 sign bit, 2 exponent bits and 1 mantissa bit.
|
|
|
|
Note that support for a native FP4 datatype has not been added yet. This means that any datatype conversions to/from the new FP4 datatype will be emulated in software rather than potentially using specialized hardware instructions. Until support for a native FP4 type is added, an application can avoid datatype conversion performance issues if it is sure that the datatype used for in-memory data buffers matches the above floating-point format. In this case, the application can specify the above macro for both the file datatype when creating a dataset or attribute and the memory datatype when performing I/O on the dataset or attribute.
|
|
|
|
Also note that HDF5 currently has incomplete support for datatype conversions involving non-IEEE floating-point format datatypes. Refer to the 'Known Problems' section for information about datatype conversions with these new datatypes.
|
|
|
|
## Parallel Library
|
|
|
|
## Fortran Library
|
|
|
|
## C++ Library
|
|
|
|
## Java Library
|
|
|
|
## Tools
|
|
|
|
## High-Level APIs
|
|
|
|
## C Packet Table API
|
|
|
|
## Internal header file
|
|
|
|
## Documentation
|
|
|
|
|
|
# 🪲 Bug Fixes
|
|
|
|
## Library
|
|
|
|
### Fixed a problem with using filters with variable length datatypes
|
|
|
|
When using data filters with a dataset with a variable length datatype, the library would not invoke the filters' can_apply and set_local callbacks. This has been addressed and the library will now make those callbacks no matter what the datatype is.
|
|
|
|
### Fixed a potential out of bound read
|
|
|
|
When a file is corrupted such that an array datatype's size, the number of elements, and the element size are not in agreement, it can trigger an out of bounds read. A check has been added to detect such situation.
|
|
|
|
### Fixed security issue [CVE-2025-44904](https://www.cve.org/CVERecord?id=CVE-2025-44904)
|
|
|
|
For unfiltered dataset chunks, the size on disk should be constant for all chunks in a dataset. In some cases the size of each chunk is stored even in this case where it can be inferred from the chunk dimensions and datatype. The code previously assumed this stored size was equal to the inferred size, leading to a mismatch in the expected and actual buffer size. Modified the library to throw an error if the size does not match the expected size.
|
|
|
|
### Fixed security issue [CVE-2025-2309](https://www.cve.org/CVERecord?id=CVE-2025-2309)
|
|
|
|
Fixed a critical vulnerability in `H5T__bit_copy` of the component Type Conversion Logic which could lead to heap-based buffer overflow.
|
|
|
|
### Fixed security issue [CVE-2025-2308](https://www.cve.org/CVERecord?id=CVE-2025-2308)
|
|
|
|
Fixed a critical vulnerability in the function `H5Z__scaleoffset_decompress_one_byte` of the component Scale-Offset Filter which could lead to heap-based buffer overflow.
|
|
|
|
### Fixed a double-free bug in `H5D__chunk_copy`
|
|
|
|
Fixed a double-free bug in the internal `H5D__chunk_copy()` function which occurred when a buffer was re-allocated without updating the original pointer freed later on.
|
|
|
|
Fixes GitHub issues [#6123](https://github.com/HDFGroup/hdf5/issues/6123)
|
|
[#6124](https://github.com/HDFGroup/hdf5/issues/6124)
|
|
[#6125](https://github.com/HDFGroup/hdf5/issues/6125)
|
|
[#6126](https://github.com/HDFGroup/hdf5/issues/6126)
|
|
[#6133](https://github.com/HDFGroup/hdf5/issues/6133)
|
|
### Fixes potential security issues
|
|
|
|
The `get_name` API functions allow passing NULL when querying the object name length. However, passing a non-NULL buffer with size == 0 will result in security vulnerability of invalid write. That was because the library wrote a null terminator to the buffer regardless of what the size of the buffer was as long as the buffer was non-NULL.
|
|
These functions are now fixed to treat (buffer != NULL, size == 0) as a length-only query to eliminate Valgrind error of invalid write.
|
|
|
|
### Fixed a performance issue with chunked dataset I/O
|
|
|
|
When dataset chunks are unable to be placed in the dataset chunk cache (for example, if a chunk is too large), the library falls back to an alternative approach for I/O on dataset chunks. An issue with the logic in this approach prevented chunked dataset I/O from making use of the library's data sieve buffer I/O optimization functionality. For chunk shapes that are non-contiguous with the memory layout of a buffer, this could result in severely degraded I/O performance, with the worst-case behavior causing I/O to be performed on a single data element at a time.
|
|
|
|
The data sieve buffer functionality has been extended to cover the case of uncached chunks and will be used as long as the underlying Virtual File Driver supports data sieving.
|
|
|
|
## Java Library
|
|
|
|
## Configuration
|
|
|
|
### Removed force-setting of `ZLIB_USE_EXTERNAL` and `SZIP_USE_EXTERNAL` CMake variables to `ON`
|
|
|
|
When the CMake variable `HDF5_ALLOW_EXTERNAL_SUPPORT` is set to `GIT` or `TGZ`, the library's build process previously force-set the `ZLIB_USE_EXTERNAL` and `SZIP_USE_EXTERNAL` variables to `ON`. This prevented the ability to independently choose whether zlib and szip are built from system libraries or from external sources. These variables are no longer forced to `ON` in this case and can be set individually.
|
|
|
|
|
|
## Tools
|
|
|
|
## Performance
|
|
|
|
## Fortran API
|
|
|
|
### Added Fortran wrappers for SWMR functionality
|
|
|
|
Added four new Fortran wrappers that provide direct access to SWMR (Single Writer Multiple Reader) C APIs:
|
|
- `h5fstart_swmr_write_f` - Enables SWMR writing mode for a file
|
|
- `h5dflush_f` - Flushes dataset buffers to disk
|
|
- `h5pset_append_flush_f` - Sets append flush property values including optional callback function
|
|
- `h5pget_append_flush_f` - Retrieves append flush property values including callback function
|
|
|
|
## High-Level Library
|
|
|
|
## Fortran High-Level APIs
|
|
|
|
## Documentation
|
|
|
|
## F90 APIs
|
|
|
|
## C++ APIs
|
|
|
|
## Testing
|
|
|
|
# ✨ Support for new platforms and languages
|
|
|
|
# ☑️ Platforms Tested
|
|
|
|
A table of platforms tested can be seen on the [wiki](https://github.com/HDFGroup/hdf5/wiki/Platforms-Tested).
|
|
Current test results are available [here](https://my.cdash.org/index.php?project=HDF5).
|
|
|
|
# ⛔ Known Problems
|
|
|
|
- When performing implicit datatype conversion on specific non-IEEE floating-point format data, HDF5 may improperly convert some data values:
|
|
|
|
When performing I/O operations using a non-IEEE floating-point format datatype, HDF5 may improperly convert some data values due to incomplete handling of non-IEEE types. Such types include the following pre-defined datatypes:
|
|
|
|
`H5T_FLOAT_F8E4M3`<br>
|
|
`H5T_FLOAT_F8E5M2`<br>
|
|
`H5T_FLOAT_F6E2M3`<br>
|
|
`H5T_FLOAT_F6E3M2`<br>
|
|
`H5T_FLOAT_F4E2M1`
|
|
|
|
If possible, an application should perform I/O with these datatypes using an in-memory type that matches the specific floating-point format and perform explicit data conversion outside of HDF5, if necessary. Otherwise, read/written values should be verified to be correct.
|
|
|
|
- When the library detects and builds in support for the _Float16 datatype, an issue has been observed on at least one MacOS 14 system where the library fails to initialize due to not being able to detect the byte order of the _Float16 type [#4310](https://github.com/HDFGroup/hdf5/issues/4310):
|
|
|
|
#5: H5Tinit_float.c line 308 in `H5T__fix_order()`: failed to detect byte order
|
|
major: Datatype
|
|
minor: Unable to initialize object
|
|
|
|
If this issue is encountered, support for the _Float16 type can be disabled with a configuration option:
|
|
|
|
`CMake: HDF5_ENABLE_NONSTANDARD_FEATURE_FLOAT16=OFF`
|
|
|
|
- When HDF5 is compiled with NVHPC versions 23.5 - 23.9 (additional versions may also be applicable) and with -O2 (or higher) and -DNDEBUG, test failures occur in the following tests:
|
|
|
|
- H5PLUGIN-filter_plugin
|
|
- H5TEST-flush2
|
|
- H5TEST-testhdf5-base
|
|
- MPI_TEST_t_filters_parallel
|
|
|
|
Sporadic failures (even with lower -O levels):
|
|
|
|
- Java JUnit-TestH5Pfapl
|
|
- Java JUnit-TestH5D
|
|
|
|
Also, NVHPC will fail to compile the test/tselect.c test file with a compiler error of `use of undefined value` when the optimization level is -O2 or higher.
|
|
|
|
This is confirmed to be a [bug in the nvc compiler](https://forums.developer.nvidia.com/t/hdf5-no-longer-compiles-with-nv-23-9/269045) that has been fixed as of 23.11. If you are using an affected version of the NVidia compiler, the work-around is to set the optimization level to -O1.
|
|
|
|
- CMake files do not behave correctly with paths containing spaces
|
|
|
|
Do not use spaces in paths because the required escaping for handling spaces results in very complex and fragile build files.
|
|
|
|
- At present, metadata cache images may not be generated by parallel applications. Parallel applications can read files with metadata cache images, but since this is a collective operation, a deadlock is possible if one or more processes do not participate.
|
|
|
|
- The subsetting option in `ph5diff` currently will fail and should be avoided
|
|
|
|
The subsetting option works correctly in serial `h5diff`.
|
|
|
|
- Flang Fortran compilation will fail (last check version 17) due to not yet implemented: (1) derived type argument passed by value (H5VLff.F90), and (2) support for REAL with KIND = 2 in intrinsic SPACING used in testing.
|
|
|
|
- Fortran tests HDF5_1_8.F90 and HDF5_F03.F90 will fail with Cray compilers greater than version 16.0 due to a compiler bug. The latest version verified as failing was version 17.0.
|
|
|
|
- Several tests currently fail on certain platforms:
|
|
MPI_TEST-t_bigio fails with spectrum-mpi on ppc64le platforms.
|
|
|
|
MPI_TEST-t_subfiling_vfd and MPI_TEST_EXAMPLES-ph5_subfiling fail with
|
|
cray-mpich on theta and with XL compilers on ppc64le platforms.
|
|
|
|
- File space may not be released when overwriting or deleting certain nested variable length or reference types.
|
|
|
|
Known problems in previous releases can be found in the HISTORY*.txt files in the HDF5 source. Please report any new problems found to <a href="mailto:help@hdfgroup.org">help@hdfgroup.org</a>.
|
|
|
|
|
|
%%%%2.0.0%%%%
|
|
|
|
HDF5 version 2.0.0 released on 2025-11-11
|
|
|
|
# 🔺 HDF5 Changelog
|
|
All notable changes to this project will be documented in this file. This document describes the differences between this release and the previous
|
|
HDF5 release, platforms tested, and known problems in this release.
|
|
|
|
For releases prior to version 2.0.0, please see the release.txt file and for more details check the HISTORY*.txt files in the HDF5 source.
|
|
|
|
# 🔗 Quick Links
|
|
* [HDF5 documentation](https://support.hdfgroup.org/documentation/hdf5/latest/)
|
|
* [Official HDF5 releases](https://support.hdfgroup.org/downloads/index.html)
|
|
* [Changes from Release to Release and New Features in the HDF5-2.x.y](https://support.hdfgroup.org/releases/hdf5/documentation/release_specific_info.md)
|
|
* [Getting help, questions, or comments](https://github.com/HDFGroup/hdf5#help-and-support)
|
|
|
|
## 📖 Contents
|
|
* [Executive Summary](CHANGELOG.md#execsummary)
|
|
* [Breaking Changes](CHANGELOG.md#%EF%B8%8F-breaking-changes)
|
|
* [New Features & Improvements](CHANGELOG.md#-new-features--improvements)
|
|
* [Bug Fixes](CHANGELOG.md#-bug-fixes)
|
|
* [Support for new platforms and languages](CHANGELOG.md#-support-for-new-platforms-and-languages)
|
|
* [Platforms Tested](CHANGELOG.md#%EF%B8%8F-platforms-tested)
|
|
* [Known Problems](CHANGELOG.md#-known-problems)
|
|
|
|
# 🔆 Executive Summary: HDF5 Version 2.0.0
|
|
|
|
## Performance Enhancements:
|
|
|
|
- Up to [2500% faster](CHANGELOG.md#rtree) Virtual Dataset read/write operations
|
|
- [30% faster opening](CHANGELOG.md#layoutcopydelay) and [25% faster closing](CHANGELOG.md#fileformat) of virtual datasets.
|
|
- [Reduced memory overhead](CHANGELOG.md#fileformat) via shared name strings and optimized spatial search algorithms for virtual datasets.
|
|
|
|
## Significant Advancements:
|
|
|
|
- Full [UTF-8](CHANGELOG.md#utf-8) filename support on Windows, resolving encoding issues from previous versions.
|
|
- Introduction of [bfloat16 predefined datatypes](CHANGELOG.md#added-predefined-datatypes-for-bfloat16-data) for efficient machine learning conversions.
|
|
- First-class support for [complex numbers](CHANGELOG.md#complex), eliminating manual workarounds in scientific applications.
|
|
- A [new, larger chunk size limit](CHANGELOG.md#added-support-for-large-chunks), in multi-petabytes, replaces the previous 4 GiB limit.
|
|
|
|
## Updated Foundation:
|
|
|
|
- New [file format](CHANGELOG.md#fileformat) version (4.0) and compliance with the C11 standard.
|
|
- Adopted [semantic versioning](https://github.com/HDFGroup/hdf5/wiki/HDF5-Version-Numbers-and-Branch-Strategy) to clearly convey changes between versions.
|
|
|
|
> [!IMPORTANT]
|
|
>
|
|
> - Transitioned to [CMake-only](CHANGELOG.md#cmake) builds, and Autotools is no longer in use.
|
|
> - Renamed library state variables, notably `HDF5_ENABLE_PARALLEL` is now `HDF5_PROVIDES_PARALLEL`, see PR [#5716](https://github.com/HDFGroup/hdf5/pull/5716) for more details.
|
|
> - The default setting for `H5Fset_libver_bounds` has been updated to set the lower bound to the HDF5 library version 1.8. This change ensures that users can take advantage of the library's optimal performance and the latest features by default. If users need their files to be compatible with older versions of the HDF5 library, they will need to adjust this lower bound manually.
|
|
|
|
## Enhanced Features:
|
|
|
|
- Improved [ROS3 VFD](CHANGELOG.md#ros3) capabilities using the aws-c-s3 library.
|
|
|
|
## Java Enhancements:
|
|
|
|
- Java FFM bindings generated for the C library if Java 25+ available and JNI option is FALSE.
|
|
- Enhanced Maven artifact deployment with comprehensive multi-platform support (Linux, Windows, macOS x86_64, macOS aarch64).
|
|
- Complete Java examples Maven integration (`org.hdfgroup:hdf5-java-examples`) with cross-platform CI/CD testing.
|
|
|
|
## Acknowledgements:
|
|
|
|
We would like to thank the many HDF5 community members who contributed to HDF5 2.0.
|
|
|
|
# ⚠️ Breaking Changes
|
|
|
|
### Updated default file format to 1.8
|
|
|
|
By default, HDF5 will now use the 1.8 file format (`H5F_LIBVER_V18`). This provides improved performance and space efficiency, particularly with groups and links. However, HDF5 library versions 1.6 and earlier will not be able to read files created with the default settings. The previous behavior can be restored using `H5Pset_libver_bounds(fapl_id, H5F_LIBVER_EARLIEST, H5F_LIBVER_LATEST)`.
|
|
|
|
### Renamed the option: `HDF5_ENABLE_Z_LIB_SUPPORT`
|
|
|
|
The option has been renamed to `HDF5_ENABLE_ZLIB_SUPPORT` to be consistent with the naming of other options. Also, the option defaults to OFF. This requires the user to explicitly enable zlib support when configuring the library.
|
|
|
|
### Autotools support was removed from HDF5<a name="cmake">
|
|
|
|
CMake is now the build system available in HDF5 code. Version 3.26 or later is required. See the [AutotoolsToCMakeOptions.md](AutotoolsToCMakeOptions.md) file for highlights of the CMake HDF5 install layout and CMake options to use in place of former Autotools options.
|
|
|
|
### Fixed problems with family driver and user block
|
|
|
|
When using a user block with the family driver, the driver would inappropriately subtract the user block size for each member file when calculating member EOAs. This could cause a failure when an address overflowed the calculated eoa. The driver would also add the user block size when returning the EOF. Modified the family driver to not consider the user block, as it is handled by the H5FD layer. The user block now spans the first X bytes of the family array, for example a 4 KiB user block with 3 KiB member size will take up the entire first member and the first 1 KiB of the second. This may cause compatibility issues with preexisting family files with user blocks, though the way it worked before was inconsistent if it worked at all.
|
|
|
|
# 🚀 New Features & Improvements
|
|
|
|
## Configuration
|
|
|
|
### Improved the cross-compile support in the build system
|
|
|
|
The CMake build system has been improved to better support cross-compiling. This includes the following changes:
|
|
- The CMake option, `CMAKE_CROSS_COMPILING`, indicates that the library is being cross-compiled. This option, usually in a toolchain file, can be set to ON when cross-compiling.
|
|
- Removed the `CMAKE_CROSSCOMPILING_EMULATOR` when cross-compiling, CMake will automatically insert it in the command.
|
|
- Added an option `HDF5_USE_PREGEN` to supply pre-generated files located in `HDF5_USE_PREGEN_DIR` directory, to bypass running feature detection programs when cross-compiling.
|
|
- Added a variable `CROSSCOMPILING_PATH` to specify a path to search for programs when cross-compiling. This is useful when the build system needs to run programs that were built for the host system.
|
|
|
|
### Refactored `HDF5_BUILD/ENABLE_{feature}` variable in hdf5-config.cmake file
|
|
|
|
The variables used in hdf5-config.cmake to indicate what options were used to build the installed library have been renamed. All `HDF5_BUILD/ENABLE_{feature}` variables are now `HDF5_PROVIDES_{feature}`. This more clearly indicates that these variables reflect whether the feature is supported by the installed library, instead of whether the feature is an option that can be changed when building an application with the library.
|
|
|
|
Created macro `EXTERNAL_HDF5_STATUS` to convert between the old and new names. The macro is in the config/examples/HDF5SubdirMacros.cmake file and can be copied into a project's CMakeLists.txt file to provide backward compatibility.
|
|
|
|
### CMake minimum version is now 3.26
|
|
|
|
The minimum version of CMake is now 3.26; this will remove workarounds to handle versions between 3.18 (the previous minimum) and 3.26.
|
|
|
|
### Removed `HDF5_ENABLE_THREADS` option
|
|
|
|
The `HDF5_ENABLE_THREADS` option has been removed, as it no longer functions as a proper build option. The library will always check for thread support and set the internal status variable, `HDF5_THREADS_ENABLED`. The `HDF5_ENABLE_THREADSAFE` option is still available to build with thread-safe API calls.
|
|
|
|
### Enhanced Maven repository deployment support
|
|
|
|
Added comprehensive Maven integration with optimized workflows for Java artifact deployment:
|
|
- **New CMake options**: `HDF5_ENABLE_MAVEN_DEPLOY` and `HDF5_MAVEN_SNAPSHOT` for Maven repository deployment
|
|
- **Minimal build presets**: Added `ci-MinShar-*-Maven*` presets for efficient Java-only artifact generation
|
|
- **Multi-platform support**: Automated generation of platform-specific JARs with classifiers (linux-x86_64, windows-x86_64, macos-x86_64, macos-aarch64)
|
|
- **CI/CD integration**: Enhanced GitHub Actions workflows (`maven-staging.yml`, `maven-deploy.yml`) with cross-platform build matrix
|
|
- **Artifact validation**: Comprehensive validation framework for Maven artifacts before deployment
|
|
- **Deployment targets**: Support for GitHub Packages and Maven Central staging repositories
|
|
- **Java Examples Maven Integration**: Added complete Maven artifact for Java examples (`org.hdfgroup:hdf5-java-examples`) with cross-platform compatibility
|
|
- **Multi-platform testing**: Comprehensive CI/CD testing of Java examples across all supported platforms (Linux, Windows, macOS x86_64, macOS aarch64)
|
|
- **Native library error handling**: Enhanced validation logic for Maven-only environments to properly handle expected native library loading errors
|
|
- **Dynamic repository support**: Enhanced workflows to use `github.repository` variable for seamless testing on forks before canonical deployment
|
|
- **Fork-based testing**: Complete testing framework allowing validation on repository forks (e.g., fork-name/hdf5) before merging to HDFGroup/hdf5
|
|
- **Multi-artifact deployment**: Enhanced deployment workflow to handle both `hdf5-java` (platform-specific) and `hdf5-java-examples` (platform-independent) artifacts
|
|
- **Production deployment validation**: Successfully resolved HTTP 409 version conflicts through snapshot versioning strategy
|
|
- **Deployment status**: ✅ Fully validated and production-ready with comprehensive error resolution and testing documentation
|
|
|
|
### Reorganized the files in the config/cmake folder into the config folder structure
|
|
|
|
The config folder CMake files have been reorganized to make it easier to maintain and add new features. This includes the following changes:
|
|
- The files in the config folder are the macros and templates for the build process.
|
|
- The files in the config/cmake folder are primarily for optional features.
|
|
- The files in the config/install folder are for installation support.
|
|
- The files in the config/flags folder are for compiler support.
|
|
- The remaining folders remain unchanged.
|
|
|
|
### Added CMake configuration options to override compilers in h5cc:
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `HDF5_H5CC_C_COMPILER` | for the C compiler |
|
|
| `HDF5_H5CC_CXX_COMPILER` | for the C++ compiler |
|
|
| `HDF5_H5CC_Fortran_COMPILER` | for the Fortran compiler |
|
|
|
|
These default to the currently used compiler, preserving the current behavior. However, they can be overridden by users who need to use a different compiler at runtime, for example, when they build via cache.
|
|
|
|
### Aligned the CMake compiler wrappers with the old Autotools versions
|
|
|
|
The versions of `h5cc`, `h5fc`, `h5c++`, etc. generated by CMake were missing several options and features from the Autotools counterparts. Some of these options and features have now been implemented in the CMake versions, while some of them have not:
|
|
- The missing `--help`/`-h`, `-c`, `-echo`, and `-shlib`/`-noshlib` options have been implemented.
|
|
- The `-prefix` option was not implemented, as it didn't appear to function in the Autotools wrappers and is generally covered by pkg-config in the CMake wrappers.
|
|
- A new `-nohl` option has been added to avoid building and linking against the high-level HDF5 libraries if desired.
|
|
- Similar to the Autotools wrappers, the CMake wrappers now add the HDF5 installation library directory to the rpath of the resulting executable/library by default when linking against shared HDF5 libraries. This behavior can be avoided by specifying the new `-norpath` option.
|
|
- Parsing of the `HDF5_USE_SHLIB` environment variable has been added to determine whether to link against shared or static HDF5 libraries. Precedence is still given to the `-shlib`/`-noshlib` options.
|
|
- Parsing of the `HDF5_PKG_CONFIG_ARGS` environment variable has been added to separate pkg-config-specific options from compiler-specific options and prevent conflicts between them.
|
|
|
|
Several issues were also fixed in the pkg-config files that are generated by CMake.
|
|
|
|
### Added `CMAKE_INSTALL_PREFIX` to the default plugin path
|
|
|
|
To help users find their plugins, the default plugin path has been changed to include the `CMAKE_INSTALL_PREFIX`. Adding the install prefix allows users to skip setting the `HDF5_PLUGIN_PATH` environment variable when using plugins with the default lib/plugin location.
|
|
|
|
### Converted documentation in the source folder, doc, to doxygen files.
|
|
|
|
### Added configuration option for API concurrency support:
|
|
|
|
CMake: `HDF5_ENABLE_CONCURRENCY` (ON/OFF) (Default: OFF)
|
|
|
|
This option enables support for concurrent multithreaded operation of supported API routines. This option also provides thread-safe execution of all other, non-concurrent operations. The 'concurrency' option thus is a superset of the existing 'threadsafe' option. Both options are currently available, although mutually exclusive. As the 'concurrency' code becomes more stable over time, the 'threadsafe' option may be deprecated in favor of the new 'concurrency' option.
|
|
|
|
The following API routines support concurrent multithreaded operation:
|
|
<none yet>
|
|
|
|
### Added support for MinGW + MSYS2 when building with CMake
|
|
|
|
We added support for this to the appropriate configure-time checks in CMake. CMake + MinGW + MSYS2 is now tested with the following environments:
|
|
- mingw32
|
|
- mingw64
|
|
- ucrt64
|
|
- clang64
|
|
|
|
### Added CMake build mode flags to the libhdf5.settings file
|
|
|
|
Flags from the CMake build mode (e.g., optimization) are not a part of `CMAKE_<language>_FLAGS` and were not exported to the libhdf5.settings file. This has been fixed, and the C, Fortran, and C++ build mode flags are now exported to the file.
|
|
|
|
This also affects the text output of `H5check_version()` and the libhdf5.settings string stored in the library (for those who use strings(1), etc. to get build info from the binary).
|
|
|
|
### CMake: Split compiler-specific flags into separate files
|
|
|
|
The compiler-specific flags have been split into separate files to make it easier to maintain and add new compiler flags. The flags for NVHPC, Intel, GNU and Clang compilers are now in separate files included from the current compiler flags files; `HDFCompiler<language>Flags.cmake`.
|
|
|
|
### Added support for native zlib-ng compression
|
|
|
|
Changed the zlib-ng CMake logic to prefer the native zlib-ng library. Added `#ifdef` around the compression function calls. Added including the correct header file with the same `#ifdef`.
|
|
|
|
### Renamed HDF5Examples build options from `H5EXAMPLE_<option>` to `H5EXAMPLE_<option>`
|
|
|
|
Changed the prefix to better distinguish the examples build options from the library options when building the examples along with the library.
|
|
|
|
### Renamed remaining HDF5 library CMake options except for CMake BUILD* variables
|
|
|
|
| Old | New |
|
|
| --- | --- |
|
|
| `DEFAULT_API_VERSION` | `HDF5_DEFAULT_API_VERSION` |
|
|
| `DISABLE_PDB_FILES` | `HDF5_DISABLE_PDB_FILES` |
|
|
| `ONLY_SHARED_LIBS` | `HDF5_ONLY_SHARED_LIBS` |
|
|
| `ALLOW_UNSUPPORTED` | `HDF5_ALLOW_UNSUPPORTED` |
|
|
| `TEST_SHELL_SCRIPTS` | `HDF5_TEST_SHELL_SCRIPTS` |
|
|
|
|
All other HDF5 library CMake options are prefixed with `HDF5_`
|
|
|
|
### bin/cmakehdf5 has been removed
|
|
|
|
This was an unsupported build script that made building HDF5 via CMake work like building HDF5 via the Autotools. It has been unmaintained for a long time, has been marked deprecated, and is being removed.
|
|
|
|
### Generated files in src are now checked into version control
|
|
|
|
These files are infrequently updated, and generating them adds a dependency on Perl. The listed files are now checked in and do not need to be recreated when checking out development branches.
|
|
- H5Edefin.h
|
|
- H5Einit.h
|
|
- H5Emajdef.h
|
|
- H5Emindef.h
|
|
- H5Epubgen.h
|
|
- H5Eterm.h
|
|
- H5overflow.h
|
|
- H5version.h
|
|
|
|
### Dropped some old Solaris Studio work-arounds
|
|
|
|
Solaris Studio no longer seems to be maintained and the last version (12.4, circa 2015) doesn't seem to fully support C11. We've removed some hacks that work around things like `__attribute__()` support.
|
|
|
|
### Dropped support for the traditional MSVC preprocessor
|
|
|
|
Visual Studio has [recently started using a standards-compliant preprocessor](https://learn.microsoft.com/en-us/cpp/preprocessor/preprocessor-experimental-overview?view=msvc-170) (In VS2019+) and this is the default in C11. Because of this, we've dropped support for the traditional MSVC preprocessor.
|
|
|
|
The standard for building the library is now C11. We have updated the build files to set the C standard to C11, though some platforms use gnu11 to get some GNU things to work.
|
|
|
|
## Library
|
|
|
|
### Added support for large chunks
|
|
|
|
The library now supports chunks larger than 4 GiB using 64 bit addressing. Creating chunks with size >= 4 GiB will upgrade the file format and prevent the dataset from being opened with earlier versions of the library. 32 bit systems will not be able to use these chunks in all circumstances, such as with data filters or a fill value.
|
|
|
|
### Changed default chunk cache hash table size to 8191
|
|
|
|
In order to reduce hash collisions and take advantage of modern memory capacity, the default hash table size for the chunk cache has been increased from 521 to 8191. This means the hash table will consume approximately 64 KiB per open dataset. This value can be changed with `H5Pset_cache()` or `H5Pset_chunk_cache()`. This value was chosen because it is a prime number close to 8K.
|
|
|
|
### Updated default file format to 1.8
|
|
|
|
By default, HDF5 will now use the 1.8 file format (`H5F_LIBVER_V18`). This provides improved performance and space efficiency, particularly with groups and links. This behavior can be overridden with `H5Pset_libver_bounds()`.
|
|
|
|
### Added predefined datatypes for bfloat16 data
|
|
|
|
Predefined datatypes have been added for little- and big-endian [bfloat16](https://en.wikipedia.org/wiki/Bfloat16_floating-point_format) data.
|
|
|
|
The following new macros have been added:
|
|
|
|
- `H5T_FLOAT_BFLOAT16LE` / `H5T_FLOAT_BFLOAT16BE`
|
|
|
|
These macros map to IDs of HDF5 datatypes representing a little- or big-endian 16-bit floating-point datatype with 1 sign bit, 8 exponent bits and 7 fraction bits.
|
|
|
|
Note that support for a native bfloat16 datatype has not been added yet. This means that any datatype conversions to/from the new bfloat16 datatypes will be emulated in software rather than potentially using specialized hardware instructions. Until support for a native bfloat16 type is added, an application can avoid datatype conversion performance issues if it is sure that the datatype used for in-memory data buffers matches the above floating-point format (such as the __bf16 type). In this case, the application can specify one of the above macros for both the file datatype when creating a dataset or attribute and the memory datatype when performing I/O on the dataset or attribute.
|
|
|
|
### Added predefined datatypes for FP8 data
|
|
|
|
Predefined datatypes have been added for FP8 data in E4M3 and E5M2 formats (https://arxiv.org/abs/2209.05433).
|
|
|
|
The following new macros have been added:
|
|
|
|
- H5T_FLOAT_F8E4M3
|
|
- H5T_FLOAT_F8E5M2
|
|
|
|
These macros map to IDs of HDF5 datatypes representing an 8-bit floating-point datatype with 1 sign bit and either 4 exponent bits and 3 mantissa bits (E4M3 format) or 5 exponent bits and 2 mantissa bits (E5M2 format).
|
|
|
|
Note that support for a native FP8 datatype has not been added yet. This means that any datatype conversions to/from the new FP8 datatypes will be emulated in software rather than potentially using specialized hardware instructions. Until support for a native FP8 type is added, an application can avoid datatype conversion performance issues if it is sure that the datatype used for in-memory data buffers matches one of the above floating-point formats. In this case, the application can specify one of the above macros for both the file datatype when creating a dataset or attribute and the memory datatype when performing I/O on the dataset or attribute.
|
|
|
|
Also note that HDF5 currently has incomplete support for datatype conversions involving non-IEEE floating-point format datatypes. Refer to the 'Known Problems' section for information about datatype conversions with these new datatypes.
|
|
|
|
### Removed `hbool_t` from public API calls
|
|
|
|
The `hbool_t` type was introduced before the library supported C99's Boolean type. Originally typedef'd to an integer, it has been typedef'd to C99's bool for many years.
|
|
|
|
It had been previously purged from the bulk of the library code and only remained in public API signatures. In HDF5 2.0, it has also been removed from public API signatures.
|
|
|
|
The `hbool_t` typedef remains in H5public.h so existing code does not need to be updated.
|
|
|
|
### `H5public.h` no longer includes `features.h`
|
|
|
|
`features.h` is supposed to be included by glibc headers and not used in application code. It is unnecessary given our use of feature test macros like `_POSIX_C_SOURCE` and has been removed.
|
|
|
|
### Improved performance of opening a virtual dataset with many mappings
|
|
|
|
When opening a virtual dataset, the library would previously decode the mappings in the object header package, then copy them to the dataset struct, then copy them to the internal dataset creation property list. Copying the VDS mappings could be very expensive if there were many mappings. Changed this to delay decoding the mappings until the dataset code, and delay copying the layout to the DCPL until it is needed. This results in only the decoding and no copies in most use cases, as opposed to the decoding and two copies with the previous code.
|
|
|
|
### Changed the default page buffer size for the ROS3 driver
|
|
|
|
Calling `H5Pset_fapl_ros3()` now has the side effect of setting the page buffer size in the FAPL to 64 MiB if it was not previously set. This will only have an effect if the file uses paged allocation. Also added the `H5F_PAGE_BUFFER_SIZE_DEFAULT` to allow the user to unset the page buffer size in an FAPL so it can be similarly overridden.
|
|
|
|
### Default dataset chunk cache size increased
|
|
|
|
The default dataset chunk cache size was increased to 8 MiB (8,388,608 bytes).
|
|
|
|
### The file format has been updated to 4.0<a name="fileformat"></a>
|
|
|
|
The Virtual Dataset Global Heap Block format has been updated to version 1 to support shared string storage for source filenames and dataset names, reducing file size when multiple mappings reference the same sources. This new format is only used when the HDF5 library version bounds lower bound is set to 2.0 or later.
|
|
|
|
Use of the shared strings option for Virtual Datasets reduces memory overhead and optimizes dataset close operations.
|
|
|
|
The chunked dataset file format has been updated to always use 64 bits to encode the size of filtered chunks. This will allow data filters that expand the chunks by a large amount to still work. Chunk sizes are still limited to `2^32 - 1`. This new format is only used when the HDF5 library version bounds lower bound is set to 2.0 or later.
|
|
|
|
### The `H5Dread_chunk()` signature has changed
|
|
|
|
A new parameter, `nalloc`, has been added to `H5Dread_chunk()`. This parameter contains a pointer to a variable that holds the size of the buffer buf. If *nalloc is not large enough to hold the entire chunk being read, no data is read. On exit, the value of this variable is set to the buffer size needed to read the chunk.
|
|
|
|
The old signature has been renamed to `H5Dread_chunk1()` and is considered deprecated:
|
|
```c
|
|
herr_t H5Dread_chunk1(hid_t dset_id, hid_t dxpl_id,
|
|
const hsize_t *offset, uint32_t *filters,
|
|
void *buf);
|
|
```
|
|
|
|
The new signature is `H5Dread_chunk2()`. All code should be updated to use this version:
|
|
```c
|
|
herr_t H5Dread_chunk2(hid_t dset_id, hid_t dxpl_id,
|
|
const hsize_t *offset, uint32_t *filters,
|
|
void *buf, size_t *nalloc);
|
|
```
|
|
|
|
`H5Dread_chunk()` will map to the new signature unless the library is explicitly configured to use an older version of the API.
|
|
|
|
### Replaced the ROS3 VFD's S3 backend based on libcurl with a new backend based on the [aws-c-s3 library](https://github.com/awslabs/aws-c-s3)<a name="ros3"></a>
|
|
|
|
The ROS3 VFD now requires the `aws-c-s3` library in order to be built. This library offers several useful features for the VFD, including the following:
|
|
- Automatic retries of non-fatal failed requests (where the libcurl backend would simply return an error),
|
|
- Built-in sourcing of credentials and other configuration information from standard AWS configuration files and environment variables,
|
|
- Automatic splitting and parallelization of large S3 requests,
|
|
- Built-in handling of the formation of HTTP request authentication headers (where the libcurl backend had to manage this manually), and more.
|
|
|
|
Additional benefits with the rewrite of this backend include:
|
|
- Support for `s3://` object URIs,
|
|
- The ability to enable debugging information printouts at runtime by setting a new environment variable `HDF5_ROS3_VFD_DEBUG` (refer to `H5FDros3.h`),
|
|
- The ability to capture logging information from the `aws-c-s3` library by setting a new environment variable `HDF5_ROS3_VFD_LOG_LEVEL` (refer to `H5FDros3.h`).
|
|
|
|
With these changes, the behavior of the `authenticate` field of the ROS3 VFD's FAPL structure has changed slightly. If `authenticate` is true, the ROS3 VFD will _only_ use credentials specified in the FAPL structure and will not attempt to load credentials from other places. In this case, the `secret_id` and `secret_key` fields must still both be non-empty strings, or an error will be returned when opening a file. If a session token is to be used, it must be specified with `H5Pset_fapl_ros3_token()`.
|
|
|
|
If `authenticate` is false, the ROS3 VFD will instead attempt to load credentials from several different places, in this order:
|
|
- From the environment, by checking AWS environment variables such as `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN` and `AWS_ACCOUNT_ID`,
|
|
- From the AWS profile files, by reading from `~/.aws/config` and `~/.aws/credentials`, by default. The specific files read from can be overridden with the `AWS_CONFIG_FILE` and `AWS_SHARED_CREDENTIALS_FILE` environment variables,
|
|
- From STS, by using `AssumeRoleWithWebIdentity`,
|
|
- From EC2 instance metadata.
|
|
|
|
If the ROS3 VFD cannot source credentials from any of these locations, it will fall back to using anonymous credentials.
|
|
|
|
This functionality effectively deprecates the `--s3-cred` option for the `h5dump`, `h5ls`, and `h5stat` tools. However, this option has been kept for compatibility reasons and can still be used to force the specified credentials to take precedence over any credentials that the VFD would otherwise try to source.
|
|
|
|
Note that with these changes, the AWS region to be used must always be specified. However, the region can now be specified in several ways other than just in the FAPL structure. The VFD will search for a specified AWS region in the following order:
|
|
- The FAPL, if `aws_region` is not an empty string
|
|
- The `AWS_REGION` environment variable
|
|
- The `AWS_DEFAULT_REGION` environment variable
|
|
- The AWS configuration file (`~/.aws/config` by default)
|
|
- The `default` profile from this file is used, unless a different profile is specified with the AWS_PROFILE environment variable
|
|
|
|
If the ROS3 VFD cannot determine an AWS region from one of these locations, an error will be returned when opening a file.
|
|
|
|
New API functions `H5Pset_fapl_ros3_endpoint()` and `H5Pget_fapl_ros3_endpoint()` have been added for use with the ROS3 VFD. These functions set/get an alternate endpoint URL to use when opening files with the ROS3 VFD. This is useful in cases where the application needs to access files that are in a location other than the standard `s3.<region-code>.amazonaws.com`, which is what the ROS3 VFD uses when an alternate endpoint URL isn't specified. The ROS3 VFD also checks the `AWS_ENDPOINT_URL_S3` and `AWS_ENDPOINT_URL` environment variables for an alternate endpoint URL if one isn't specified with `H5Pset_fapl_ros3_endpoint()`.
|
|
|
|
Instructions for building the ROS3 VFD with the `aws-c-s3` library are in the [INSTALL_S3.txt](./INSTALL_S3.txt) file. The ROS3 VFD and information about the usage of the driver are described in the HDF5 user's guide.
|
|
|
|
### Renamed some API decorations
|
|
|
|
Some API decorations (used to hide `__declspec` on Windows, among other things) have been renamed:
|
|
|
|
| Old | New |
|
|
| --- | --- |
|
|
| `H5_DLLCPP(VAR)` | `H5CPP_DLL(VAR)` |
|
|
| `H5_HLDLL(VAR)` | `H5HL_DLL(VAR)` |
|
|
| `H5_HLCPPDLL(VAR)` | `H5CPP_HL_DLL(VAR)` |
|
|
| `H5_FCDLL(VAR)` | `H5FC_DLL(VAR)` |
|
|
| `H5_FCTESTDLL(VAR)` | `H5FC_TEST_DLL(VAR)` |
|
|
| `HDF5_HL_F90CSTUBDLL(VAR)` | `H5FC_HL_DLL(VAR)` |
|
|
|
|
### The `H5Iregister_type()` signature has changed
|
|
|
|
The hash_size parameter has not been used since early versions of HDF5 1.8, so it has been removed, and the API call has been versioned.
|
|
|
|
The old signature has been renamed to `H5Iregister_type1()` and is considered deprecated:
|
|
|
|
```c
|
|
H5I_type_t H5Iregister_type1(size_t hash_size, unsigned reserved, H5I_free_t free_func);
|
|
```
|
|
|
|
The new signature is `H5Iregister_type2()`. New code should use this version:
|
|
|
|
```c
|
|
H5I_type_t H5Iregister_type2(unsigned reserved, H5I_free_t free_func);
|
|
```
|
|
|
|
`H5Iregister_type()` will map to the new signature unless the library is explicitly configured to use an older version of the API.
|
|
|
|
### The `H5Tdecode()` signature has changed
|
|
|
|
When provided malformed or too-small buffers, `H5Tdecode()` would crash. The new buffer size parameter allows this to be reliably avoided.
|
|
|
|
The old signature has been renamed to `H5Tdecode1()` and is considered deprecated:
|
|
|
|
hid_t H5Tdecode1(const void *buf);
|
|
|
|
The new signature is `H5Tdecode2()`. New code should use this version:
|
|
|
|
hid_t H5Tdecode2(const void *buf, size_t buf_size);
|
|
|
|
`H5Tdecode()` will map to the new signature unless the library is explicitly configured to use an older version of the API.
|
|
|
|
### `H5F_LIBVER_LATEST` is now an enum value
|
|
|
|
This was previously #defined to the latest `H5F_libver_t API` version, but is now an enum value with an integer value equal to the latest `H5F_libver_t` API version's value. e.g.:
|
|
|
|
```
|
|
<snip>
|
|
H5F_LIBVER_V200 = 5,
|
|
H5F_LIBVER_LATEST = 5,
|
|
</snip>
|
|
```
|
|
### Added support for complex number datatypes<a name="complex"></a>
|
|
|
|
Support for the C99 `float _Complex`, `double _Complex` and `long double _Complex` (with MSVC, `_Fcomplex`, `_Dcomplex` and `_Lcomplex`) types has been added for platforms/compilers that support them. These types have been implemented with a new datatype class, `H5T_COMPLEX`. Note that any datatypes of class H5T_COMPLEX will not be readable with previous versions of HDF5. If a file is accessed with a library version bounds "high" setting less than `H5F_LIBVER_V200`, an error will occur if the application tries to create an object with a complex number datatype. If compatibility with previous versions of HDF5 is desired, applications should instead consider adopting [one of the existing conventions](https://nc-complex.readthedocs.io/en/latest/#conventions-used-in-applications).
|
|
|
|
The following new macros have been added:
|
|
|
|
| Macro | Description |
|
|
| --- | --- |
|
|
| `H5_HAVE_COMPLEX_NUMBERS` | This macro is defined in `H5pubconf.h` and will have the value 1 if native support for complex numbers is available. It will not be defined otherwise. |
|
|
| `H5_HAVE_C99_COMPLEX_NUMBERS` | This macro is defined in `H5pubconf.h` and will have the value 1 if native support for C99 complex numbers is available. It will not be defined otherwise. If this macro is not defined but `H5_HAVE_COMPLEX_NUMBERS` is defined, the complex number types supported are the MSVC types. |
|
|
| `H5_SIZEOF_FLOAT_COMPLEX` | This macro is defined in `H5pubconf.h` and will have a value corresponding to the size of the native float complex datatype, as computed by sizeof(). If C99 complex number support is available, this will be the size of the "float _Complex" type. Otherwise, it will be the size of the `_Fcomplex` type. It will have the value 0 if support for a native float complex datatype is not available. |
|
|
| `H5_SIZEOF_DOUBLE_COMPLEX` | This macro is defined in `H5pubconf.h` and will have a value corresponding to the size of the native double complex datatype, as computed by sizeof(). If C99 complex number support is available, this will be the size of the `double _Complex` type. Otherwise, it will be the size of the `_Dcomplex` type. It will have the value 0 if support for a native double complex datatype is not available. |
|
|
| `H5_SIZEOF_LONG_DOUBLE_COMPLEX` | This macro is defined in `H5pubconf.h` and will have a value corresponding to the size of the native long double complex datatype, as computed by sizeof(). If C99 complex number support is available, this will be the size of the `long double _Complex` type. Otherwise, it will be the size of the `_Lcomplex` type. It will have the value 0 if support for a native long double complex datatype is not available. |
|
|
| ``H5T_NATIVE_FLOAT_COMPLEX`` | This macro maps to the ID of an HDF5 datatype representing the native C float complex datatype (either "float _Complex" or `_Fcomplex`) for the platform. If support for a native float complex datatype is not available (`H5_HAVE_COMPLEX_NUMBERS` is not defined), the macro will map to `H5I_INVALID_HID` and should not be used. |
|
|
| ``H5T_NATIVE_DOUBLE_COMPLEX`` | This macro maps to the ID of an HDF5 datatype representing the native C double complex datatype (either `double _Complex` or `_Dcomplex`) for the platform. If support for a native double complex datatype is not available (`H5_HAVE_COMPLEX_NUMBERS` is not defined), the macro will map to `H5I_INVALID_HID` and should not be used. |
|
|
| ``H5T_NATIVE_LDOUBLE_COMPLEX`` | This macro maps to the ID of an HDF5 datatype representing the native C long double complex datatype (either `long double _Complex` or `_Lcomplex`) for the platform. If support for a native long double complex datatype is not available (`H5_HAVE_COMPLEX_NUMBERS` is not defined), the macro will map to `H5I_INVALID_HID` and should not be used. |
|
|
| `H5T_COMPLEX_IEEE_F16LE` / `H5T_COMPLEX_IEEE_F16BE` | These macros map to IDs of HDF5 datatypes representing a complex number of two parts, each of which is an IEEE 754 16-bit floating-point datatype in little- or big-endian order. These datatypes are available regardless of whether complex number support is available or not. |
|
|
| `H5T_COMPLEX_IEEE_F32LE` / `H5T_COMPLEX_IEEE_F32BE` | These macros map to IDs of HDF5 datatypes representing a complex number of two parts, each of which is an IEEE 754 32-bit floating-point datatype in little- or big-endian order. These datatypes are available regardless of whether complex number support is available or not. |
|
|
| `H5T_COMPLEX_IEEE_F64LE` / `H5T_COMPLEX_IEEE_F64BE` | These macros map to IDs of HDF5 datatypes representing a complex number of two parts, each of which is an IEEE 754 64-bit floating-point datatype in little- or big-endian order. These datatypes are available regardless of whether complex number support is available or not. |
|
|
|
|
The following new API function has been added:
|
|
|
|
| Function | Description |
|
|
| --- | --- |
|
|
| `hid_t H5Tcomplex_create(hid_t base_type_id)` | Creates a new complex number datatype from the base datatype specified by the given HDF5 ID `base_type_id`. The base datatype must be a floating-point datatype. |
|
|
|
|
The following new hard datatype conversion paths have been added, but will only be used when complex number support is available:
|
|
|
|
| | |
|
|
| -- | -- |
|
|
| `H5T_NATIVE_SCHAR` <-> `H5T_NATIVE_FLOAT_COMPLEX` | `H5T_NATIVE_UCHAR` <-> `H5T_NATIVE_FLOAT_COMPLEX` |
|
|
| `H5T_NATIVE_SHORT` <-> `H5T_NATIVE_FLOAT_COMPLEX` | `H5T_NATIVE_USHORT` <-> `H5T_NATIVE_FLOAT_COMPLEX` |
|
|
| `H5T_NATIVE_INT` <-> `H5T_NATIVE_FLOAT_COMPLEX` | `H5T_NATIVE_UINT`<-> `H5T_NATIVE_FLOAT_COMPLEX` |
|
|
| `H5T_NATIVE_LONG`<-> `H5T_NATIVE_FLOAT_COMPLEX` | `H5T_NATIVE_ULONG` <-> `H5T_NATIVE_FLOAT_COMPLEX` |
|
|
| `H5T_NATIVE_LLONG` <-> `H5T_NATIVE_FLOAT_COMPLEX` | `H5T_NATIVE_ULLONG` <-> `H5T_NATIVE_FLOAT_COMPLEX` |
|
|
| `H5T_NATIVE_FLOAT16` <-> `H5T_NATIVE_FLOAT_COMPLEX` | `H5T_NATIVE_FLOAT` <-> `H5T_NATIVE_FLOAT_COMPLEX` |
|
|
| `H5T_NATIVE_DOUBLE` <-> `H5T_NATIVE_FLOAT_COMPLEX` | `H5T_NATIVE_LDOUBLE` <-> `H5T_NATIVE_FLOAT_COMPLEX` |
|
|
| | |
|
|
| `H5T_NATIVE_SCHAR` <-> `H5T_NATIVE_DOUBLE_COMPLEX` | `H5T_NATIVE_UCHAR` <-> `H5T_NATIVE_DOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_SHORT` <-> `H5T_NATIVE_DOUBLE_COMPLEX` | `H5T_NATIVE_USHORT` <-> `H5T_NATIVE_DOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_INT` <-> `H5T_NATIVE_DOUBLE_COMPLEX` | `H5T_NATIVE_UINT`<-> `H5T_NATIVE_DOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_LONG`<-> `H5T_NATIVE_DOUBLE_COMPLEX` | `H5T_NATIVE_ULONG` <-> `H5T_NATIVE_DOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_LLONG` <-> `H5T_NATIVE_DOUBLE_COMPLEX` | `H5T_NATIVE_ULLONG` <-> `H5T_NATIVE_DOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_FLOAT16` <-> `H5T_NATIVE_DOUBLE_COMPLEX` | `H5T_NATIVE_FLOAT` <-> `H5T_NATIVE_DOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_DOUBLE` <-> `H5T_NATIVE_DOUBLE_COMPLEX` | `H5T_NATIVE_LDOUBLE` <-> `H5T_NATIVE_DOUBLE_COMPLEX` |
|
|
| | |
|
|
| `H5T_NATIVE_SCHAR` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | `H5T_NATIVE_UCHAR` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_SHORT` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | `H5T_NATIVE_USHORT` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_INT` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | `H5T_NATIVE_UINT`<-> `H5T_NATIVE_LDOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_LONG` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | `H5T_NATIVE_ULONG` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_LLONG` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | `H5T_NATIVE_ULLONG` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_FLOAT16` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | `H5T_NATIVE_FLOAT` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` |
|
|
| `H5T_NATIVE_DOUBLE` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | `H5T_NATIVE_LDOUBLE` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` |
|
|
| | |
|
|
| `H5T_NATIVE_FLOAT_COMPLEX` <-> `H5T_NATIVE_DOUBLE_COMPLEX` | |
|
|
| `H5T_NATIVE_FLOAT_COMPLEX` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | |
|
|
| `H5T_NATIVE_DOUBLE_COMPLEX` <-> `H5T_NATIVE_LDOUBLE_COMPLEX` | |
|
|
|
|
Alternative software implementation conversion paths have been added for all of the above for use when native complex number support is not available. All of these conversion paths follow the behavior outlined in the C standard for conversions of complex number values.
|
|
|
|
Additionally, a special datatype conversion path has been added between complex number datatypes and array or compound datatypes, where the in-memory layout of data is the same between the datatypes and data can be directly converted. This conversion path is subject to the following rules:
|
|
|
|
- An array datatype must consist of exactly two elements, where each element is of the same floating-point datatype as the complex number datatype's base floating-point datatype.
|
|
- A compound datatype must consist of exactly two fields, where each field is of the same floating-point datatype as the complex number datatype's base floating-point datatype. The compound datatype must not have any leading or trailing structure padding or any padding between its two fields. The fields must also have compatible names, must have compatible offsets within the datatype and must be in the order of "real" part -> "imaginary" part, such that the compound datatype matches the following representation:
|
|
|
|
```
|
|
H5T_COMPOUND {
|
|
<float_type> "r(e)(a)(l)"; OFFSET 0
|
|
<float_type> "i(m)(a)(g)(i)(n)(a)(r)(y)"; OFFSET SIZEOF("r(e)(a)(l)")
|
|
}
|
|
```
|
|
|
|
where `r(e)(a)(l)` means the field may be named any substring of "real", such as "r", or "re", and `i(m)(a)(g)(i)(n)(a)(r)(y)` means the field may be named any substring of "imaginary", such as "im" or "imag".
|
|
|
|
Support for complex numbers has been added to the `h5dump`, `h5ls`, and `h5diff`/`ph5diff` tools. The `h5dump` command-line option '-m' can be used to change the floating-point printing format for the float complex and double complex datatypes, as well as the long double complex datatype if it has the same size as a double complex datatype.
|
|
|
|
Support for the predefined complex number datatypes and the `H5Tcomplex_create` function has been added to the Java wrappers. However, Java does not have official types for complex numbers, so an application must be sure that data is in an appropriate format in-memory when using these datatypes.
|
|
|
|
Support for the Fortran wrappers has not yet been added.
|
|
|
|
Support for the predefined complex number datatypes and the `H5Tcomplex_create` function has been added to the high-level library, allowing them to work with the `H5LTtext_to_dtype` and `H5LTdtype_to_text` functions.
|
|
|
|
Simple example programs showing how to use complex number datatypes have been added in the following files:
|
|
|
|
- `HDF5Examples/C/H5T/200/h5ex_t_complex.c` (Uses C99 complex number types)
|
|
- `HDF5Examples/C/H5T/200/h5ex_t_complex_msvc.c` (Uses MSVC complex number types)
|
|
- `HDF5Examples/C/H5T/200/h5ex_t_complex_custom.c` (Uses `H5Tcomplex_create()` to create a custom complex number type)
|
|
|
|
### FOR VOL DEVELOPERS: Renamed `H5VLstart_lib_state` and `H5VLfinish_lib_state`
|
|
|
|
The APIs `H5VLstart_lib_state` and `H5VLfinish_lib_state` have been renamed to `H5VLopen_lib_context` and `H5VLclose_lib_context`, respectively, with the addition of a "context" argument.
|
|
|
|
### Removed `H5FDperform_init` API routine.
|
|
|
|
Virtual File Driver (VFD) developers who wish to provide an ID for their driver should create a routine specific to their individual implementation.
|
|
|
|
### `H5Pset_external()` now uses `HDoff_t`, which is always a 64-bit type
|
|
|
|
The `H5Pset_external()` call took an off_t parameter in HDF5 1.14.x and earlier. On POSIX systems, off_t is specified as a 64-bit type via POSIX large-file support (LFS). On Windows, however, off_t is defined as a 32-bit type, even on 64-bit Windows.
|
|
|
|
HDoff_t has been added to H5public.h and is defined to be int64_t on Windows and the library has been updated to use HDoff_t in place of off_t throughout. The H5Pset_external() offset parameter has also been updated to be HDoff_t.
|
|
|
|
There is no API compatibility wrapper for this change.
|
|
|
|
Fixes GitHub issue [#3506](https://github.com/HDFGroup/hdf5/issues/3506)
|
|
|
|
### `H5Pset*` routines now fail when used on default property lists
|
|
|
|
Modifying default property lists was never fully supported and could produce inconsistent and unexpected behavior.
|
|
|
|
### `H5Pset_vol()` now fails when used on a non-file-access property list
|
|
|
|
Similar to the above. Setting the connector on a non-FAPL had no effect on library behavior, and the connector ID and information could not be read back from that plist later.
|
|
|
|
### Optimized Virtual Dataset opens by delaying layout copy<a name="layoutcopydelay"></a>
|
|
|
|
On dataset open, the dataset performed an internal copy of the layout in order to populate its internal DCPL. For virtual datasets, this added a significant amount of overhead to the open operation.
|
|
|
|
This layout copy is now delayed until either a user requests the DCPL, or until the start of an operation that needs to read the layout from the DCPL.
|
|
|
|
### Virtual datasets now use a spatial tree to optimize searches<a name="rtree"></a>
|
|
|
|
Virtual dataset operations with many (>1,000) mappings were much slower than
|
|
corresponding operations on normal datasets. This was due to the need
|
|
to iterate through every source dataset's dataspace and check for an intersection
|
|
with the user-selected region for a read/write in the virtual dataset.
|
|
|
|
Virtual datasets with many mappings now use an r-tree (defined in H5RT.c) to
|
|
perform a spatial search. This allows the dataspaces that intersect the
|
|
user-selection to be computed with, in most cases, much fewer intersection checks,
|
|
improving the speed of VDS read/write operations.
|
|
|
|
Virtual datasets will use the r-tree by default, since the majority of use cases,
|
|
should see improvements from use of the tree. However, because some workflows may
|
|
find that the overhead of the tree outweighs the time saved on searches, there is
|
|
a new Dataset Access Property List (DAPL) property to control use of the spatial tree.
|
|
|
|
This property can be set or queried with the new API functions
|
|
`H5Pset_virtual_spatial_tree()`/`H5Pget_virtual_spatial_tree()`.
|
|
|
|
## Parallel Library
|
|
|
|
### Added `H5FDsubfiling_get_file_mapping()` API function for subfiling VFD
|
|
|
|
Added `H5FDsubfiling_get_file_mapping()` API function to retrieve the names of all physical subfiles that collectively make up a logical HDF5 file when using the subfiling Virtual File Driver.
|
|
|
|
## Fortran Library
|
|
|
|
### Added Fortran wrapper `h5fdsubfiling_get_file_mapping_f()` for subfiling VFD
|
|
|
|
Added Fortran wrapper `h5fdsubfiling_get_file_mapping_f()` for the subfiling file mapping functionality, ensuring complete language binding support.
|
|
|
|
## C++ Library
|
|
|
|
## Java Library
|
|
|
|
## Tools
|
|
|
|
### Added AWS endpoint command option to allow specifying an alternate endpoint URL when using the ROS3 VFD
|
|
|
|
The new option is `--endpoint-url`, which allows the user to set an alternate endpoint URL other than the standard "protocol://service-code.region-code.amazonaws.com". If `--endpoint-url` is not specified, the ROS3 VFD will first check the `AWS_ENDPOINT_URL_S3` and `AWS_ENDPOINT_URL` environment variables for an alternate endpoint URL before using a default one, with the region-code being supplied by the FAPL or standard AWS locations/environment variables.
|
|
|
|
This option is supported by the following tools:
|
|
`h5dump`, `h5ls`, `h5stat`
|
|
|
|
### Specifying ROS3 VFD on the command line is not required when using S3 URI
|
|
|
|
If using an S3 URI to reference an HDF5 file in S3 (example: s3://mybucket/myfile.h5), then ROS3 VFD will be automatically selected unless the command-line option for the virtual file driver is used.
|
|
|
|
This feature applies to the following tools: `h5dump`, `h5ls`, `h5stat`.
|
|
|
|
### Deprecated `h5dump` XML option
|
|
|
|
The `h5dump` XML option is deprecated and will be removed in a future release. The XML output format has not been maintained and is not up-to-date with the latest features of HDF5.
|
|
|
|
### Added `h5dump` command option to set the floating point format for long double
|
|
|
|
The new option is --lformat, which allows the user to set the floating point format for long double. The default format is %Lg. There is already an option --format to set the floating point format for double and float. The default format is %g.
|
|
|
|
### Removed the high-level GIF tools
|
|
|
|
The high-level GIF tools, `h52gif` and `gif2h5`, have unfixed CVE issues (with no proof-of-concept files). They are not critical tools, are not well-maintained, and are an odd fit for building with the library. Because of this, they have been removed. We may move them to a separate repository in the future.
|
|
|
|
This also removed the following configure options:
|
|
CMake: `HDF5_BUILD_HL_GIF_TOOLS`
|
|
|
|
## High-Level APIs
|
|
|
|
## C Packet Table API
|
|
|
|
## Internal header file
|
|
|
|
## Documentation
|
|
|
|
### The COPYING file has been renamed to LICENSE
|
|
|
|
This is where most people will expect to find license information. The `COPYING_LBNL_HDF5` file has also been renamed to `LICENSE_LBNL_HDF5`. The licenses are unchanged.
|
|
|
|
# 🪲 Bug Fixes
|
|
|
|
## Library
|
|
|
|
### Fixed problems with the family driver and user block
|
|
|
|
When using a user block with the family driver, the driver would inappropriately subtract the user block size for each member file when calculating member EOAs. This could cause a failure when an address overflowed the calculated eoa. The driver would also add the user block size when returning the EOF. Modified the family driver to not consider the user block, as it is handled by the H5FD layer. The user block now spans the first X bytes of the family array, for example a 4 KiB user block with 3 KiB member size will take up the entire first member and the first 1 KiB of the second. This may cause compatibility issues with preexisting family files with user blocks, though the way it worked before was inconsistent if it worked at all.
|
|
|
|
### Fixed security issue CVE-2025-7067
|
|
|
|
Fixed a heap buffer overflow in H5FS__sinfo_serialize_node_cb() by discarding file free space sections from the file free space manager when they are found to be invalid. Specifically crafted HDF5 files can result in an attempt to insert duplicate or overlapping file free space sections into a file free space manager, later resulting in a buffer overflow when the same free space section is serialized to the file multiple times.
|
|
|
|
Fixes GitHub issue [#5577](https://github.com/HDFGroup/hdf5/issues/5577)
|
|
|
|
### Fixed security issue [CVE-2025-2915](https://nvd.nist.gov/vuln/detail/CVE-2025-2915) and [OSV-2024-381](https://osv.dev/vulnerability/OSV-2024-381)
|
|
|
|
Fixed a heap-based buffer overflow in H5F__accum_free caused by an integer overflow when calculating new_accum_size. Added validation in H5O__mdci_decode to detect and reject invalid values early, preventing the overflow condition.
|
|
|
|
Fixes GitHub issue [#5380](https://github.com/HDFGroup/hdf5/issues/5380)
|
|
|
|
### Fixed security issue [CVE-2025-7068](https://nvd.nist.gov/vuln/detail/CVE-2025-7068)
|
|
|
|
Failures during the discard process on a metadata cache entry could cause the library to skip calling the callback to free the cache entry. This could result in resource leaks and issues with flushing and closing the metadata cache during file close. This has been fixed by noting errors during the discard process, but attempting to fully free a cache entry before signalling that an error has occurred.
|
|
|
|
Fixes GitHub issue [#5578](https://github.com/HDFGroup/hdf5/issues/5578)
|
|
|
|
### Fix bugs in object header operations
|
|
|
|
In some rare circumstances, such as deleting hard links that point to their own parent group in a file using the new file format, memory corruption could occur due to recursive operations changing data structures being operated on by multiple levels of recursion. Made changes to delay changing the data structure in a dangerous way until recursion is complete.
|
|
|
|
Fixes GitHub issue [#5854](https://github.com/HDFGroup/hdf5/issues/5854)
|
|
|
|
### Fixed security issues [CVE-2025-6816](https://nvd.nist.gov/vuln/detail/CVE-2025-6816), [CVE-2025-6818](https://nvd.nist.gov/vuln/detail/CVE-2025-6818), [CVE-2025-6856](https://nvd.nist.gov/vuln/detail/CVE-2025-6856) and [CVE-2025-2923](https://nvd.nist.gov/vuln/detail/CVE-2025-2923)
|
|
|
|
A specially constructed HDF5 file could contain a corrupted object header with a continuation message that points back to itself. This could result in an internal buffer being allocated with too small of a size, leading to a heap buffer overflow. This has been fixed by checking the expected number of object header chunks against the actual value as chunks are being deserialized.
|
|
|
|
Fixes GitHub issues [#5571](https://github.com/HDFGroup/hdf5/issues/5571), [#5573](https://github.com/HDFGroup/hdf5/issues/5573), [#5574](https://github.com/HDFGroup/hdf5/issues/5574) and [#5381](https://github.com/HDFGroup/hdf5/issues/5381)
|
|
|
|
### Fixed security issue [CVE-2025-6750](https://nvd.nist.gov/vuln/detail/CVE-2025-6750)
|
|
|
|
A heap buffer overflow occurred because an mtime message was not properly decoded, resulting in a buffer of size 0 being passed into the encoder. This has been fixed by decoding old and new mtime messages which will allow invalid message size to be detected.
|
|
|
|
Fixes GitHub issue [#5549](https://github.com/HDFGroup/hdf5/issues/5549)
|
|
|
|
### Fixed [CVE-2025-7069](https://nvd.nist.gov/vuln/detail/CVE-2025-7069)
|
|
|
|
An invalid page size in file space info was decoded, which caused a heap buffer overflow when reading certain compromised data files. The invalid value has been detected for a graceful failure.
|
|
|
|
Fixes GitHub issues [#5550](https://github.com/HDFGroup/hdf5/issues/5550) and OSS issues [OSV-2024-379](https://osv.dev/vulnerability/OSV-2024-379), [OSV-2024-575](https://osv.dev/vulnerability/OSV-2024-575), and [OSV-2024-772](https://osv.dev/vulnerability/OSV-2024-772)
|
|
|
|
### Fixed [CVE-2025-6269](https://nvd.nist.gov/vuln/detail/CVE-2025-6269) and [CVE-2025-6516](https://nvd.nist.gov/vuln/detail/CVE-2025-6516)
|
|
|
|
There were several security vulnerabilities found in the function H5C__reconstruct_cache_entry(), including buffer overflows and memory leaks. The function has been hardened with bounds checks, input validation, and safe cleanup.
|
|
|
|
Fixes GitHub issues [#5579](https://github.com/HDFGroup/hdf5/issues/5579) and [#5581](https://github.com/HDFGroup/hdf5/issues/5581)
|
|
|
|
### Fixed a problem with the scale-offset filter
|
|
|
|
A security fix added to 1.14.6 introduced a regression where certain data values could trigger a library error (not a crash or segfault).
|
|
|
|
Fixes GitHub issue [#5861](https://github.com/HDFGroup/hdf5/issues/5861)
|
|
|
|
### Fixed security issue [CVE-2025-2153](https://nvd.nist.gov/vuln/detail/CVE-2025-2153)
|
|
|
|
The message flags field could be modified such that a message that is not sharable according to the `share_flags` field in `H5O_msg_class_t` can be treated as sharable. An assert has been added in `H5O__msg_write_real` to make sure messages that are not sharable can't be modified to shared. Additionally, the check in `H5O__chunk_deserialize` that catches unsharable messages being marked as sharable has been improved.
|
|
|
|
Fixes GitHub issue [#5329](https://github.com/HDFGroup/hdf5/issues/5329) and [OSV-2023-76](https://osv.dev/vulnerability/OSV-2023-76)
|
|
|
|
### Fixed security issue [CVE-2025-2925](https://nvd.nist.gov/vuln/detail/CVE-2025-2925)
|
|
|
|
`Actual_len` + `H5C_IMAGE_EXTRA_SPACE`, which was used by `H5MM_realloc` as the size input, could equal 0 due to bad inputs. When `H5MM_realloc` was called, it freed image, but then could get sent to done before new_image could be assigned to image. Because the pointer for image wasn't null, it was freed again in done, causing a double-free vulnerability. `H5C__load_entry()` now checks for an image buffer length of 0 before calling `H5MM_realloc`.
|
|
|
|
Fixes GitHub issue [#5383](https://github.com/HDFGroup/hdf5/issues/5383)
|
|
|
|
### Fixed security issue [CVE-2025-6857](https://nvd.nist.gov/vuln/detail/CVE-2025-6857)
|
|
|
|
An HDF5 file had a corrupted v1 B-tree that would result in a stack overflow when performing a lookup on it. This has been fixed with additional integrity checks.
|
|
|
|
Fixes GitHub issue [#5575](https://github.com/HDFGroup/hdf5/issues/5575)
|
|
|
|
### Check for overflow in decoded heap block addresses
|
|
|
|
Currently, we do not check for overflow when decoding addresses from the heap, which can cause overflow problems. We've added a check in `H5HL__fl_deserialize` to ensure no overflow can occur.
|
|
|
|
Fixes GitHub issue [#5382](https://github.com/HDFGroup/hdf5/issues/5382)
|
|
|
|
### Fixed a segfault with a corrupted file based on netCDF-4
|
|
|
|
When the internal function H5G__dense_iterate failed early, allocated structures were never populated, causing cleanup code to read uninitialized values and attempt to free invalid pointers. The structures are now initialized with proper values and NULL pointers immediately after allocation.
|
|
|
|
Fixes GitHub issue [#5375](https://github.com/HDFGroup/hdf5/issues/5375)
|
|
|
|
### Fixed security issues [CVE-2025-2913](https://nvd.nist.gov/vuln/detail/CVE-2025-2913), [CVE-2025-2926](https://nvd.nist.gov/vuln/detail/CVE-2025-2926), [CVE-2025-6817](https://nvd.nist.gov/vuln/detail/CVE-2025-6817), and [CVE-2025-6858](https://nvd.nist.gov/vuln/detail/CVE-2025-6858)
|
|
|
|
The size of a continuation message was decoded as 0, causing multiple vulnerabilities. An error check was added to return failure to prevent further processing of invalid data.
|
|
|
|
Fixes GitHub issue [#5376](https://github.com/HDFGroup/hdf5/issues/5376), [#5384](https://github.com/HDFGroup/hdf5/issues/5384), [#5572](https://github.com/HDFGroup/hdf5/issues/5572), and [#5576](https://github.com/HDFGroup/hdf5/issues/5576)
|
|
|
|
### Revised handling of Unicode filenames on Windows<a name="utf-8">
|
|
|
|
In the HDF5 1.14.4 release, a change was made to address some issues with the library's handling of code pages and file paths on Windows. This change introduced other issues with the handling of UTF-8 file names that caused breakage for software using the 1.14.4 and 1.14.5 releases of HDF5. That change was reverted for the 1.14.6 release and the behavior has been slightly modified for this release.
|
|
|
|
On Windows, the library once again assumes that filename strings will be UTF-8 encoded strings and will attempt to convert them to UTF-16 before passing them to Windows API functions. However, if the library fails to convert a filename string to UTF-16, it will now fallback to the equivalent Windows "ANSI" API functions which will interpret the string according to the active Windows code page.
|
|
|
|
Support for a new environment variable, `HDF5_PREFER_WINDOWS_CODE_PAGE`, was added in order to instruct HDF5 to prefer interpreting filenames according to the active Windows code page rather than assuming UTF-8 encoding. If this environment variable is set to "1" or "TRUE" (case-insensitive), the active code page will be preferred. If it is unset or set to "0" or "FALSE" (case-insensitive), UTF-8 will be preferred.
|
|
|
|
### Fixed an issue with caching in the ROS3 VFD
|
|
|
|
The ROS3 VFD uses a very simple caching mechanism that caches the first 16MiB of a file during file open and serves later reads from that cache if the offset + length falls within the cached range of bytes. Combinations of offset + length that extended exactly to the end of the cached range of bytes (for example, offset=0 and len=16777216) would end up not being served from the cache due to an incorrect range check. This has now been fixed.
|
|
|
|
### Fixed an error with `H5Fget_file_image()` with the latest file format
|
|
|
|
When using `H5Fget_file_image()` on a file created with the latest file format (or any format newer than the earliest), the library failed to recalculate the superblock checksum after changing the access flags in the superblock, causing any subsequent attempt to open the returned file image to fail due to the checksum failing to verify. Fixed `H5Fget_file_image()` to recalculate the checksum.
|
|
|
|
Fixed GitHub issue [#1915](https://github.com/HDFGroup/hdf5/issues/1915)
|
|
|
|
### Fixed an assertion failure in `H5S__hyper_make_spans()`
|
|
|
|
Calling H5Sselect_hyperslab() on dataspaces with invalid extents could result in an assertion failure in debug builds of the library if the dataspace has an extent with a rank value of 0. This has been fixed by converting the assertion failure into a normal error check.
|
|
|
|
### Fixed an assertion failure in `H5S__hyper_new_span_info()`
|
|
|
|
Calling `H5Scopy()` on hyperslab selection dataspaces with invalid extents could result in an assertion failure in debug builds of the library if the dataspace has an extent with a rank value of 0. This has been fixed by converting the assertion failure into a normal error check.
|
|
|
|
### Fixed a segfault in `H5S__get_select_hyper_blocklist()`
|
|
|
|
When attempting to retrieve the list of hyperslab blocks selected within a dataspace, a segfault or bus error could occur when the dataspace has an extent with a rank value of 0. This would cause indexing into an array variable on the stack using a negative value. An error check was added to return failure from the function for such dataspaces.
|
|
|
|
### Fixed an error in `H5Ddebug`
|
|
|
|
`H5Ddebug` would fail for any chunked dataset with a chunk index, due to its failure to tag the dataset before performing metadata operations. This caused `h5ls -va` to silently fail to print chunk addresses. This has been fixed.
|
|
|
|
### Fixed a bug in the `H5Oexists` and `H5Oexists_by_name` API routines that would cause those routines to return FAIL instead of FALSE when checking the existence of a non-existent object with a file ID instead of a group ID.
|
|
|
|
### Fixed a segfault in h5dump when a B-tree node level is corrupted
|
|
`h5dump` produced a segfault on a malformed file because a B-tree node level was corrupted.
|
|
|
|
An internal function was modified to help detect when a decoded B-tree node level has an unexpected value, and an error will be produced.
|
|
|
|
Fixed GitHub issue [#4432](https://github.com/HDFGroup/hdf5/issues/4432)
|
|
|
|
### Fixed `H5Ovisit2` to recursively visit all objects
|
|
|
|
`H5Ovisit2` visited only the root group and not all the nested groups.
|
|
|
|
This behavior occurred when the fields are not `H5O_INFO_BASIC` or `H5O_INFO_ALL` because an internal function did not obtain the basic information needed by its caller. This problem is now fixed.
|
|
|
|
Fixed GitHub issue [#4941](https://github.com/HDFGroup/hdf5/issues/4941)
|
|
|
|
### Only clear `FE_INVALID` when that symbol is present on the system
|
|
|
|
When we initialize the floating-point types at library startup, it's possible to raise floating-point exceptions when we check which things are supported. Normally, we clear these floating-point exceptions via `feclearexcept(FE_INVALID)`, but `FE_INVALID` may not be present on all systems. Specifically, this was reported as being a problem when using Emscripten 3.1.68 to compile HDF5 1.14.5 to WebAssembly.
|
|
|
|
We've added an #ifdef `FE_INVALID` block around the exception clearing code to correct this.
|
|
|
|
Fixed GitHub issue [#4952](https://github.com/HDFGroup/hdf5/issues/4952)
|
|
|
|
### Fixed security issue [CVE-2025-2310](https://nvd.nist.gov/vuln/detail/CVE-2025-2310)
|
|
|
|
A malformed HDF5 file could have an attribute with a recorded name length of zero.This would lead to an overflow and an invalid memory access. An integrity check has been added to detect this case and safely stop file decoding.
|
|
|
|
## Java Library
|
|
|
|
### Renamed the Callbacks.java file to H5Callbacks.java
|
|
|
|
The Callbacks.java file was renamed to H5Callbacks.java to match the file pattern used by doxygen. This change only affects the Java filenames and does not change the classname or the package name.
|
|
|
|
## Configuration
|
|
|
|
### Reorganized CMake HDF5 configuration options
|
|
|
|
The CMake configuration options have been reorganized to identify the primary options that are relevant to the build. These options are now in a separate file, CMakeBuildOptions.cmake, which is included by the root CMakeLists.txt file. In addition, some options have been converted to `cmake_dependent_option()` calls, which allows the options to be hidden from the CMake GUI when they are not relevant to the build.
|
|
|
|
### Remove default setting of `CMAKE_DEBUG_POSTFIX`
|
|
|
|
Move the default setting of `CMAKE_DEBUG_POSTFIX` to the cacheinit.cmake file usually used by testing. If `CMAKE_DEBUG_POSTFIX` is not set with a -D option then `CMAKE_DEBUG_POSTFIX` will be the default provided by CMake itself.
|
|
|
|
### The relative rpaths ($ORIGIN / @loader_path) are appended to the `CMAKE_INSTALL_RPATH`
|
|
|
|
The RPATH settings were removed by a pull-request [#5271](https://github.com/HDFGroup/hdf5/pull/5271), but the settings are needed under certain conditions. These settings have been restored by appending the necessary paths and will not override/overwrite any existing settings.
|
|
|
|
### When using a system-installed zlib library, the shared library is expected to be found in the system library path.
|
|
|
|
Setting the `HDF5_MODULE_MODE_ZLIB` option to OFF will force find_package to use config mode first. An installed zlib, or an alternate installed zlib library, is expected to have a correct zlib-config.cmake file for config mode. Current zlib installs usually do not have a zlib-config.cmake file, so the option is set to ON by default.
|
|
|
|
### Use pre-installed libaec compression library
|
|
|
|
The CMake logic for finding the libaec compression library has been modified for a system-installed version of the library. Two options
|
|
must be set:
|
|
|
|
```
|
|
HDF5_ALLOW_EXTERNAL_SUPPORT:STRING=NO
|
|
<LIB_PKG_NAME>_USE_EXTERNAL:BOOL=OFF
|
|
```
|
|
|
|
where `<LIB_PKG_NAME>` is one of `ZLIB`, `ZLIBNG`, `SZIP`, `PLUGIN`.
|
|
|
|
Note that `HDF5_ALLOW_EXTERNAL_SUPPORT:STRING=NO` disables building all plugins and external libraries in-line with the HDF5 library.
|
|
|
|
In addition, the `<LIB_PKG_NAME>_ROOT` environment variables must be set, where `<LIB_PKG_NAME>` is one of `ZLIB`, `ZLIBNG`, `SZIP`, `libaec`, `PLUGIN`. Note that libaec is the expected name for using the libaec library in place of original szip.
|
|
|
|
See INSTALL_CMake.txt for more detailed information.
|
|
|
|
### Changed the zlib/szip compression find message to FATAL ERROR
|
|
|
|
The message was changed to indicate that zlib/szip compression was requested and that it was not found. If an option is requested, not finding it should always be an error.
|
|
|
|
### Removed the module search `find_package` for szip library
|
|
|
|
There is not an szip module file to use, so the `find_package` only uses `find_package` in config mode. The choice then is to either build szip, with libaec, inline, or find a system installed szip library, built with CMake.
|
|
|
|
## Tools
|
|
|
|
### `h5repack` did not properly parse User Defined filters
|
|
|
|
The `h5repack` tool did not properly parse user-defined filter command-line arguments when the number of elements value was 0 (zero). Also, using a colon without a preceding object was enforced to behave the same as not using a colon.
|
|
|
|
Fixed GitHub issue [#5132](https://github.com/HDFGroup/hdf5/issues/5132)
|
|
|
|
### Changed the default value for number of cd_values in filters.
|
|
|
|
The tools used an arbitrary value 0f 20 for the number of cd_values used in a filter. Created a new define `DEFAULT_CDELEMTS` in H5tools.h for the default value, which currently matches the library restriction of 256.
|
|
|
|
Fixed GitHub issue [#5414](https://github.com/HDFGroup/hdf5/issues/5414)
|
|
|
|
## Performance
|
|
|
|
## Fortran API
|
|
|
|
Added missing parameters H5F_ACC_SWMR_READ_F and H5F_ACC_SWMR_WRITE_F
|
|
Fixed GitHub issue [#5959](https://github.com/HDFGroup/hdf5/issues/5959)
|
|
|
|
## High-Level Library
|
|
|
|
### Fixed an issue with H5TB functions
|
|
|
|
The H5TB functions were not correctly creating the FILL_INFO attribute for tables. This has been fixed by using the field offsets array from the call to H5TBAget_fill instead of using the compound type member offsets from the H5Tget_member_offset call for each compound type.
|
|
|
|
## Fortran High-Level APIs
|
|
|
|
## Documentation
|
|
|
|
## F90 APIs
|
|
|
|
## C++ APIs
|
|
|
|
## Testing
|
|
|
|
### Fixed an allocation in the t_bigio parallel test on 32-bit systems
|
|
|
|
A test in t_bigio.c attempts to allocate more than 4 GiB of memory, which will overflow (and wrap) the size_t type on 32-bit systems, creating a very small allocation instead of a very large allocation. The test then segfaults when it accesses memory outside of the small buffer.
|
|
|
|
The test has been fixed by limiting the buffer to 2 GiB on 32-bit systems.
|
|
|
|
Fixed GitHub [#2510](https://github.com/HDFGroup/hdf5/issues/2510)
|
|
|
|
### Added skipping of a few parallel tests for OpenMPI 5.0.5
|
|
|
|
An issue in OpenMPI 5.0.5 causes a few parallel HDF5 tests (mpiodup, props, fapl_preserve) to fail. These tests are now skipped for that release of OpenMPI. The issue has been fixed in the 5.0.6 release of OpenMPI.
|
|
|
|
# ✨ Support for new platforms and languages
|
|
|
|
# ☑️ Platforms Tested
|
|
|
|
A table of platforms tested can be seen on the [wiki](https://github.com/HDFGroup/hdf5/wiki/Platforms-Tested).
|
|
Current test results are available [here](https://my.cdash.org/index.php?project=HDF5).
|
|
|
|
# ⛔ Known Problems
|
|
|
|
- When performing implicit datatype conversion on specific non-IEEE floating-point format data, HDF5 may improperly convert some data values:
|
|
|
|
When performing I/O operations using a non-IEEE floating-point format datatype, HDF5 may improperly convert some data values due to incomplete handling of non-IEEE types. Such types include the following pre-defined datatypes:
|
|
|
|
H5T_FLOAT_F8E4M3
|
|
H5T_FLOAT_F8E5M2
|
|
|
|
If possible, an application should perform I/O with these datatypes using an in-memory type that matches the specific floating-point format and perform explicit data conversion outside of HDF5, if necessary. Otherwise, read/written values should be verified to be correct.
|
|
|
|
- When the library detects and builds in support for the _Float16 datatype, an issue has been observed on at least one MacOS 14 system where the library fails to initialize due to not being able to detect the byte order of the _Float16 type [#4310](https://github.com/HDFGroup/hdf5/issues/4310):
|
|
|
|
#5: H5Tinit_float.c line 308 in H5T__fix_order(): failed to detect byte order
|
|
major: Datatype
|
|
minor: Unable to initialize object
|
|
|
|
If this issue is encountered, support for the _Float16 type can be disabled with a configuration option:
|
|
|
|
`CMake: HDF5_ENABLE_NONSTANDARD_FEATURE_FLOAT16=OFF`
|
|
|
|
- When HDF5 is compiled with NVHPC versions 23.5 - 23.9 (additional versions may also be applicable) and with -O2 (or higher) and -DNDEBUG, test failures occur in the following tests:
|
|
|
|
- H5PLUGIN-filter_plugin
|
|
- H5TEST-flush2
|
|
- H5TEST-testhdf5-base
|
|
- MPI_TEST_t_filters_parallel
|
|
|
|
Sporadic failures (even with lower -O levels):
|
|
|
|
- Java JUnit-TestH5Pfapl
|
|
- Java JUnit-TestH5D
|
|
|
|
Also, NVHPC will fail to compile the test/tselect.c test file with a compiler error of 'use of undefined value' when the optimization level is -O2 or higher.
|
|
|
|
This is confirmed to be a [bug in the nvc compiler](https://forums.developer.nvidia.com/t/hdf5-no-longer-compiles-with-nv-23-9/269045) that has been fixed as of 23.11. If you are using an affected version of the NVidia compiler, the work-around is to set the optimization level to -O1.
|
|
|
|
- CMake files do not behave correctly with paths containing spaces
|
|
|
|
Do not use spaces in paths because the required escaping for handling spaces results in very complex and fragile build files.
|
|
|
|
- At present, metadata cache images may not be generated by parallel applications. Parallel applications can read files with metadata cache images, but since this is a collective operation, a deadlock is possible if one or more processes do not participate.
|
|
|
|
- The subsetting option in `ph5diff` currently will fail and should be avoided
|
|
|
|
The subsetting option works correctly in serial `h5diff`.
|
|
|
|
- Flang Fortran compilation will fail (last check version 17) due to not yet implemented: (1) derived type argument passed by value (H5VLff.F90), and (2) support for REAL with KIND = 2 in intrinsic SPACING used in testing.
|
|
|
|
- Fortran tests HDF5_1_8.F90 and HDF5_F03.F90 will fail with Cray compilers greater than version 16.0 due to a compiler bug. The latest version verified as failing was version 17.0.
|
|
|
|
- Several tests currently fail on certain platforms:
|
|
MPI_TEST-t_bigio fails with spectrum-mpi on ppc64le platforms.
|
|
|
|
MPI_TEST-t_subfiling_vfd and MPI_TEST_EXAMPLES-ph5_subfiling fail with
|
|
cray-mpich on theta and with XL compilers on ppc64le platforms.
|
|
|
|
- File space may not be released when overwriting or deleting certain nested variable length or reference types.
|
|
|
|
Known problems in previous releases can be found in the HISTORY*.txt files in the HDF5 source. Please report any new problems found to <a href="mailto:help@hdfgroup.org">help@hdfgroup.org</a>.
|