feat: add optional digital signature verification for HDF5 filter plugins
Introduce an opt-in plugin signing and verification system that allows
HDF5 deployments to require cryptographically signed filter plugins before
loading them. Disabled by default (HDF5_REQUIRE_SIGNED_PLUGINS=OFF).
New tool: h5sign
- Signs plugin shared libraries by appending an RSA signature and a
14-byte footer (algo_id | sig_len | 8-byte magic | format_ver) to the
binary without modifying the original content.
- Supports SHA-512 (default), SHA-256, SHA-384, and their PSS variants
(-a/--algorithm flag).
- Detects already-signed plugins; --force strips the old signature and
re-signs.
- Security hardened: keeps the file descriptor open through hashing and
appending (no TOCTOU window), enforces a 2048-bit minimum RSA key size,
rolls back partial writes on failure, and rejects paths that are not
regular files.
Verification (H5PLsig.c)
- At plugin load time, reads the footer, validates the magic and format
version, then checks the RSA signature against all public keys found in
the KeyStore directory.
- File is hashed once; per-key verification operates on the pre-computed
digest (no redundant I/O for multi-key keystores).
- Plugins whose signature hash appears in revoked_signatures.txt are
rejected regardless of key validity.
- Runtime debug output via HDF5_DEBUG=pl.
KeyStore management
- Trusted public keys are PEM files in a directory specified by
HDF5_PLUGIN_KEYSTORE_DIR (build time) or HDF5_PLUGIN_KEYSTORE (env var).
- HDF5_LOCK_PLUGIN_KEYSTORE cmake option disables the env-var override for
security-hardened deployments.
Test infrastructure
- h5signverifytest: positive, negative, tamper, re-sign, and revocation
test cases.
- CTest fixture-based dependency graph (FIXTURES_SETUP/FIXTURES_REQUIRED)
replaces fragile DEPENDS chains so tests remain correct under -R filtering.
- Dedicated signed-plugins.yml CI workflow; full test suite scoped to
H5SIGN and H5PLUGIN-signature tests to avoid unrelated flaky failures.
- Cross-platform: Linux, macOS, and Windows (MSVC-compatible, BIO-based
OpenSSL I/O, HDsleep/HDsetenv portability wrappers).
Documentation: docs/PLUGIN_SIGNATURE_README.md covers usage, footer
format, revocation file format, FAQ, and troubleshooting.
Co-authored-by: Glenn Song <gsong@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Consolidate documentation under doc/ directory
Move user-facing guides from release_docs/ and doxygen/ into a single
doc/ root. release_docs/ now holds only release artifacts (changelogs,
history, release process, maintainer info).
- git mv release_docs/INSTALL*.md, USING_*.md, README_HPC.md,
BuildSystemNotes.md, AutotoolsToCMakeOptions.md,
HDF5_Library_2.0.0_Migration_Guide.md → doc/
- git mv doxygen/ → doc/doxygen/
- Update CMakeLists.txt: HDF5_DOXYGEN_DIR and add_subdirectory path
- Update CMakeInstallation.cmake: all install paths for moved files
- Update bin/make_vers: hardcoded doxygen/ path substitution
- Update doc/doxygen/CMakeLists.txt: EXAMPLES_DIRECTORY and comments
- Update README.md, CONTRIBUTING.md, SECURITY.md, config/README.md,
release_docs/RELEASE_PROCESS.md: links to moved files
- Update doxygen .dox files: release_docs/ URLs for moved guides
- Rewrite release_docs/README.md for narrowed scope
* Add HDF5_DOCS_DIR variable for doc/ root path
Introduce HDF5_DOCS_DIR = \${HDF5_SOURCE_DIR}/doc so that
CMakeInstallation.cmake and future callers reference the doc/
directory symbolically rather than by hardcoded path.
HDF5_DOXYGEN_DIR is now derived from HDF5_DOCS_DIR.
* Generate "Extra libraries" in `libhdf5.settings` using a new variable `LINK_LIBS_SETTINGS` rather than `LINK_LIBS`
* Follow up to #6220
* Try to deal with Windows MSVC
* Re-write zlib/szip CMake logic for clarity
* Update external libaec, zlib-ng and zlib builds to not use patching process
* Add FindZLIBNG module to locate zlib-ng on system
* Rework HDF5 filter plugins support
Optimize Doxygen build process by skipping rebuilds when input files are unchanged, using timestamp checks in CMake configuration.
Behavior:
Skip Doxygen build if input files unchanged using timestamp check in CMakeLists.txt, hl/src/CMakeLists.txt, and src/CMakeLists.txt.
Introduces stamp files (hdf5lib_doc.stamp, hdf5hllib_doc.stamp) to track changes.
Documentation:
add_custom_command and add_custom_target used to manage Doxygen builds.
Shared Doxygen content inputs defined in top-level CMakeLists.txt for reuse.
Misc:
Reorder add_subdirectory(src) in CMakeLists.txt for visibility of Doxygen variables.
Addresses the "multiple @ param documentation sections" warnings are caused by the interaction of three things:
1. MACRO_EXPANSION = YES with EXPAND_ONLY_PREDEF = NO in Doxyfile.in
2. Versioned API macros in H5version.h like:
\def documentation in APIVersions.dox like:
3. \def H5Tarray_create \api_vers_2{H5Tarray_create,H5Tarray_create1,H5Tarray_create2}
Because Doxygen expands all macros, it sees H5Tarray_create as an alias for H5Tarray_create1. It then associates documentation from both the \def block (for the macro) and the function's own doc block (with \param entries) with the same function — resulting in "multiple @ param documentation sections."
* Update Doxygen version from 1.16.0 to 1.16.1
Use Doxygen 1.16.1 for all platforms. This version includes the latest
bug fixes and improvements while maintaining compatibility with our
documentation format.
* Fix Windows Doxygen installation for version 1.16.1
Handle different Windows binary naming conventions across Doxygen versions:
- 1.16.1 uses: doxygen-VERSION.windows.x64.bin.zip
- 1.16.0 uses: doxygen-VERSION.x64.bin.zip
Try the newer naming convention first, then fall back to the older one
if the download fails. This ensures compatibility across versions.
* Use ssciwr/doxygen-install in setup-doxygen action
Replace direct installation of official Doxygen binaries with
ssciwr/doxygen-install action. This avoids Java documentation parsing
errors that occur with official binaries but not with distribution
builds.
The setup-doxygen action now serves as a thin wrapper around
ssciwr/doxygen-install@f13be16c83 (v1),
using commit hash for security and reproducibility.
* Remove redundant version specification from workflows
The setup-doxygen action already has a default version (1.16.1),
so explicit version specification in workflows is unnecessary.
This follows DRY principles and makes version updates simpler.
Update version to 2.1 and derive version information from H5public.h, removing h5vers script and updating CMake and Java configurations.
Versioning:
Update version to 2.1 in H5public.h.
Derive version strings in H5public.h using macros.
CMake:
Extract version from H5public.h in HDF5config.cmake and HDF5AsSubdirMacros.cmake.
Configure README.md and CHANGELOG.md using CMakeLists.txt.
Java:
Generate H5Version.java from H5public.h for version consistency.
Update H5.java to use H5Version for version constants.
Removals:
Delete bin/h5vers script, previously used for version management.
FFM build requires Java 25, Jextract 25.
Generates FFM bindings during configure.
JNI is default when the requirements are not met or can be forced.
Presets added for maven and FFM - JNI is default selection.
Enhanced Maven options will work with either JNI or FFM
New Workflows for testing and maven uploads.
Extensive documentation changes for java.
Behavior:
Modify HDF5UseFortran.cmake to handle cross-compilation by providing default KINDs and sizes when CMAKE_CROSSCOMPILING is true.
Default values for INTEGER and REAL KINDs and their sizes are set when cross-compiling.
Error messages are adjusted to reflect cross-compilation scenarios.
Logic:
Conditional checks added for CMAKE_CROSSCOMPILING to determine if default values should be used.
Default values include INTEGER KINDs {1,2,4,8,16} and REAL KINDs {4,8,10,16}.
Default sizes for native kinds are set to 4 for INTEGER and REAL, and 8 for DOUBLE PRECISION when cross-compiling.
Misc:
Adjustments to verbose messages to indicate when defaults are used during cross-compilation.
* h5cc: Allow overriding the compilers written into the file
Add a set of `HDF5_H5CC_C_COMPILER`, `HDF5_H5CC_CXX_COMPILER`
and `HDF5_H5CC_Fortran_COMPILER` CMake variables that can be used to
override the compiler string written into `h5cc`, `h5c++`, etc. This is
particularly useful when e.g. using ccache during the build,
as the ccache path otherwise ends up in `h5cc`.
Added infrastructure support for multithreaded concurrency by adding an optional way to switch to using a non-recursive R/W lock for the global API lock. This is enabled with a new 'concurrency' configuration flag for the autotools & CMake builds, which is disabled by default.
When the 'concurrency' build option is chosen, the global API lock will use the R/W lock and all API calls currently will acquire a write lock, ensuring exclusive access by one thread. Over time, the API routines that are converted to support multithreaded concurrency will switch to acquiring a read lock instead.
Reentering the library from application callbacks is managed by the 'disable locking for this thread' (DLFTT) threadsafety protocol. This is internally handled within the H5_API_LOCK / H5_API_UNLOCK macros in H5private.h (as before), which invoke the 'dlftt' routines in H5TSint.c.
To support this change, the threadsafety configuration macros for the library have been updated:
- --enable-threadsafe now defines the H5_HAVE_THREADSAFE macro
- --enable-concurrency defines the H5_HAVE_CONCURRENCY macro
The new H5_HAVE_THREADSAFE_API macro is set if either H5_HAVE_THREADSAFE or H5_HAVE_CONCURRENCY is enabled.
New Github actions are added to include the concurrency configuration in the CI for the develop branch.
To support the new non-recursive R/W locking for API routines, some other changes are necessary:
Added macro wrappers around all callback invocations that could call an
application function, and therefore re-enter the library:
H5_BEFORE_USER_CB* / H5_AFTER_USER_CB*
Added H5_user_cb_prepare / H5_user_cb_restore routines that save the
state of the library when callback leaves the library. Includes error
stack and threadsafe reentry state currently.
There's also some small cleanups to various places in the library:
Moved the H5E_mpi_error_str / H5E_mpi_error_str_len globals to be local for
pushing MPI errors, so that multiple threads can't interfere with each
other.
Added H5TS_rwlock_trywrlock() routine to R/W lock interface.
Emulate R/W locks on MacOS because its implementation of
pthread_rwlock_wrlock() does not conform to the POSIX standard.
Don't acquire the global API lock in H5close, since it's acquired in H5_term_library, which is necessary because H5_term_library is invoked via other code paths that don't hold the global API lock.
Don't call H5Eget_auto2 API routine within H5_term_library.
Switched 'return NULL' in H5allocate_memory to HGOTO_DONE(NULL).
Switched H5Pget_file_space_strategy / H5Pset_file_space_strategy to use
internal routines instead of API routines.
Switched H5Oopen_by_addr & H5Ovisit1 to use internal routines instead of
API routines.
Fixed a few places in src/H5Odeprec.c where a major
error ID was passed as a minor ID.
* Split CMake HDFCompileFlags into specific compiler files
* Separate out CXX Flags
* Add Fortran compiler specific files
* Merge in #4816 changes and close#4816
* fix hanging endif
---------
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Added new testframe.h header to document testing framework functions and
split them away from h5test.h and from test programs that don't
integrate with the testframe.c testing framework
Added new test setup callback to testframe.c testing framework
Added parameters to AddTest() to specify size of test parameters so they
can be copied for later use
Enabled HDF5 error stacks in testframe.c framework by default and added
some error stack suppressions to some testhdf5 tests
Added new maxthreads option to testframe.c framework to allow specifying
the maximum number of threads a multi-threaded test can use
Moved TestExpress functionality out of testframe.c and into more general
h5test.c for wider use by tests through getter and setter
Updated some tests to not mix and match functionality between h5test.c/h
and testframe.c/h
Moved some functionality from testphdf5.h into testpar.h for parallel
tests that aren't part of testphdf5
Added new parallel test library that contains common shared
functionality for parallel tests (similar to h5test library)
Complete overhaul of the concurrency-related aspects of the library (threading, atomics, locking, etc.), adding private routines in the H5TS package to allow internal algorithms to use all of these capabilities.
Adds many new features & components in the H5TS package that are equivalent to common concurrency data structures and capabilities: "regular" and recursive mutices, condition variables, semaphores, thread barriers, 'once' support, thread pools, atomic variables, thread-local keys, and spawning & joining internal threads.
Now supports C11, pthreads, and Windows threading for all H5TS capabilities, except the recursive readers/writers lock, which is not supported on Windows (because Windows threads don't provide a callback on thread-local variable deletion).
The "global" API lock is switched to use a recursive mutex from the H5TS package, instead of its own variant.
API context code (H5CX package) and error stacks (H5E package) now use the common thread-local info, instead of their own variants.
Subfiling code is switched from using Mercury threading features to the new internal H5TS features.
Removes the mercury threading code.
Adds a configure option (--enable-threads / HDF5_ENABLE_THREADS), enabled by default, to control whether threading is enabled within the library.
* Add a comment on the FindZLIB.cmake module usage
* Allow choice of static/shared compression libs for Find Module
* Added new option to INSTALL_CMake file and changed option text
* Allow HDF5_LIB_INFIX to work with DLL
* Separate individual library name into parts and add suffix option
* Java cannot use alternative names and removed extra setting
* Incorporate the underscore into the CORE name
Matt L
Orion Poplawski
jhendersonHDF
SY Wang
Alex Dutka
Allen Byrne
Dana Robinson
Michał Górny
Quincey Koziol
Allen Byrne