When using the '-s' (h5ls) or '-r' (h5dump) option to display 1-byte integer datasets and attributes as ASCII characters, a closing double-quote character for data values was dropped in some cases.
For objects with multiple hard links, use hash table to map between
object tokens and pathnames during traversal to avoid linear scan over
all previous objects for each hard link seen
Use separate hash table for h5trav "table" interface to map between
object tokens and an index into the table of visited objects. This
facilitates quick lookups of objects when adding hard link name aliases
for h5repack processing
A declared cd_nelmts with no values following it (e.g. UD=<filtn>,<flag>,1)
was never committed to filt->cd_nelmts: the trailing token (no comma after
it) was always treated as a value, never checked against the l/f/p pending
flags the way the comma-triggered branch does. h5repack's own validation
then silently passed since cd_nelmts stayed at its default 0, and the
mismatch was only ever caught by coincidence when the underlying filter
plugin did its own internal cd_values check (issue #6462).
Commit the trailing token to whichever UD field is still pending, mirroring
the comma-triggered branch exactly.
Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org>
* Fix stack buffer overflows in h5repack_parse.c (issue #6433)
Five sites in parse_filter() wrote to fixed 16-byte stack buffers
(scomp[16], stype[16]) via unbounded loop indices with no prior bounds
check, causing a stack-buffer-overflow on malformed -f arguments:
Site 1: scomp[k] - filter name token before '='
Site 2: stype[m] - SZIP pixels_per_block digit sequence
Site 3: stype[m] - SOFF scale_factor digit sequence
Site 4: stype[q] - UD legacy per-field digit sequence
Site 5: stype[m] - all-other-filters digit sequence
Replace all five bare array writes with a single PARSE_BUF_WRITE macro
that checks the index against sizeof(buf)-1 before writing, then frees
obj_list and exits with an error message on overflow — consistent with
all other error handling in parse_filter().
Confirmed with AddressSanitizer: overflow is eliminated and invalid
over-long arguments produce a clean diagnostic exit instead of UB.
* Fix stack buffer overflows in h5repack_parse.c (issue #6433)
All five overflow sites in parse_filter() are now protected via a
PARSE_BUF_WRITE macro that takes an explicit buf_sz argument (passed as
sizeof(buf) at every call site) to avoid silent pointer-decay if the
buffers are ever refactored to heap allocations.
In the SZIP and SOFF inner loops the previous code wrote bare
`smask[l] = c` with no bounds check, and relied on `i = len - 1` to
stop the *outer* loop without also breaking the *inner* `u` loop.
That left one additional iteration where `l` could equal 2 and
`smask[2]` would be written by the next outer iteration before the
outer loop condition was re-evaluated. Both sites now use
PARSE_BUF_WRITE and add an explicit `break` to exit the inner loop
as soon as the two-character mask is complete.
The NULL guard around free(obj_list_ptr) is removed: free(NULL) is a
safe no-op per C99 §7.20.3.2.
* Updated h5dump --xml
The location of the files HDF5-File.xsd and HDF5-File.dtd no longer exists, and
the files were added to the repo (PR #5490). Before the --xml is actually removed,
running the h5dump tests will fail. This PR updated h5dump and the expected output
to use the correct files' location and actually completed the ticket HELP-2668.
* Added missing expected files from the previous commit
* Fixed typo
* Made corrections per review feedbacks
* Committing clang-format changes
* Missing files from previous commit
* Use valid URL for xmlns
---------
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
After PR #6352 made h5repack default to H5F_LIBVER_V18 as the low bound,
the existing metadata block size option check broke because with V18+
the larger meta_block_size shrinks the output file rather than growing
it. Loop the check over every H5F_LIBVER_* low bound and assert the
size ordering appropriate to each regime.
feat: add optional digital signature verification for HDF5 filter plugins
Introduce an opt-in plugin signing and verification system that allows
HDF5 deployments to require cryptographically signed filter plugins before
loading them. Disabled by default (HDF5_REQUIRE_SIGNED_PLUGINS=OFF).
New tool: h5sign
- Signs plugin shared libraries by appending an RSA signature and a
14-byte footer (algo_id | sig_len | 8-byte magic | format_ver) to the
binary without modifying the original content.
- Supports SHA-512 (default), SHA-256, SHA-384, and their PSS variants
(-a/--algorithm flag).
- Detects already-signed plugins; --force strips the old signature and
re-signs.
- Security hardened: keeps the file descriptor open through hashing and
appending (no TOCTOU window), enforces a 2048-bit minimum RSA key size,
rolls back partial writes on failure, and rejects paths that are not
regular files.
Verification (H5PLsig.c)
- At plugin load time, reads the footer, validates the magic and format
version, then checks the RSA signature against all public keys found in
the KeyStore directory.
- File is hashed once; per-key verification operates on the pre-computed
digest (no redundant I/O for multi-key keystores).
- Plugins whose signature hash appears in revoked_signatures.txt are
rejected regardless of key validity.
- Runtime debug output via HDF5_DEBUG=pl.
KeyStore management
- Trusted public keys are PEM files in a directory specified by
HDF5_PLUGIN_KEYSTORE_DIR (build time) or HDF5_PLUGIN_KEYSTORE (env var).
- HDF5_LOCK_PLUGIN_KEYSTORE cmake option disables the env-var override for
security-hardened deployments.
Test infrastructure
- h5signverifytest: positive, negative, tamper, re-sign, and revocation
test cases.
- CTest fixture-based dependency graph (FIXTURES_SETUP/FIXTURES_REQUIRED)
replaces fragile DEPENDS chains so tests remain correct under -R filtering.
- Dedicated signed-plugins.yml CI workflow; full test suite scoped to
H5SIGN and H5PLUGIN-signature tests to avoid unrelated flaky failures.
- Cross-platform: Linux, macOS, and Windows (MSVC-compatible, BIO-based
OpenSSL I/O, HDsleep/HDsetenv portability wrappers).
Documentation: docs/PLUGIN_SIGNATURE_README.md covers usage, footer
format, revocation file format, FAQ, and troubleshooting.
Co-authored-by: Glenn Song <gsong@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Replace all strcpy calls copying into fixed-size path buffers with snprintf to prevent buffer overflows. Also simplify the path prefix check and the empty-string initialization.
Fixes 6 critical CodeQL cpp/unbounded-write alerts.
* Re-write zlib/szip CMake logic for clarity
* Update external libaec, zlib-ng and zlib builds to not use patching process
* Add FindZLIBNG module to locate zlib-ng on system
* Rework HDF5 filter plugins support
Optimize Doxygen build process by skipping rebuilds when input files are unchanged, using timestamp checks in CMake configuration.
Behavior:
Skip Doxygen build if input files unchanged using timestamp check in CMakeLists.txt, hl/src/CMakeLists.txt, and src/CMakeLists.txt.
Introduces stamp files (hdf5lib_doc.stamp, hdf5hllib_doc.stamp) to track changes.
Documentation:
add_custom_command and add_custom_target used to manage Doxygen builds.
Shared Doxygen content inputs defined in top-level CMakeLists.txt for reuse.
Misc:
Reorder add_subdirectory(src) in CMakeLists.txt for visibility of Doxygen variables.
Intel's icx compiler defines _WIN32 (with underscore) but not WIN32,
causing unistd.h and other POSIX headers to be incorrectly included
on Windows builds.
Build-tree exports can't diverge from install-tree exports — the export(EXPORT ...) reads directly from the install export sets. No manual list to keep in sync.
Removed 3 global variables (HDF5_STATIC_LIBRARIES_TO_EXPORT, HDF5_JAVA_LIBRARIES_TO_EXPORT, HDF5_UTILS_TO_EXPORT) and their ~21 set_global_variable calls across tool/utility files.
Fixed the static-only build bug in the PR where the base export set was guarded by BUILD_SHARED_LIBS, breaking tools export.
Removed redundant utils in export files — the PR was dumping tools into all three build-tree export files (java, static, shared). Now they correctly appear only in the base export.
Refactor the cross-compilation infrastructure to rely on the standard
CMAKE_CROSSCOMPILING_EMULATOR variable instead of the custom
CROSSCOMPILING_PATH environment hack.
Key changes:
- Removed `ENVIRONMENT "${CROSSCOMPILING_PATH}"` from all test definitions.
- Updated custom commands (e.g., H5match_types, H5_gen) to invoke the
emulator explicitly when cross-compiling.
- Updated ConfigureChecks.cmake and HDF5UseFortran.cmake to enable
`try_run` checks when an emulator is defined.
- Added config/toolchain/mingw-w64-x86-64-wine.sh wrapper to automatically
locate MinGW runtime libraries (like libgfortran) for Wine execution.
- Cleaned up toolchain files to remove hardcoded emulator paths, deferring
configuration to the build environment.
Adds predefined datatypes for FP6 data in E2M3 and E3M2 formats
Does not add support for any native FP6 types; datatype conversions are performed in software
Adds predefined datatypes for FP8 data in E4M3 and E5M2 formats
Does not add support for any native FP8 types; datatype conversions are performed in software
Change default file format to 1.8 across various tests and examples, updating file creation and access logic accordingly.
Behavior:
Default file format version changed to 1.8 in H5Pfapl.c.
Updated file creation and access to use 1.8 format in h5ex_g_compact.c and test_file_image.c.
Set earliest file format in multiple test files including cache_tagging.c, dtypes.c, and links.c.
Tests:
Modified expected output in tools/test/misc/expected/*.ls files to reflect new file format locations.
Adjusted test logic in test_file_image.c and cache_tagging.c to accommodate format changes.
Misc:
Added comments and TODOs for future format testing in test_file_image.c.
Minor variable renaming for clarity in test_file_image.c.
Ensure proper closure of resources in gent_array1_big, gent_nested_compound_dt, and gent_intscalars in h5dumpgentest.c.
Add error handling and cleanup logic to prevent resource leaks.
Behavior:
Modify HDF5UseFortran.cmake to handle cross-compilation by providing default KINDs and sizes when CMAKE_CROSSCOMPILING is true.
Default values for INTEGER and REAL KINDs and their sizes are set when cross-compiling.
Error messages are adjusted to reflect cross-compilation scenarios.
Logic:
Conditional checks added for CMAKE_CROSSCOMPILING to determine if default values should be used.
Default values include INTEGER KINDs {1,2,4,8,16} and REAL KINDs {4,8,10,16}.
Default sizes for native kinds are set to 4 for INTEGER and REAL, and 8 for DOUBLE PRECISION when cross-compiling.
Misc:
Adjustments to verbose messages to indicate when defaults are used during cross-compilation.
Add VOL tests for h5dump, update test generation logic, and modify test configurations.
Tests:
Add VOL tests for h5dump in CMakeTests.cmake and CMakeTestsXML.cmake.
Modify ADD_H5_TEST macro to support VOL-specific tests and filtering.
Add gent_tvms() function in h5dumpgentest.c and h5dumpgentest.h for generating test files.
A couple of places in the h5import.c code use fscanf with %s to read
strings into a fixed-size buffer without restricting the number
of characters, which could lead to a stack buffer overflow.
This fix restricts the number of characters that can be read to
the size of the buffer.
Adds predefined datatypes for little- and big-endian bfloat16 data
Does not add support for any native bfloat16 types; datatype conversions are performed in software
Also adds missing float16 predefined types to fortran
The trefer_extR test depends upon this pair of files, but always uses the pre-generated ones checked into the repo. This adds a routine to generate them, which will soon be used by the h5dump VOL tests.
Refactor ADD_H5_TEST macro in CMakeTests.cmake to consolidate test definitions and improve consistency.
Macros:
Consolidated ADD_H5_TEST and ADD_H5_ERR_TEST into a single ADD_H5_TEST macro in CMakeTests.cmake.
ADD_H5_TEST now accepts RESULT_CODE and optional RESULT_ERRCHECK as keyword arguments.
Test Definitions:
Updated all test invocations to use the new ADD_H5_TEST macro format with RESULT_CODE and RESULT_ERRCHECK where applicable.
Removed redundant ADD_H5_ERR_TEST macro usage.
Behavior:
Ensures consistent test setup and error handling across all h5ls tests.
Improve test file cleanup in h5repacktst.c by using h5_delete_test_file and avoiding deletion of default-driver exclusive files with non-default drivers.
Behavior:
Use h5_delete_test_file in h5repacktst.c for file deletions, ensuring compatibility with Family VFD filename requirements.
Avoid deleting default-driver exclusive files with non-default drivers in h5repacktst.c.
File Management:
Reorganize file arrays in h5repackgentest.h to separate default-driver files from others.
tbeu
jhendersonHDF
scivisionandscivision
Scot BreitenfeldandH. Joe Lee
Aleksandar Jelenak
Julien Schueller
Matt L
efferre79
Neil Fortner
Allen Byrne
Dana Robinson