Addresses 11 findings from a 12-agent review of the string-based filter
configuration feature:
- H5Opline.c: fail H5Dcreate/H5Ocopy with H5E_BADRANGE when a filter's
config string cannot be persisted because the file's high libver bound
doesn't support pipeline v3, instead of silently dropping the string
with a success return.
- H5Pocpl.c: H5Pget_filter_params_by_idx() now raises H5E_OVERFLOW on
truncation in all three source branches, matching its own sibling
H5Zconfig_get_str(); also stops discarding a genuine H5Z_filter_avail()
failure.
- H5Zdevelop.h / H5Z.c: fix two independently-wrong comments about
H5Z_class3_t's field layout, one of which gave plugin authors unsafe
migration advice for the filter callback's incompatible type change.
- h5zImp.c: fix an out-of-bounds heap write in the JNI H5Zconfig_get_param
long/double/boolean overloads when called with a zero-length array.
- h5tools_dump.c: h5dump -p no longer silently omits an entire filter's
FILTERS{} entry when a decoration buffer fails to allocate.
- tomlc17: add h5_toml_prefix.h, an HDF5-authored (not vendored) header
that force-includes symbol-renaming macros ahead of tomlc17.c, so a
statically-linked libhdf5.a cannot collide with an application's own
copy of tomlc17. Does not touch the pristine vendored files.
- H5Zconfig.c: H5Z_CONFIG_MAX_PARAMS is now actually enforced.
- Six built-in filters: use H5Z_CLASS3_T_VERS instead of a hardcoded
literal for the class version field.
test/tfilter2.c updated to match the new libver-bound failure contract.
Verified via a full rebuild: tfilter2, t_filters_parallel, h5repack
plugin_test_cfg*, and h5dump filter tests all pass.
Deferred by explicit decision (documented in
COMPREHENSIVE_REVIEW_filter_config_string.md): bumping H5P_ENCODE_VERS
and hardening the H5Pdecode() buffer-length gap, both of which need
architecture beyond this fix pass; JNI test coverage for the new API;
and the remaining ~13 Medium/Low findings.
* Close datatype IDs derived from the memory type in the JNI translate helpers
The object-tree read/write helpers in h5util.c derive a base datatype from
the memory type with H5Tget_super() for the variable-length, array and
complex classes, but never closed it. Because an hid_t is not reclaimed when
the native method returns, every read or write of such data leaked at least
one datatype ID for the lifetime of the process, and nested types leaked
one per level.
This PR updates the helpers to close the derived type in their done: blocks,
which covers both the success and the error paths, and to reset the id in the
compound loops to avoid the potential for double closes.
It also has the helpers release the class references that the per-element
helpers look up on entry. These are local references, so they were reclaimed
when the enclosing native method returned, but a compound read calls the helper
once per member per element and held one set per call until then. Releasing
them at the single exit bounds the count of references to the recursion depth.
* Add CHANGELOG entry for the JNI datatype ID leak fix
* Restrict the derived datatype close guards to strictly positive IDs
hid_t 0 is not a valid datatype ID, so H5Tclose(0) would fail.
* Revert "Restrict the derived datatype close guards to strictly positive IDs"
An hid_t of 0 not being a valid ID is a property of the current H5I
encoding rather than a documented guarantee, so the JNI helpers should
not depend on it.
Verified against the fork's actual GitHub Actions CI (not just local
builds) and found two real regressions plus applied formatting the CI
bot detected but couldn't push:
- test/CMakeTests.cmake: register test_filters_v3.h5 (the Critical #3
golden file) in HDF5_REFERENCE_TEST_FILES so a fresh checkout's build
tree actually stages it -- it only passed locally because the file had
been generated directly inside an existing local build directory. Was
failing H5TEST-tfilter2 on every CI job running the standard suite.
- tools/lib/CMakeLists.txt: link the tools library against LINK_LIBS
(libm), matching src/CMakeLists.txt's existing pattern. The new
hex-float code in h5tools_dump.c is the first tools-library code to
need floor()/isfinite(), and FreeBSD/OpenBSD's stricter linkers reject
the resulting undefined symbols (Linux tolerated it silently). Was
breaking the link step on FreeBSD 14.3/15.0 and OpenBSD 7.9.
- Applied the clang-format fixes the CI bot found but couldn't push back
(pre-existing fork credentials gap, confirmed unrelated to this
branch): tfilter.cpp, h5pDCPLImp.c, H5Znbit.c, tfilter2.c,
h5tools_dump.c.
Verified via ctest -R H5TEST-tfilter2 on a build tree with the stale
golden-file copy removed (reproducing CI's exact test invocation), plus
a full local rebuild (0 warnings/errors) and the Fortran/parallel suites.
Details and CI failure logs analyzed in REVIEW-6153.md's new
"Post-push CI verification" section.
Fixes all 3 Critical, 10 High, 12/13 Medium, and all 7 Low findings from
the branch's comprehensive review (full detail in REVIEW-6153.md):
- Critical: canonicalization-length stack over-read, false case-sensitivity
doc claims (RFC amended to match), missing pipeline-v3 format-spec entry
and golden regression file.
- High: unbounded H5Pdecode() config length, H5Zregister() doc/validator
mismatch, CHANGELOG errors, dead Fortran test code, JNI truncation bug,
Java struct visibility, incomplete H5Pget_filter_params_by_idx docs.
(H5P_ENCODE_VERS bump deliberately deferred as a separate, larger change.)
- Medium: h5dump control-char injection, H5Z_find() type-confusion doc,
missing C++ modifyFilterByIdx, doc wording fixes, TOML 0b/0o mis-lex,
scaleoffset upper-bound check, missing error check, parallel stored-string
coverage, C++ test gaps. (Java test porting deliberately skipped, no JDK
available to verify.)
- Low: dead H5Z_CLASS_T_VERS_MAX constant, duplicate nbit set_config,
false JNI Javadoc, canonical-name cross-plugin collision check, stale
doc anchors, compiler-ID visibility gate, CHANGELOG omissions.
Also tightens newly-added comments across the branch (src, tools, tests,
java, c++/fortran) to be concise and WHY-focused, and fixes two compiler
warnings found during final verification: an intentional exact
floating-point comparison in h5dump's hex-float annotation, and two
oversized stack buffers in the same file's filter-listing loop (now
heap-allocated).
All changes verified via full rebuild (0 warnings/errors) plus tfilter2,
testhdf5_fortran, parallel filter tests under mpirun, and the relevant
h5repack/h5dump ctests -- all passing with 0 errors.
Two CI failures, both from the preceding commits.
Missing prototypes (hdf5 dev cmake CI). H5Pmodify_filter_by_idx_str_c
and _raw_c were defined in fortran/src/H5Zf.c but never declared in
H5f90proto.h alongside the H5Pappend_filter_*_c helpers. That broke two
different jobs for one reason:
- "Special Workflows / gcc REL -Werror (build only)" builds with
-Werror=missing-prototypes and rejected both definitions outright.
- The Intel and windows-oneapi jobs failed at link with LNK2019 on
both symbols, because H5FC_DLL on the declaration is what carries
the dllexport attribute; with no declaration the symbols were never
exported from hdf5_fortran.dll.
Adding the two H5FC_DLL declarations fixes both. Verified locally by
compiling H5Zf.c with -Werror=missing-prototypes, which the ordinary
build does not enable.
Formatting (clang-format Commit Changes). This job was green on the
commit before this series and has failed on every commit since; the
fault was entirely mine. Ran clang-format 17 -- the version CI pins --
over every file this series touched, C and Java alike, since the check
covers .java here too. The whole tree now passes a local run using CI's
source path and exclusion list.
Full build clean; ctest 3458/3459, the one failure being the
pre-existing ph5_f90_filtered_writes_no_sel example bug fixed upstream
in HDFGroup/hdf5#6633.
Implements the two overloads specified in RFC-HDFG-2026-001
sec:modify-bindings -- one taking a String, one an int[] -- mirroring
the existing H5Pappend_filter overloads.
Both Java binding paths are covered, since the tree maintains two:
- JNI (java/src-jni): two private native methods dispatched from the
public overloads, plus the JNI implementations and declarations in
h5pDCPLImp.c/.h, following the H5Pappend_filter_str/_raw pattern
exactly.
- Panama/FFM (java/hdf): the H5Z_params_t struct is built directly in
an Arena and passed to the generated org.hdfgroup.javahdf5.hdf5_h
downcall, as the append string form already does. Unlike the append
case the cd_values form cannot delegate to H5Pmodify_filter, which
addresses an entry by filter ID and resolves to the first match
rather than by index, so it constructs the struct explicitly. The
generated binding is produced by jextract from the public headers,
so it picks up the new declaration automatically.
Adds testH5Pmodify_filter_by_idx_string and
testH5Pmodify_filter_by_idx_cdvalues to TestH5Z.java and updates the
JUnit golden file. Placement and the test count were derived from
@FixMethodOrder(MethodSorters.NAME_ASCENDING) rather than guessed, and
the file's surrounding format is unchanged.
Not compile-verified locally: this configuration has HDF5_BUILD_JAVA
OFF and the machine has a JRE but no JDK (no javac) and no jextract, so
nothing under java/ can be built here. The C library still builds clean
and the full C/Fortran suite is unchanged. What was checked statically:
the JNI symbol names against the mangling rule (underscores in the Java
method name become _1) and against the existing append pair; the
H5Z_params_t offsets (int type at 0, union at 8, cd_nelmts at 8,
cd_values at 16, 24 bytes total on 64-bit) and the enum values
(H5Z_PARAMS_CDVALUES 0, H5Z_PARAMS_STRING 1) against H5Zpublic.h; and
the H5Pget_filter call in the new test against the working call in
testH5Pappend_filter_cdvalues. Java CI should be treated as the real
gate.
The base type lookups in the object-tree helpers tested the
returned hid_t for truth rather than for a negative value. A failed
lookup returns H5I_INVALID_HID (-1), resolving true when checked, so any
failure wouldn't be caught until later.
* Fix NULL pointer access when H5A_operator2_t is NULL
Passing NULL for the callback function pointer to H5Aiterate2 and
H5Aiterate_by_name was not detected, leading to a subsequent access
of an uninitialized pointer. Add a check for this "no operator
specified" case in both functions so they fail gracefully instead.
Fixes GHSA-r7g4-hv2f-5c66 - CVE-2025-9274
* Fix format
* Fix Java test to handle NULL callback to H5Aiterate2
* Fix JNI datatype ID leak in h5str_detect_vlen_str()
The JNI H5Dread/H5Dwrite/H5Aread/H5Awrite wrappers call h5str_detect_vlen()
on the memory type. For an H5T_ARRAY/H5T_VLEN of a fixed (non-vlen-string)
base type, h5str_detect_vlen_str() acquired the base type via H5Tget_super()
but only closed it when the recursive check returned 1 or a negative error.
When the recursive call returned 0 because no vlen string was found, the base type ID
was leaked.
This PR changes h5str_detect_vlen_str() to close the id unconditionally after the recursive check,
in the same style as the compound-member case in the same function.
A JNI regression test exists at TestH5D.testH5DArray_super_no_id_leak, which reads
an H5T_ARRAY-of-int dataset in a loop and asserts via H5Fget_obj_count() that
no datatype IDs leak.
* Assert non-negative H5Fget_obj_count in array datatype ID leak test
Guard the before/after open-datatype counts against a negative
(failed) H5Fget_obj_count return, which would otherwise let the
equality check pass spuriously. Keep the count scoped to
H5F_OBJ_ALL: the leaked IDs are transient datatypes not attached to
any file, so a per-file count would not see them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Modify CHANGELOG entry
* Modify CHANGELOG again
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org>
JUnit4's default method sorter (MethodSorters.DEFAULT) orders test
methods by a hash of the method name, not declaration order -- so any
rename or addition shifts the whole run order unpredictably relative
to a hand-maintained golden file. Both java/test/TestH5Z.java (added
this session, 13 methods) and java/src-jni/test/TestH5Z.java (5
methods, unmodified but apparently never correctly regenerated after
the testH5Zget_filter_info2 -> testH5Zget_filter_class_info rename)
had golden JUnit-TestH5Z.txt files that didn't match the real
hash-sorted order, both confirmed by computing Java's documented
String.hashCode() algorithm directly and matching it byte-for-byte
against actual CI failure output.
Verified against upstream develop (an unrelated, currently-open
develop-based PR): this test passes cleanly there, confirming the
mismatch was never a general JUnit/reflection flakiness -- it was
always a genuinely wrong reference file.
Fixed by adding @FixMethodOrder(MethodSorters.NAME_ASCENDING) to both
classes (alphabetical by method name, a properly documented and stable
JUnit4 ordering mode) and regenerating both reference files to match
the resulting deterministic order. This also makes the test immune to
the same class of breakage going forward: alphabetical order is
computable by inspection, unlike the hash order.
RFC-HDFG-2026-001 v5 reverses the earlier "no on-disk format change"
decision: the verbatim key=value string passed to H5Pappend_filter is
now retained on the pipeline entry and written to the file, so it can be
recovered losslessly without loading the filter plugin.
H5O_PLINE_VERSION_3 appends [config_length:2][config] after each filter's
name. H5O_pline_set_version selects v3 only when a filter carries a
stored string and the file's high bound admits it; otherwise it stays at
v2 and the string is omitted, keeping string-free files byte-identical.
The new bound H5F_LIBVER_V300 (now H5F_LIBVER_LATEST) gates emission and
is mirrored in every *_ver_bounds[] table, H5trace, h5test, and the
Fortran/Java constants.
The string is threaded through the property machinery: H5Pappend_filter
retains it, H5Z_append/decode initialize it, H5O__pline copy/size/reset
and the H5Pencode/H5Pdecode pipeline enc/dec carry it, H5Z_modify
(H5Pmodify_filter) clears it, and H5Z_delete frees it.
H5Pget_filter_params_by_idx now returns the stored string first, falling
back to get_config then a cd_values listing; h5dump -p shows it for every
filter. H5Z_get_config_func_t's doc no longer frames cd_values packing
as normative (it is filter-private).
Tests (test/tfilter2.c) cover the verbatim round-trip recovered without
the plugin, the silent v2 downgrade when the libver bound is too low,
H5Pmodify_filter clearing, and survival through H5Pencode/H5Pdecode and
H5Ocopy. Updated the h5repack plugin_test_cfg reference to the verbatim
PARAMS_STRING.
write_blob/read_blob/close_blob and the corresponding H5Z_PARAMS_BLOB
enum value and has_blob_callbacks info field were speculative
reservations for a not-yet-designed feature. Pre-reserving generic
void* slots doesn't avoid the ABI/version bump the feature will need
once its callback signatures are actually decided, so there's no
benefit to carrying them now. Drop them from the public struct, the
internal filter table entry, all built-in filter registrations, the
Java bindings (JNI and Panama/FFM), and the test plugin/test suite.
Comments should describe current behavior, not cite external documents
by section number that will drift as the RFC is revised -- and should
stick to ASCII. Sweeps beyond this session's own edits since these
patterns were already present in this feature's earlier commits: em-dashes,
arrows, curly quotes, ellipses, and multiplication signs replaced with
ASCII equivalents; "RFC-HDFG-2026-001 SecN" citations replaced with
plain descriptions of the behavior itself.
The trailing "2" follows HDF5's convention for versioned/deprecated
symbols (H5Dopen2, H5Gcreate2, etc.), implying this is a successor to
H5Zget_filter_info registered in H5vers.txt -- it isn't. The two functions
are independent: H5Zget_filter_info returns only encode/decode config
flags, H5Zget_filter_class_info additionally exposes name, description,
and set_config/get_config presence. Renamed across C, Fortran, and both
Java binding variants (JNI + FFM) since it hadn't shipped yet.
* Fix H5DreadVL failing for pre-allocate cmpd-of-seq dsets
* Fix bad vlen of cmpd with null slot read
* Fix bad cmpd of cmpd read in java
`translate_rbuf`'s H5T_VLEN case had a similar bug where when `found_jList` was set to false due to an entyr in `ret_buf` being null, `ret_buf.add()` would be invoked on an array of objects without the list .add() method. This would occur whenever a read was invoked of a vlen sequence with a null (non-preallocated) entry. The pre-existing tests only tested the pre-allocated cases.
I removed the use of the `found_jList` flag, since it conflated the passing of an unallocated slot with `ret_buf` not being an array. Instead use `ret_buflen == 0` as the check to match the pattern in H5T_INTEGER and other branches.
The test for this fix is testH5Dread_vlen_of_compound_nullslot.
---
`translate_atomic_rebuf` had two issues related to handling of nested compounds. First, it discarded recursive returns, resulting in the construction of empty lists. Secondly, its member offset (`char_buf + i * typeSize + memb_offset`) was incorrect. In this case, `i` was the member index and `memberSize` was the entire cmpd size, so the offset would be erroneously large. It seems like this came from copying of the offset computation from `translate_rbuf`, which had to advance over entire elements of compound data. This error was duplicated on the write side in `translate_atomic_wbuf`'s H5T_COMPOUND case (h5util.c:4611).
I changed `translate_atomic_rbuf` to capture the resultant object, and dropped the `i * typeSize` term in both routines.
The new test verifying the fix works is `testH5Dread_vlen_of_nested_compound`.
* Add exception checks
* Update NULL checks in translate_wbuf
* Correct potentially bad array length check
* Clang format
* Fix readVL/writeVL crash on malformed buffer
* Committing clang-format changes
* Add bufSize checks to wbuf/rbuf translation
* Remove vlen pre-allocation support
* Harden JNI buffer interface
* Handle opaque types as byte[] and document JNI buffer data model
Opaque elements were grouped with H5T_INTEGER in the nested-type
translation path, which boxed them as Integer/Long and rejected
arbitrary-sized opaque blobs. Treat H5T_OPAQUE like H5T_REFERENCE
(a byte[] per element) in translate_atomic_rbuf, translate_atomic_wbuf,
and h5validate_atomic_wbuf so nested opaque round-trips correctly.
Also add "Buffer data model" header comments on translate_rbuf() and
translate_wbuf() and note the reference/opaque byte[] leaves in the
H5.java javadocv.
* Initialize typeSize to fix -Werror=maybe-uninitialized
typeSize was assigned only inside the vl_data_class branch but read in
a second, separate vl_data_class branch, which gcc -O2 flags as
maybe-uninitialized under -Werror. Initialize it to 0 at declaration in
H5Aread/H5Awrite/H5Dread/H5Dwrite, matching the existing vl_array_len
pattern.
* Port nested cmpd/vlen tests to java/test and sync reference
The legacy java/test tree's JUnit-TestH5D.txt reference listed the new
nested compound/vlen tests, but the corresponding @Test methods existed
only in java/src-jni/test/TestH5D.java. Port the 10 tests and the
writeCompoundOfVlenDataset helper into java/test/TestH5D.java, remove
debug prints, and
regenerate the reference to match the actual JUnit output.
* Support nested vlen/compound datatypes in Java FFM compat layer
The FFM compatibility layer (java/hdf) lacked the vlen/compound read and
write support that the JNI interface gained, so the nested cmpd/vlen tests
ported into java/test (TestH5D) failed and leaked an id.
VLDataConverter now has recursive encodeValue/decodeValue helpers that pack
and unpack any member class (integer, float, fixed/vl string, nested
compound, and VLEN) in the native HDF5 in-memory layout. These are wired
into convertCompoundDatatype, readCompoundDatatype and convertRawDataToArrayList,
and a type-aware convertToHVLAuto handles top-level VLEN-of-compound writes.
Compound reads now reclaim VL memory, and type/count mismatches raise
IllegalArgumentException instead of silently corrupting data.
H5DwriteVL rejects an undersized buffer up front and routes VLEN writes
through convertToHVLAuto. The JUnit-TestH5D reference regains its trailing
blank line to match the actual JUnit output.
* Committing clang-format changes
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
The FFM TestH5Z.java gained 8 new test methods covering the new filter
string-config API (H5Pappend_filter, H5Pget_filter_params_by_idx,
H5Zconfig_get_param_*), but the expected-output reference file was left
at 5 tests, causing the JUnit-TestH5Z CTest comparison to fail.
Three bugs in H5Pget_filter2:
1. cd_nelmts_segment was allocated as JAVA_INT (4 bytes) but size_t* needs
8 bytes on 64-bit — the C write overflowed into cd_values_segment.
2. cd_nelmts[0] was read back from cd_values_segment (wrong) instead of
cd_nelmts_segment.
3. cd_values and flags were never copied back from their native segments.
Fix: allocate cd_nelmts_segment as JAVA_LONG, seed it with the caller's
capacity on input, and copy all three output arrays back correctly.
H5Pappend_filter with CDVALUES is documented as identical to H5Pset_filter.
Routing through H5Pset_filter avoids constructing an H5Z_params_t struct
manually in FFM heap memory, which was silently producing cd_nelmts=0.
SymbolLookup.loaderLookup() only finds symbols loaded via
System.loadLibrary(); jextract loads libhdf5 via its own mechanism so
loaderLookup() never finds the RFC symbols at runtime.
Replace all SymbolLookup.loaderLookup()+MethodHandle patterns in the RFC
methods with direct hdf5_h.* calls, consistent with every other method in
H5.java:
H5Pappend_filter (both overloads)
H5Pget_filter_params_by_idx
H5Zget_filter_info2
H5Zconfig_has_key
H5Zconfig_get_param (long[], double[], boolean[], String[])
Two test failures from the h5dump nesting fix (2604f5dd6f5):
1. tools/test/h5repack/expected/deflate_limit.h5repack_layout.h5.ddl and
h5repack_layout.h5-plugin_test.ddl had PARAMS_STRING (and DESCRIPTION)
as flat siblings of the filter block rather than nested inside it —
the h5dump fixture files were updated in 2604f5dd6f5 but these two
h5repack expected files were missed.
2. java/src-jni/jni/h5zImp.c called CALL_CONSTRUCTOR with a 6-arg array
and signature "(IILjava/lang/String;Ljava/lang/String;ZZ)V" but
H5Z_class_info_t's constructor takes 7 args (adds has_blob_callbacks).
GetMethodID failed at runtime with the wrong arity. Add args[6] =
JNI_FALSE and update the descriptor to ZZZ)V.
Adds a human-readable key=value parameter string API for HDF5 filters,
alongside the existing integer cd_values arrays.
New C API:
- H5Pappend_filter(plist, filter_id, flags, params) — appends a filter
using either a key=value string or raw cd_values (H5Z_params_t)
- H5Pget_filter_params_by_idx(plist, idx, buf, buf_size, content_len) —
retrieves the parameter string for a filter by pipeline index
- H5Zconfig_get_int/double/bool/str — typed accessors for use inside
filter set_config callbacks
- H5Z_filter_id_by_name(name) — look up a filter id by registered name
- H5Zget_filter_info2(id, info) — extended filter info including v3 fields
New H5Z_class3_t fields: name, description, set_config, get_config,
and reserved blob-callback placeholders (write_blob/read_blob/close_blob).
H5Z_pipeline gains dxpl_id, scaled[], and ndims arguments threaded
through from all call sites so v3 filter callbacks have full context.
All six built-in filters (deflate, shuffle, fletcher32, nbit, szip,
scaleoffset) implement set_config/get_config callbacks.
TOML subset parser: tomlc17 (MIT) vendored in src/tomlc17/ and compiled
unconditionally into libhdf5. Hex-float literals are transparently
rewritten to decimal before parsing. tomlc17 symbols are hidden via
-fvisibility=hidden to prevent namespace collisions.
On-disk format: no new pipeline version. Parameter strings are converted
to cd_values by set_config at H5Pappend_filter time and stored using the
existing v2 pipeline message. On read, get_config reconstructs the string.
Full backward read compatibility is preserved.
Fortran, C++, and Java bindings added. Tests in test/tfilter2.c
(~2300 lines) and testpar/t_filters_parallel.c (par-01–par-04).
h5dump displays filter parameter strings; h5repack accepts TOML-form
UD= filter specs.
Code-review fixes included: tomlc17 visibility, H5Pget_filter_params_by_idx
arg validation and true-length two-pass contract, flags re-validation after
set_config, H5Z_register3 runtime plugin validation, Java two-pass protocol
and h5libraryError() consistency, CHANGELOG corrections.
Fixes GitHub issue #6153
Build-tree exports can't diverge from install-tree exports — the export(EXPORT ...) reads directly from the install export sets. No manual list to keep in sync.
Removed 3 global variables (HDF5_STATIC_LIBRARIES_TO_EXPORT, HDF5_JAVA_LIBRARIES_TO_EXPORT, HDF5_UTILS_TO_EXPORT) and their ~21 set_global_variable calls across tool/utility files.
Fixed the static-only build bug in the PR where the base export set was guarded by BUILD_SHARED_LIBS, breaking tools export.
Removed redundant utils in export files — the PR was dumping tools into all three build-tree export files (java, static, shared). Now they correctly appear only in the base export.
Addresses the "multiple @ param documentation sections" warnings are caused by the interaction of three things:
1. MACRO_EXPANSION = YES with EXPAND_ONLY_PREDEF = NO in Doxyfile.in
2. Versioned API macros in H5version.h like:
\def documentation in APIVersions.dox like:
3. \def H5Tarray_create \api_vers_2{H5Tarray_create,H5Tarray_create1,H5Tarray_create2}
Because Doxygen expands all macros, it sees H5Tarray_create as an alias for H5Tarray_create1. It then associates documentation from both the \def block (for the macro) and the function's own doc block (with \param entries) with the same function — resulting in "multiple @ param documentation sections."
* Update Doxygen version from 1.16.0 to 1.16.1
Use Doxygen 1.16.1 for all platforms. This version includes the latest
bug fixes and improvements while maintaining compatibility with our
documentation format.
* Fix Windows Doxygen installation for version 1.16.1
Handle different Windows binary naming conventions across Doxygen versions:
- 1.16.1 uses: doxygen-VERSION.windows.x64.bin.zip
- 1.16.0 uses: doxygen-VERSION.x64.bin.zip
Try the newer naming convention first, then fall back to the older one
if the download fails. This ensures compatibility across versions.
* Use ssciwr/doxygen-install in setup-doxygen action
Replace direct installation of official Doxygen binaries with
ssciwr/doxygen-install action. This avoids Java documentation parsing
errors that occur with official binaries but not with distribution
builds.
The setup-doxygen action now serves as a thin wrapper around
ssciwr/doxygen-install@f13be16c83 (v1),
using commit hash for security and reproducibility.
* Remove redundant version specification from workflows
The setup-doxygen action already has a default version (1.16.1),
so explicit version specification in workflows is unnecessary.
This follows DRY principles and makes version updates simpler.
Update version to 2.1 and derive version information from H5public.h, removing h5vers script and updating CMake and Java configurations.
Versioning:
Update version to 2.1 in H5public.h.
Derive version strings in H5public.h using macros.
CMake:
Extract version from H5public.h in HDF5config.cmake and HDF5AsSubdirMacros.cmake.
Configure README.md and CHANGELOG.md using CMakeLists.txt.
Java:
Generate H5Version.java from H5public.h for version consistency.
Update H5.java to use H5Version for version constants.
Removals:
Delete bin/h5vers script, previously used for version management.
Adds predefined datatypes for FP6 data in E2M3 and E3M2 formats
Does not add support for any native FP6 types; datatype conversions are performed in software
FFM build requires Java 25, Jextract 25.
Generates FFM bindings during configure.
JNI is default when the requirements are not met or can be forced.
Presets added for maven and FFM - JNI is default selection.
Enhanced Maven options will work with either JNI or FFM
New Workflows for testing and maven uploads.
Extensive documentation changes for java.
Adds predefined datatypes for FP8 data in E4M3 and E5M2 formats
Does not add support for any native FP8 types; datatype conversions are performed in software
In order to reduce hash collisions and take advantage of modern memory capacity, the default hash table size for the chunk cache has been increased from 521 to 8191. This means the hash table will consume approximately 64 KiB per open dataset. This value can be changed with H5Pset_cache() or H5Pset_chunk_cache(). This value was chosen because it is a prime number close to 8K.
Change default file format to 1.8 across various tests and examples, updating file creation and access logic accordingly.
Behavior:
Default file format version changed to 1.8 in H5Pfapl.c.
Updated file creation and access to use 1.8 format in h5ex_g_compact.c and test_file_image.c.
Set earliest file format in multiple test files including cache_tagging.c, dtypes.c, and links.c.
Tests:
Modified expected output in tools/test/misc/expected/*.ls files to reflect new file format locations.
Adjusted test logic in test_file_image.c and cache_tagging.c to accommodate format changes.
Misc:
Added comments and TODOs for future format testing in test_file_image.c.
Minor variable renaming for clarity in test_file_image.c.
Behavior:
Modify HDF5UseFortran.cmake to handle cross-compilation by providing default KINDs and sizes when CMAKE_CROSSCOMPILING is true.
Default values for INTEGER and REAL KINDs and their sizes are set when cross-compiling.
Error messages are adjusted to reflect cross-compilation scenarios.
Logic:
Conditional checks added for CMAKE_CROSSCOMPILING to determine if default values should be used.
Default values include INTEGER KINDs {1,2,4,8,16} and REAL KINDs {4,8,10,16}.
Default sizes for native kinds are set to 4 for INTEGER and REAL, and 8 for DOUBLE PRECISION when cross-compiling.
Misc:
Adjustments to verbose messages to indicate when defaults are used during cross-compilation.
Optimize VDS operations using R-tree spatial index, adding new API functions and tests for improved performance.
Behavior:
Introduces R-tree spatial index for optimizing VDS operations in H5Dvirtual.c.
Adds H5Pset_dset_use_spatial_tree() and H5Pget_dset_use_spatial_tree() to control R-tree usage.
Default behavior uses R-tree for VDS with more than 1000 mappings.
Implementation:
Adds H5RT.c, H5RTprivate.h, and H5RTpkg.h for R-tree implementation.
Updates H5Pdapl.c and H5Pdcpl.c to include R-tree properties.
Modifies H5Dvirtual.c to integrate R-tree in VDS I/O operations.
Testing:
Adds rtree.c for testing R-tree creation, search, and copy operations.
Tests R-tree integration with VDS in test/dsets.c.
Verifies R-tree behavior with different dataset access property list settings.
Adds predefined datatypes for little- and big-endian bfloat16 data
Does not add support for any native bfloat16 types; datatype conversions are performed in software
Also adds missing float16 predefined types to fortran
* Remove hbool_t from the public API
This was previously removed from the source code, but the API calls
retained hbool_t in the public API calls. This commit removes
hbool_t from the public APIs.
hbool_t is retained as a typedef for bool in H5public.h so existing
code doesn't break.
Scot Breitenfeld
Matt L
bmribler
Orion Poplawski
jhendersonHDF
Allen Byrne
Larry Knox
Neil Fortner
Dana Robinson