write_blob/read_blob/close_blob and the corresponding H5Z_PARAMS_BLOB
enum value and has_blob_callbacks info field were speculative
reservations for a not-yet-designed feature. Pre-reserving generic
void* slots doesn't avoid the ABI/version bump the feature will need
once its callback signatures are actually decided, so there's no
benefit to carrying them now. Drop them from the public struct, the
internal filter table entry, all built-in filter registrations, the
Java bindings (JNI and Panama/FFM), and the test plugin/test suite.
The trailing "2" follows HDF5's convention for versioned/deprecated
symbols (H5Dopen2, H5Gcreate2, etc.), implying this is a successor to
H5Zget_filter_info registered in H5vers.txt -- it isn't. The two functions
are independent: H5Zget_filter_info returns only encode/decode config
flags, H5Zget_filter_class_info additionally exposes name, description,
and set_config/get_config presence. Renamed across C, Fortran, and both
Java binding variants (JNI + FFM) since it hadn't shipped yet.
* Fix H5DreadVL failing for pre-allocate cmpd-of-seq dsets
* Fix bad vlen of cmpd with null slot read
* Fix bad cmpd of cmpd read in java
`translate_rbuf`'s H5T_VLEN case had a similar bug where when `found_jList` was set to false due to an entyr in `ret_buf` being null, `ret_buf.add()` would be invoked on an array of objects without the list .add() method. This would occur whenever a read was invoked of a vlen sequence with a null (non-preallocated) entry. The pre-existing tests only tested the pre-allocated cases.
I removed the use of the `found_jList` flag, since it conflated the passing of an unallocated slot with `ret_buf` not being an array. Instead use `ret_buflen == 0` as the check to match the pattern in H5T_INTEGER and other branches.
The test for this fix is testH5Dread_vlen_of_compound_nullslot.
---
`translate_atomic_rebuf` had two issues related to handling of nested compounds. First, it discarded recursive returns, resulting in the construction of empty lists. Secondly, its member offset (`char_buf + i * typeSize + memb_offset`) was incorrect. In this case, `i` was the member index and `memberSize` was the entire cmpd size, so the offset would be erroneously large. It seems like this came from copying of the offset computation from `translate_rbuf`, which had to advance over entire elements of compound data. This error was duplicated on the write side in `translate_atomic_wbuf`'s H5T_COMPOUND case (h5util.c:4611).
I changed `translate_atomic_rbuf` to capture the resultant object, and dropped the `i * typeSize` term in both routines.
The new test verifying the fix works is `testH5Dread_vlen_of_nested_compound`.
* Add exception checks
* Update NULL checks in translate_wbuf
* Correct potentially bad array length check
* Clang format
* Fix readVL/writeVL crash on malformed buffer
* Committing clang-format changes
* Add bufSize checks to wbuf/rbuf translation
* Remove vlen pre-allocation support
* Harden JNI buffer interface
* Handle opaque types as byte[] and document JNI buffer data model
Opaque elements were grouped with H5T_INTEGER in the nested-type
translation path, which boxed them as Integer/Long and rejected
arbitrary-sized opaque blobs. Treat H5T_OPAQUE like H5T_REFERENCE
(a byte[] per element) in translate_atomic_rbuf, translate_atomic_wbuf,
and h5validate_atomic_wbuf so nested opaque round-trips correctly.
Also add "Buffer data model" header comments on translate_rbuf() and
translate_wbuf() and note the reference/opaque byte[] leaves in the
H5.java javadocv.
* Initialize typeSize to fix -Werror=maybe-uninitialized
typeSize was assigned only inside the vl_data_class branch but read in
a second, separate vl_data_class branch, which gcc -O2 flags as
maybe-uninitialized under -Werror. Initialize it to 0 at declaration in
H5Aread/H5Awrite/H5Dread/H5Dwrite, matching the existing vl_array_len
pattern.
* Port nested cmpd/vlen tests to java/test and sync reference
The legacy java/test tree's JUnit-TestH5D.txt reference listed the new
nested compound/vlen tests, but the corresponding @Test methods existed
only in java/src-jni/test/TestH5D.java. Port the 10 tests and the
writeCompoundOfVlenDataset helper into java/test/TestH5D.java, remove
debug prints, and
regenerate the reference to match the actual JUnit output.
* Support nested vlen/compound datatypes in Java FFM compat layer
The FFM compatibility layer (java/hdf) lacked the vlen/compound read and
write support that the JNI interface gained, so the nested cmpd/vlen tests
ported into java/test (TestH5D) failed and leaked an id.
VLDataConverter now has recursive encodeValue/decodeValue helpers that pack
and unpack any member class (integer, float, fixed/vl string, nested
compound, and VLEN) in the native HDF5 in-memory layout. These are wired
into convertCompoundDatatype, readCompoundDatatype and convertRawDataToArrayList,
and a type-aware convertToHVLAuto handles top-level VLEN-of-compound writes.
Compound reads now reclaim VL memory, and type/count mismatches raise
IllegalArgumentException instead of silently corrupting data.
H5DwriteVL rejects an undersized buffer up front and routes VLEN writes
through convertToHVLAuto. The JUnit-TestH5D reference regains its trailing
blank line to match the actual JUnit output.
* Committing clang-format changes
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Three bugs in H5Pget_filter2:
1. cd_nelmts_segment was allocated as JAVA_INT (4 bytes) but size_t* needs
8 bytes on 64-bit — the C write overflowed into cd_values_segment.
2. cd_nelmts[0] was read back from cd_values_segment (wrong) instead of
cd_nelmts_segment.
3. cd_values and flags were never copied back from their native segments.
Fix: allocate cd_nelmts_segment as JAVA_LONG, seed it with the caller's
capacity on input, and copy all three output arrays back correctly.
H5Pappend_filter with CDVALUES is documented as identical to H5Pset_filter.
Routing through H5Pset_filter avoids constructing an H5Z_params_t struct
manually in FFM heap memory, which was silently producing cd_nelmts=0.
SymbolLookup.loaderLookup() only finds symbols loaded via
System.loadLibrary(); jextract loads libhdf5 via its own mechanism so
loaderLookup() never finds the RFC symbols at runtime.
Replace all SymbolLookup.loaderLookup()+MethodHandle patterns in the RFC
methods with direct hdf5_h.* calls, consistent with every other method in
H5.java:
H5Pappend_filter (both overloads)
H5Pget_filter_params_by_idx
H5Zget_filter_info2
H5Zconfig_has_key
H5Zconfig_get_param (long[], double[], boolean[], String[])
Adds a human-readable key=value parameter string API for HDF5 filters,
alongside the existing integer cd_values arrays.
New C API:
- H5Pappend_filter(plist, filter_id, flags, params) — appends a filter
using either a key=value string or raw cd_values (H5Z_params_t)
- H5Pget_filter_params_by_idx(plist, idx, buf, buf_size, content_len) —
retrieves the parameter string for a filter by pipeline index
- H5Zconfig_get_int/double/bool/str — typed accessors for use inside
filter set_config callbacks
- H5Z_filter_id_by_name(name) — look up a filter id by registered name
- H5Zget_filter_info2(id, info) — extended filter info including v3 fields
New H5Z_class3_t fields: name, description, set_config, get_config,
and reserved blob-callback placeholders (write_blob/read_blob/close_blob).
H5Z_pipeline gains dxpl_id, scaled[], and ndims arguments threaded
through from all call sites so v3 filter callbacks have full context.
All six built-in filters (deflate, shuffle, fletcher32, nbit, szip,
scaleoffset) implement set_config/get_config callbacks.
TOML subset parser: tomlc17 (MIT) vendored in src/tomlc17/ and compiled
unconditionally into libhdf5. Hex-float literals are transparently
rewritten to decimal before parsing. tomlc17 symbols are hidden via
-fvisibility=hidden to prevent namespace collisions.
On-disk format: no new pipeline version. Parameter strings are converted
to cd_values by set_config at H5Pappend_filter time and stored using the
existing v2 pipeline message. On read, get_config reconstructs the string.
Full backward read compatibility is preserved.
Fortran, C++, and Java bindings added. Tests in test/tfilter2.c
(~2300 lines) and testpar/t_filters_parallel.c (par-01–par-04).
h5dump displays filter parameter strings; h5repack accepts TOML-form
UD= filter specs.
Code-review fixes included: tomlc17 visibility, H5Pget_filter_params_by_idx
arg validation and true-length two-pass contract, flags re-validation after
set_config, H5Z_register3 runtime plugin validation, Java two-pass protocol
and h5libraryError() consistency, CHANGELOG corrections.
Fixes GitHub issue #6153
Addresses the "multiple @ param documentation sections" warnings are caused by the interaction of three things:
1. MACRO_EXPANSION = YES with EXPAND_ONLY_PREDEF = NO in Doxyfile.in
2. Versioned API macros in H5version.h like:
\def documentation in APIVersions.dox like:
3. \def H5Tarray_create \api_vers_2{H5Tarray_create,H5Tarray_create1,H5Tarray_create2}
Because Doxygen expands all macros, it sees H5Tarray_create as an alias for H5Tarray_create1. It then associates documentation from both the \def block (for the macro) and the function's own doc block (with \param entries) with the same function — resulting in "multiple @ param documentation sections."
* Update Doxygen version from 1.16.0 to 1.16.1
Use Doxygen 1.16.1 for all platforms. This version includes the latest
bug fixes and improvements while maintaining compatibility with our
documentation format.
* Fix Windows Doxygen installation for version 1.16.1
Handle different Windows binary naming conventions across Doxygen versions:
- 1.16.1 uses: doxygen-VERSION.windows.x64.bin.zip
- 1.16.0 uses: doxygen-VERSION.x64.bin.zip
Try the newer naming convention first, then fall back to the older one
if the download fails. This ensures compatibility across versions.
* Use ssciwr/doxygen-install in setup-doxygen action
Replace direct installation of official Doxygen binaries with
ssciwr/doxygen-install action. This avoids Java documentation parsing
errors that occur with official binaries but not with distribution
builds.
The setup-doxygen action now serves as a thin wrapper around
ssciwr/doxygen-install@f13be16c83 (v1),
using commit hash for security and reproducibility.
* Remove redundant version specification from workflows
The setup-doxygen action already has a default version (1.16.1),
so explicit version specification in workflows is unnecessary.
This follows DRY principles and makes version updates simpler.
Update version to 2.1 and derive version information from H5public.h, removing h5vers script and updating CMake and Java configurations.
Versioning:
Update version to 2.1 in H5public.h.
Derive version strings in H5public.h using macros.
CMake:
Extract version from H5public.h in HDF5config.cmake and HDF5AsSubdirMacros.cmake.
Configure README.md and CHANGELOG.md using CMakeLists.txt.
Java:
Generate H5Version.java from H5public.h for version consistency.
Update H5.java to use H5Version for version constants.
Removals:
Delete bin/h5vers script, previously used for version management.
FFM build requires Java 25, Jextract 25.
Generates FFM bindings during configure.
JNI is default when the requirements are not met or can be forced.
Presets added for maven and FFM - JNI is default selection.
Enhanced Maven options will work with either JNI or FFM
New Workflows for testing and maven uploads.
Extensive documentation changes for java.
M. Scot Breitenfeld
Matt Landgithub-actions
Larry Knox
Allen Byrne