Adds HDF5Examples/C/H5PAR/ph5_comm_split.c, showing parallel HDF5 use on a
communicator produced by MPI_Comm_split rather than on MPI_COMM_WORLD.
The example splits MPI_COMM_WORLD by rank parity. The even ranks form a
writer group that collectively creates a dataset with one row per writer
rank, then reopens the file read-only on the same sub-communicator and
verifies what it wrote. The odd ranks stand in for a group with other
responsibilities and make no HDF5 calls at all.
The comments cover the two rules this pattern depends on: MPI_Comm_split is
collective over the parent communicator, so every rank must call it even if
it never opens the file; and once a file is opened with a FAPL carrying a
sub-communicator, every collective HDF5 call on that file is collective over
that sub-communicator alone.
The example runs with any number of MPI ranks.
The Fortran example set sel_type = H5S_BLOCK_F on the branch where a rank
calls H5Sselect_none (no data to contribute) and H5S_ALL_F on the branch
where a rank sets a real hyperslab, which is backwards. The equivalent C
example (ph5_filtered_writes_no_sel.c) does the opposite:
H5Dwrite(..., no_selection ? H5S_ALL : H5S_BLOCK, ...);
i.e. H5S_ALL when there is no selection (mirrors the file dataspace's own
empty selection) and H5S_BLOCK when a hyperslab was set (memory buffer
matches the block shape of the file selection).
With the inversion, H5Dwrite_f feeds a bogus selection into the collective
filtered I/O path, corrupting the element/byte-length accounting in
H5D__mpio_collective_filtered_chunk_update, which segfaults inside
H5D_select_io_mem -> H5VM_memcpyvv on 12 MPI ranks. Reproduces on current
develop; the bug has been present since the example was added in #3916.
* Build the examples as C++11 to match the HDF5 C++ library
The standalone examples build forces CMAKE_CXX_STANDARD 98, but
H5public.h includes <cinttypes>, which requires C++11. Any C++
translation unit that includes hdf5.h is therefore affected, not just
users of the C++ API, and the HDF5 C++ library itself is built as C++11
(config/flags/HDFCompilerCXXFlags.cmake). The C++ examples do not
compile as a result, against static or shared HDF5 alike.
Only the standalone build is affected, which is why this is not visible
in ordinary use. The C++98 setting lives in BASIC_SETTINGS, and
HDF5Examples/CMakeLists.txt skips that whole block when
EXAMPLES_EXTERNALLY_CONFIGURED is set -- which HDF5 does for its own
in-tree example build (config/cmake/HDF5ExampleCache.cmake). Built in
tree, the examples inherit HDF5's C++11 and compile normally, and that
is the path the CI workflows exercise. The standalone path, where the
C++98 setting does apply, is driven by the release scripts rather than
by the workflows, and has the C++ examples off by default.
* Select the examples' HL, Fortran and C++ libraries on the right variable
When the examples are built standalone against an installed HDF5, the
HL, Fortran and C++ branches choose between the shared and static
libraries using BUILD_SHARED_LIBS, while the C branch just above them
uses H5EXAMPLE_USE_SHARED_LIBS.
H5EXAMPLE_USE_SHARED_LIBS is what decides whether the "shared" or the
"static" component is requested from find_package, so only the matching
HDF5_<linkage>_<lang>_FOUND variables are ever set. BUILD_SHARED_LIBS
cannot select a linkage on its own; it can only agree or fail to match.
Of its four combinations with H5EXAMPLE_USE_SHARED_LIBS, three produce
no observable difference. In the fourth, H5EXAMPLE_USE_SHARED_LIBS=ON
with BUILD_SHARED_LIBS unset, the shared branch is not taken and the
static branch cannot be, so the HL, Fortran and C++ examples are
disabled with "libs not found" even though the libraries are installed
and were found.
Use H5EXAMPLE_USE_SHARED_LIBS, which is the declared option and is
already what the C branch uses.
A build driven through config/examples/CTestScript.cmake does not reach
the broken combination, because it configures with
HDF5Examples/config/cmake/cacheinit.cmake, which forces
BUILD_SHARED_LIBS=ON. A direct cmake invocation without that cache file
does. In either case the HL, Fortran and C++ examples are off by
default, so this is only visible once they are enabled.
BUILD_SHARED_LIBS remains documented as a user option in
config/examples/HDF5_Examples_options.cmake but no longer influences
library selection; that comment should be revisited separately.
The previous `lib/cmake/hdf5-config.cmake` isn't in CMake's find_package
search path since CMake expects that to be a common path with each
package having it's own subdirectory. This changes it to
`lib/cmake/hdf5/hdf5-config.cmake` so the config is now found when the
install prefix is in `CMAKE_PREFIX_PATH`.
This also sets `HDF5_USE_GNU_DIRS=ON` by default for non-Windows builds.
Fixes#6137
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* Fix runExecute.cmake: restore ERROR_APPEND and fix zero-or-more mask patterns
Three bugs were introduced when runTest.cmake was refactored into runExecute.cmake:
1. ERROR_APPEND=1 support was dropped. Tests that redirect their error-stack
output to stderr and rely on ERROR_APPEND to combine it with stdout before
comparison (e.g. H5TEST-err_compat, H5TEST-error_test) fail silently because
the reference file never matches the truncated actual output.
2. " line [0-9]*" uses zero-or-more digits, so it matches " line " (with zero
digits) in already-masked reference text like " line (number)", prepending
another "(number)" and producing " line (number)(number)".
3. "HDF5 [1-9]*[.][0-9]*..." has the "HDF5 " prefix and uses [1-9]* (zero-or-
more), so it does not match "2.2.0" in actual output (blocked by the leading
"("), and double-masks "version (number)" in the reference.
"H5Eget_auto[1-2]*" / "H5Eset_auto[1-2]*" have the same zero-or-more issue,
double-masking "(1 or 2)" already present in reference files.
Fix: restore ERROR_APPEND handling; change * to + for all digit/char class
quantifiers that can match zero occurrences in already-masked reference text;
remove the "HDF5 " prefix from the version pattern. Same fix applied to
HDF5Examples/config/cmake/runExecute.cmake which has identical bugs.
* Fix AOCC CI: source setenv_AOCC.sh from its actual install location
install.sh generates setenv_AOCC.sh inside the aocc-compiler-VERSION/
directory (where it runs from), not in the repo root one level up.
* Fix AOCC CI: replace hardcoded runner paths with GITHUB_WORKSPACE
All /home/runner/work/hdf5/hdf5/ paths were hardcoded to the HDFGroup/hdf5
repo name. Running the workflow in any fork (e.g. brtnfld/hdf5_swmr_2)
puts the checkout under a different path, breaking AOCC/OpenMPI install,
configure, and build steps. Use $GITHUB_WORKSPACE / ${{ github.workspace }}
so the paths resolve correctly regardless of repo name.
* fix(aocc): source setenv_AOCC.sh from workspace root, not compiler subdir
install.sh places setenv_AOCC.sh in the workspace root, not inside the
aocc-compiler-X.X.X/ directory.
* Fix runExecute.cmake: anchor HDF5 version regex to full token
Replace the over-broad version pattern with one that matches the complete
'HDF5 (x.y.z)' token, preventing over-matching of IP addresses and other
version-like strings, and avoiding nested substitution artifacts in .err
reference comparisons. Applied identically to both copies of the file.
Refactor the cross-compilation infrastructure to rely on the standard
CMAKE_CROSSCOMPILING_EMULATOR variable instead of the custom
CROSSCOMPILING_PATH environment hack.
Key changes:
- Removed `ENVIRONMENT "${CROSSCOMPILING_PATH}"` from all test definitions.
- Updated custom commands (e.g., H5match_types, H5_gen) to invoke the
emulator explicitly when cross-compiling.
- Updated ConfigureChecks.cmake and HDF5UseFortran.cmake to enable
`try_run` checks when an emulator is defined.
- Added config/toolchain/mingw-w64-x86-64-wine.sh wrapper to automatically
locate MinGW runtime libraries (like libgfortran) for Wine execution.
- Cleaned up toolchain files to remove hardcoded emulator paths, deferring
configuration to the build environment.
* Fix MPI Fortran configuration for HDF5Examples (#6205)
When building Fortran parallel examples standalone without MPI
wrappers, the compiler cannot find mpi.mod because
MPI_Fortran_INCLUDE_DIRS is not populated.
* Use MPI::MPI_Fortran imported target instead of raw MPI variables
Replace manual MPI_Fortran_LIBRARIES, MPI_Fortran_LINK_FLAGS, and
MPI_Fortran_INCLUDE_DIRS usage with the MPI::MPI_Fortran imported
target, which transitively carries all include dirs, compile flags,
and link flags. This eliminates the CMAKE_Fortran_EXE_LINKER_FLAGS
hack and the manual target_include_directories in H5PAR.
When building Fortran parallel examples without MPI wrappers, the
compiler couldn't find mpi.mod because MPI_Fortran_INCLUDE_DIRS
was not populated. This occurred because find_package(MPI) was only
called for C, not Fortran.
Added enable_language(Fortran) and find_package(MPI REQUIRED
COMPONENTS Fortran) to properly configure MPI for Fortran examples,
ensuring MPI include directories and libraries are correctly set.
Update version to 2.1 and derive version information from H5public.h, removing h5vers script and updating CMake and Java configurations.
Versioning:
Update version to 2.1 in H5public.h.
Derive version strings in H5public.h using macros.
CMake:
Extract version from H5public.h in HDF5config.cmake and HDF5AsSubdirMacros.cmake.
Configure README.md and CHANGELOG.md using CMakeLists.txt.
Java:
Generate H5Version.java from H5public.h for version consistency.
Update H5.java to use H5Version for version constants.
Removals:
Delete bin/h5vers script, previously used for version management.
Add generate-index-html.sh script and update workflows to generate and upload index.html files for HDF5 releases, enhancing Maven package testing and deployment.
FFM build requires Java 25, Jextract 25.
Generates FFM bindings during configure.
JNI is default when the requirements are not met or can be forced.
Presets added for maven and FFM - JNI is default selection.
Enhanced Maven options will work with either JNI or FFM
New Workflows for testing and maven uploads.
Extensive documentation changes for java.
Change default file format to 1.8 across various tests and examples, updating file creation and access logic accordingly.
Behavior:
Default file format version changed to 1.8 in H5Pfapl.c.
Updated file creation and access to use 1.8 format in h5ex_g_compact.c and test_file_image.c.
Set earliest file format in multiple test files including cache_tagging.c, dtypes.c, and links.c.
Tests:
Modified expected output in tools/test/misc/expected/*.ls files to reflect new file format locations.
Adjusted test logic in test_file_image.c and cache_tagging.c to accommodate format changes.
Misc:
Added comments and TODOs for future format testing in test_file_image.c.
Minor variable renaming for clarity in test_file_image.c.
Behavior:
Modify HDF5UseFortran.cmake to handle cross-compilation by providing default KINDs and sizes when CMAKE_CROSSCOMPILING is true.
Default values for INTEGER and REAL KINDs and their sizes are set when cross-compiling.
Error messages are adjusted to reflect cross-compilation scenarios.
Logic:
Conditional checks added for CMAKE_CROSSCOMPILING to determine if default values should be used.
Default values include INTEGER KINDs {1,2,4,8,16} and REAL KINDs {4,8,10,16}.
Default sizes for native kinds are set to 4 for INTEGER and REAL, and 8 for DOUBLE PRECISION when cross-compiling.
Misc:
Adjustments to verbose messages to indicate when defaults are used during cross-compilation.
Renamed all HDF5 examples build options from H5EX_* to H5EXAMPLE_* to better distinguish them from library options when building examples alongside the library.
Updated all CMake files, including:
Main examples CMakeLists.txt files
CMake configuration files in config/ directory
CMake preset files
Updated documentation files to reflect new option names
* Remove hbool_t from the public API
This was previously removed from the source code, but the API calls
retained hbool_t in the public API calls. This commit removes
hbool_t from the public APIs.
hbool_t is retained as a typedef for bool in H5public.h so existing
code doesn't break.
This update systematically renames variables across 2.0+ CMake files in the HDF5Examples directory and configuration templates. The goal is to change variables from sounding like build "options" (e.g., HDF5_BUILD_TOOLS, HDF5_ENABLE_PARALLEL) to reflecting library "state" (e.g., what features the installed library provides).
Implemented missing --help/-h, -c, -echo and -shlib/-noshlib options
Added new -nohl option to avoid building and linking against
high-level HDF5 libraries if desired
Implemented adding of HDF5 installation library directory to rpath of
resulting executable/library and added new -norpath option to avoid
this behavior if desired
Added parsing of HDF5_USE_SHLIB environment variable to determine
whether to link against shared or static HDF5 libraries
Added parsing of HDF5_PKG_CONFIG_ARGS environment variable to separate
pkg-config-specific options from compiler-specific options and prevent
conflicts
Scot Breitenfeld
Matt L
Chuck AtkinsandLarry Knox
jhendersonHDF
Julien Schueller
H. Joe Lee
bmribler
Allen Byrne
Neil Fortner
Dana Robinson
Aleksandar Jelenak