Fix issue where chunked datasets could get setup with an incorrect
chunking index type in parallel HDF5
Fix issue where metadata cache images with an undefined address
and size of 0 couldn't be properly decoded
Fix issue where a flag in H5Cimage.c wasn't getting set correctly
for release builds of the library, leading to incorrect error
checking when reconstructing metadata cache entries
Ensure that the Subfiling VFD's stub file gets truncated when subfiles are
being truncated to avoid a potentially unreadable file when the EOF doesn't
match the stored EOF value
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Refactor the cross-compilation infrastructure to rely on the standard
CMAKE_CROSSCOMPILING_EMULATOR variable instead of the custom
CROSSCOMPILING_PATH environment hack.
Key changes:
- Removed `ENVIRONMENT "${CROSSCOMPILING_PATH}"` from all test definitions.
- Updated custom commands (e.g., H5match_types, H5_gen) to invoke the
emulator explicitly when cross-compiling.
- Updated ConfigureChecks.cmake and HDF5UseFortran.cmake to enable
`try_run` checks when an emulator is defined.
- Added config/toolchain/mingw-w64-x86-64-wine.sh wrapper to automatically
locate MinGW runtime libraries (like libgfortran) for Wine execution.
- Cleaned up toolchain files to remove hardcoded emulator paths, deferring
configuration to the build environment.
Behavior:
Modify HDF5UseFortran.cmake to handle cross-compilation by providing default KINDs and sizes when CMAKE_CROSSCOMPILING is true.
Default values for INTEGER and REAL KINDs and their sizes are set when cross-compiling.
Error messages are adjusted to reflect cross-compilation scenarios.
Logic:
Conditional checks added for CMAKE_CROSSCOMPILING to determine if default values should be used.
Default values include INTEGER KINDs {1,2,4,8,16} and REAL KINDs {4,8,10,16}.
Default sizes for native kinds are set to 4 for INTEGER and REAL, and 8 for DOUBLE PRECISION when cross-compiling.
Misc:
Adjustments to verbose messages to indicate when defaults are used during cross-compilation.
Adds H5FDsubfiling_get_file_mapping() API to map logical HDF5 files to physical subfiles, with Fortran support and comprehensive testing.
Adds H5FDsubfiling_get_file_mapping() in H5FDsubfiling.c to retrieve subfile paths for a logical HDF5 file.
Updates Fortran interface in H5VFDff.F90 to include h5fdsubfiling_get_file_mapping_f().
Updates h5fuse.in to support a list of subfiles for processing with a new -l option.
* Refactor warning macros
Replaces the scheme in H5warnings.h with a more compiler-agnostic
one that relocates ifdef complexity to the warnings header file
and uses better naming.
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fix an oversize allocation in t_bigio on 32-bit systems
A test in t_bigio.c attempts to allocate more than 4 GiB of memory,
which will overflow (and wrap) the size_t type on 32-bit systems,
creating a very small allocation instead of a very large allocation.
The test then segfaults when it accesses memory outside of the small
buffer.
The test has been fixed by limiting the buffer to 2 GiB on 32-bit
systems.
Adds new -testexpress command-line option to the testframe testing
framework to allow setting or overriding of the TestExpress level
at runtime
Adds macros for the different currently defined TestExpress levels
Makes testing parameters specified with AddTest non-const again when
passed to test functions. This allows test functions to update their
parameters during execution and is particularly useful for tests
which use those parameters to accumulate statistics, error counts,
etc. without resorting to global variables.
Since each API context is local to a thread, use the stack to
store the context instead of allocating & releasing it each time.
This improves performance (slightly), reduces alloc/free calls,
and eliminates the H5FL package from the push & pop operations,
which helps simplify threadsafe operation.
One effect of this change is that the H5VLstart_lib_state /
H5VLfinish_lib_state API routines for pass through connector
authors now require a parameter that can be used to store
the library's context. It was probably a mistake to assume
that these two routines would not do this previously, so this
is essentially a bug fix for them.
Some other minor things:
* Added API context push+pop operations to cache tests
(I'm not actually certain why this was working before) and
a few other places
* Cleaned up a bunch of warnings in test code (calloc args, mainly)
* Made header file inclusions more standard in some source files
This is where most people will expect to find license information. The
COPYING_LBNL_HDF5 file has also been renamed to LICENSE_LBNL_HDF5.
The licenses are unchanged.
Added setup and cleanup callback parameters to TestInit() to perform
setup and cleanup tasks once for whole test program
Removed TestCleanup() function since its functionality is covered by
PerformTests()
Added check of the HDF5_NOCLEANUP environment variable in
GetTestCleanup()
Added new testframe.h header to document testing framework functions and
split them away from h5test.h and from test programs that don't
integrate with the testframe.c testing framework
Added new test setup callback to testframe.c testing framework
Added parameters to AddTest() to specify size of test parameters so they
can be copied for later use
Enabled HDF5 error stacks in testframe.c framework by default and added
some error stack suppressions to some testhdf5 tests
Added new maxthreads option to testframe.c framework to allow specifying
the maximum number of threads a multi-threaded test can use
Moved TestExpress functionality out of testframe.c and into more general
h5test.c for wider use by tests through getter and setter
Updated some tests to not mix and match functionality between h5test.c/h
and testframe.c/h
Moved some functionality from testphdf5.h into testpar.h for parallel
tests that aren't part of testphdf5
Added new parallel test library that contains common shared
functionality for parallel tests (similar to h5test library)
Fixes a bug where the vector I/O sizes weren't being extended when
one of the entries in the array is 0. This caused an over-read of
the I/O sizes buffer and on some machines would cause a memory
allocation failure due to the calculated I/O vector size being too
large.
The t_pmulti_dset and t_select_io_dset tests rely on the behavior
of the previous private rand_r-like implementation to get the
correct sequence of random numbers to pass. This has been restored
using a fully private rand_r-like implementation that doesn't
rely on rand_r and will work on Windows and other platforms
where rand_r doesn't exist.
* Clean up random number generator code
Depending on the platform, we use a mix of random, rand, and rand_r
to generate pseudo-random numbers, along with a messy set of ifdefs
in H5private.h. We are not a cryptographic library, only use random
numbers in our test code, and have no need for anything more than the
C standard's (s)rand(). There's no point dithering about using rand()
vs random() when we're also doing bad things like using mod to
restrict the range, which introduces bias.
Also removes CMake/configure checks for rand_r and random
* Remove random/rand_r checks from build system
* Fix missed HDrandom after GitHub merge
Changes Autotools testing to use HDF5_TEST_DRIVER environment
variable to avoid running tests that don't work well with several
VFDs
Restores old h5_get_vfd_fapl() testing function to setup a FAPL
with a particular VFD
Adds a macro for the default VFD name
* Fix issue with Subfiling VFD and multiple opens of same file
* Update H5_subfile_fid_to_context to return error value instead of ID
* Add helper routine to initialize open file mapping
The buffers passed to stat-like calls are only partially filled in by
the call, leaving ununitialized memory areas when the stat buffers are
created on the stack.
This change memsets the buffers to 0 before the stat calls, quieting
the -fsanitze=memory complaints.
off_t is a 32-bit signed value on Windows, so we should use HDoff_t
(which is __int64 on Windows) internally instead.
Also defines HDftell on Windows to be _ftelli64().
Vector I/O requests are now processed within a single
set of I/O call batches, rather than each I/O vector
entry (tuple constructed from the types, addrs, sizes
and bufs arrays) being processed individually. This allows I/O to be
more efficiently parallelized among the I/O concentrator processes
during large I/O requests.
* Fixed some calculations and add test cases for issues spotted from review
* Removed a variable that was compensating for previous miscalculations
* Changes for ECP-344: Implement selection vector I/O with collective chunk filling.
Also fix a bug in H5FD__mpio_write_vector() to account for fixed size optimization
when computing max address.
* Fixes based on PR review comments:
For H5Dchunk.c: fix H5MM_xfree()
For H5FDmpio.c:
1) Revert the fix to H5FD__mpio_write_vector()
2) Apply the patch from Neil on the proper length of s_sizes reported by H5FD__mpio_vector_build_types()
* Put back the logic of dividing up the work among all the mpi ranks similar to the
original H5D__chunk_collective_fill() routine.
* Add a test to verify the fix for the illegal reference problem in H5FD__mpio_write_vector().
jhendersonHDF
Julien Schueller
efferre79
Allen Byrne
Scot Breitenfeld
Dana Robinsonandgithub-actions
Matt L
Quincey Koziol
H. Joe Lee
Glenn Song
Sean McBride
vchoi-hdfgroup