mirror of
https://github.com/HDFGroup/hdf5.git
synced 2026-09-25 04:09:44 +03:00
1306 lines
58 KiB
Plaintext
1306 lines
58 KiB
Plaintext
HDF5 version 2.0.0-2 currently under development
|
|
================================================================================
|
|
|
|
|
|
INTRODUCTION
|
|
============
|
|
|
|
This document describes the differences between this release and the previous
|
|
HDF5 release. It contains information on the platforms tested and known
|
|
problems in this release. For more details check the HISTORY*.txt files in the
|
|
HDF5 source.
|
|
|
|
Note that documentation in the links below will be updated at the time of each
|
|
final release.
|
|
|
|
Links to HDF5 documentation can be found on:
|
|
|
|
https://support.hdfgroup.org/releases/hdf5/latest-docs.html
|
|
|
|
The official HDF5 releases can be obtained from:
|
|
|
|
https://support.hdfgroup.org/downloads/index.html
|
|
|
|
Changes from Release to Release and New Features in the HDF5-2.x.y release series
|
|
can be found at:
|
|
|
|
https://support.hdfgroup.org/releases/hdf5/documentation/release_specific_info.md
|
|
|
|
If you have any questions or comments, please send them to the HDF Help Desk:
|
|
|
|
help@hdfgroup.org
|
|
|
|
|
|
CONTENTS
|
|
========
|
|
|
|
- New Features
|
|
- Support for new platforms and languages
|
|
- Bug Fixes since HDF5-2.0.0
|
|
- Platforms Tested
|
|
- Known Problems
|
|
- CMake Installations
|
|
|
|
|
|
New Features
|
|
============
|
|
|
|
Configuration:
|
|
-------------
|
|
- Refactored HDF5_BUILD/ENABLE_{feature} variable in hdf5-config.cmake file
|
|
|
|
The variables used in hdf5-config.cmake to indicate what options were used to
|
|
build the installed library have been renamed. All HDF5_BUILD/ENABLE_{feature}
|
|
variables are now HDF5_PROVIDES_{feature}. This more clearly indicates that these
|
|
variables reflect whether the feature is supported by the installed library,
|
|
instead of whether the feature is an option that can be changed when building
|
|
an application with the library.
|
|
|
|
Created MACRO EXTERNAL_HDF5_STATUS to convert between the old and new names.
|
|
The macro is in the config/examples/HDF5AsSubdirMacros.cmake file and can be
|
|
copied into a project's CMakeLists.txt file to provide backward compatibility.
|
|
|
|
- CMake minimum version is now 3.26
|
|
|
|
The minimum version of CMake is now 3.26, this will remove workarounds to handle
|
|
versions between 3.18 (previous minimum) and 3.26.
|
|
|
|
- Removed HDF5_ENABLE_THREADS option
|
|
|
|
The HDF5_ENABLE_THREADS option has been removed, as it no longer functions
|
|
as a proper build option. The library will always check for thread support
|
|
and set the internal status variable, HDF5_THREADS_ENABLED.
|
|
The HDF5_ENABLE_THREADSAFE option is still available to build with
|
|
thread-safe API calls.
|
|
|
|
- Reorganized the files in the config/cmake folder into the config folder structure
|
|
|
|
The config folder CMake files have been reorganized to make it easier to
|
|
maintain and add new features. The following changes have been made:
|
|
- The files in the config folder are the macros and templates for the
|
|
build process.
|
|
- The files in the config/cmake folder are primarily for optional features.
|
|
- The files in the config/install folder are for installation support.
|
|
- The files in the config/flags folder are for compiler support.
|
|
- The remaining folders remain unchanged.
|
|
|
|
- Added CMake configuration options to override compilers in h5cc:
|
|
|
|
HDF5_H5CC_C_COMPILER for the C compiler
|
|
HDF5_H5CC_CXX_COMPILER for the C++ compiler
|
|
HDF5_H5CC_Fortran_COMPILER for the Fortran compiler
|
|
|
|
These default to the currently used compiler, preserving the current
|
|
behavior. However, they can be overridden by users who need to use
|
|
a different compiler at runtime, for example when they build via ccache.
|
|
|
|
- Added CMAKE_INSTALL_PREFIX to the default plugin path
|
|
|
|
To help users find their plugins, the default plugin path has been
|
|
changed to include the CMAKE_INSTALL_PREFIX. Adding the install prefix
|
|
allows users to skip setting the HDF5_PLUGIN_PATH environment variable
|
|
when using plugins with the default lib/plugin location.
|
|
|
|
- Removed support for autotools build system.
|
|
|
|
- The documentation in the source folder, doc, has been converted to doxygen files.
|
|
|
|
- Added configuration option for API concurrency support:
|
|
|
|
CMake: HDF5_ENABLE_CONCURRENCY (ON/OFF) (Default: OFF)
|
|
|
|
This option enables support for concurrent multithreaded operation
|
|
of supported API routines. This option also provides threadsafe
|
|
execution of all other, non-concurrent operations. The 'concurrency'
|
|
option thus is a superset of the existing 'threadsafe' option. Both
|
|
options are currently available, although mutually exclusive. As the
|
|
'concurrency' code becomes more stable over time, the 'threadsafe' option
|
|
may be deprecated in favor of the new 'concurrency' option.
|
|
The following API routines support concurrent multithreaded operation:
|
|
<none yet>
|
|
|
|
- ************ Renamed the option: HDF5_ENABLE_Z_LIB_SUPPORT ************
|
|
|
|
The option has been renamed to HDF5_ENABLE_ZLIB_SUPPORT to be consistent
|
|
with the naming of other options.
|
|
*** Also, the option defaults to OFF. This requires the user to explicitly ***
|
|
*** enable zlib support when configuring the library. ***
|
|
|
|
- Added support for MinGW + MSYS2 when building with CMake
|
|
|
|
We added support for this to the appropriate
|
|
configure-time checks in CMake.
|
|
|
|
CMake + MinGW + MSYS2 is now tested with the following environments:
|
|
|
|
* mingw32
|
|
* mingw64
|
|
* ucrt64
|
|
* clang64
|
|
|
|
- Added CMake build mode flags to the libhdf5.settings file
|
|
|
|
Flags from the CMake build mode (e.g., optimization) are not a part of
|
|
CMAKE_<language>_FLAGS and were not exported to the libhdf5.settings file. This
|
|
has been fixed and the C, Fortran, and C++ build mode flags are now exported to
|
|
the file.
|
|
|
|
This also affects the text output of H5check_version() and the libhdf5.settings
|
|
string stored in the library (for those who use strings(1), etc. to get
|
|
build info from the binary).
|
|
|
|
- CMake: Split compiler specific flags into separate files
|
|
|
|
The compiler specific flags have been split into separate files to make
|
|
it easier to maintain and add new compiler flags. The flags for NVHPC,
|
|
Intel, GNU and Clang compilers are now in separate files included from
|
|
the current compiler flags files; HDFCompiler<language>Flags.cmake.
|
|
|
|
- Added support for native zlib-ng compression.
|
|
|
|
Changed the zlib-ng CMake logic to prefer the native zlib-ng library. Added
|
|
#ifdef around the compression function calls. Added including the correct
|
|
header file with the same #ifdef.
|
|
|
|
- Renamed remaining HDF5 library CMake options except for CMake BUILD* variables
|
|
|
|
DEFAULT_API_VERSION to HDF5_DEFAULT_API_VERSION
|
|
DISABLE_PDB_FILES to HDF5_DISABLE_PDB_FILES
|
|
ONLY_SHARED_LIBS to HDF5_ONLY_SHARED_LIBS
|
|
ALLOW_UNSUPPORTED to HDF5_ALLOW_UNSUPPORTED
|
|
TEST_SHELL_SCRIPTS to HDF5_TEST_SHELL_SCRIPTS
|
|
|
|
All other HDF5 library CMake options are prefixed with HDF5_
|
|
|
|
- bin/cmakehdf5 has been removed
|
|
|
|
This was an unsupported build script that made building HDF5 via CMake
|
|
work like building HDF5 via the Autotools. It has been unmaintained
|
|
for a long time, has been marked deprecated, and is being removed.
|
|
|
|
- Generated files in src are now checked into version control
|
|
|
|
These files are infrequently updated and generating them adds a
|
|
dependency on Perl. The listed files are now checked in and do
|
|
not need to be recreated when checking out development branches.
|
|
|
|
* H5Edefin.h
|
|
* H5Einit.h
|
|
* H5Emajdef.h
|
|
* H5Emindef.h
|
|
* H5Epubgen.h
|
|
* H5Eterm.h
|
|
* H5overflow.h
|
|
* H5version.h
|
|
|
|
- Dropped some old Solaris Studio work-arounds
|
|
|
|
Solaris Studio no longer seems to be maintained and the last version
|
|
(12.4, circa 2015) doesn't seem to fully support C11. We've removed
|
|
some hacks that work around things like __attribute__() support.
|
|
|
|
- Dropped support for the traditional MSVC preprocessor
|
|
|
|
Visual Studio has recently started using a standards-compliant
|
|
preprocessor (In VS2019+) and this is the default in C11.
|
|
|
|
https://learn.microsoft.com/en-us/cpp/preprocessor/preprocessor-experimental-overview?view=msvc-170
|
|
|
|
Because of this, we've dropped support for the traditional
|
|
MSVC preprocessor.
|
|
|
|
- The standard for building the library is now C11
|
|
|
|
We have updated the build files to set the C standard to C11, though
|
|
some platforms use gnu11 to get some GNU things to work.
|
|
|
|
|
|
Library:
|
|
--------
|
|
- Removed hbool_t from public API calls
|
|
|
|
The hbool_t type was introduced before the library supported C99's
|
|
Boolean type. Originally typedef'd to an integer, it has been typedef'd
|
|
to C99's bool for many years.
|
|
|
|
It had been previously purged from the bulk of the library code and
|
|
only remained in public API signatures. In HDF5 2.0, it has also been
|
|
removed from public API signatures.
|
|
|
|
The hbool_t typedef remains in H5public.h so existing code does not
|
|
need to be updated.
|
|
|
|
- H5public.h no longer includes features.h
|
|
|
|
features.h is supposed to be included by glibc headers and not used in
|
|
application code. It is unnecessary given our use of feature test macros
|
|
like _POSIX_C_SOURCE and has been removed.
|
|
|
|
- Improved performance of opening a virtual dataset with many mappings
|
|
|
|
When opening a virtual dataset, the library would previously decode
|
|
the mappings in the object header package, then copy them to the dataset
|
|
struct, then copy them to the internal DCPL. Copying the VDS mappings
|
|
could be very expensive if there were many mappings. Changed this to delay
|
|
decoding the mappings until the dataset code, and delay copying the layout
|
|
to the DCPL until it is needed. This results in only the decoding and no
|
|
copies in most use cases, as opposed to the decoding and two copies with
|
|
the previous code.
|
|
|
|
- Aligned the CMake compiler wrappers with the old Autotools versions
|
|
|
|
The versions of h5cc, h5fc, h5c++, etc. generated by CMake were missing
|
|
several options and features from the Autotools counterparts. Some of
|
|
these options and features have now been implemented in the CMake versions,
|
|
while some of them have not:
|
|
|
|
* The missing --help/-h, -c, -echo and -shlib/-noshlib options have
|
|
been implemented.
|
|
|
|
* The -prefix option was not implemented, as it didn't appear to
|
|
function in the Autotools wrappers and is generally covered by
|
|
pkg-config in the CMake wrappers.
|
|
|
|
* A new -nohl option has been added to avoid building and linking
|
|
against the high-level HDF5 libraries if desired.
|
|
|
|
* Similar to the Autotools wrappers, the CMake wrappers now add the
|
|
HDF5 installation library directory to the rpath of the resulting
|
|
executable/library by default when linking against shared HDF5
|
|
libraries. This behavior can be avoided by specifying the new
|
|
-norpath option.
|
|
|
|
* Parsing of the HDF5_USE_SHLIB environment variable has been added
|
|
to determine whether to link against shared or static HDF5 libraries.
|
|
Precedence is still given to the -shlib/-noshlib options.
|
|
|
|
* Parsing of the HDF5_PKG_CONFIG_ARGS environment variable has been
|
|
added to separate pkg-config-specific options from compiler-specific
|
|
options and prevent conflicts between them.
|
|
|
|
Several issues were also fixed in the pkg-config files that are generated
|
|
by CMake.
|
|
|
|
- Changed the default page buffer size for the ROS3 driver
|
|
|
|
Calling H5Pset_fapl_ros3() now has the side effect of setting the page
|
|
buffer size in the FAPL to 64MiB if it was not previously set. This will
|
|
only have an effect if the file uses paged allocation. Also added the
|
|
H5F_PAGE_BUFFER_SIZE_DEFAULT to allow the user to unset the page buffer
|
|
size in a FAPL so it can be similarly overridden.
|
|
|
|
- Default dataset chunk cache size increased
|
|
|
|
The default dataset chunk cache size is increased to 8 MiB (8,388,608 bytes).
|
|
|
|
- The file format has been updated to 4.0
|
|
|
|
The Virtual Dataset Global Heap Block format has been updated to version 1
|
|
to support shared string storage for source filenames and dataset names,
|
|
reducing file size when multiple mappings reference the same sources. This
|
|
new format is only used when the HDF5 library version bounds lower bound
|
|
is set to 2.0 or later.
|
|
|
|
- The H5Dread_chunk() signature has changed
|
|
|
|
A new parameter, nalloc, has been added to H5Dread_chunk(). This parameter
|
|
contains a pointer to a variable that holds the size of the buffer buf. If
|
|
*nalloc is not large enough to hold the entire chunk being read, no data
|
|
is read. On exit, the value of this variable is set to the buffer size
|
|
needed to read the chunk.
|
|
|
|
The old signature has been renamed to H5Dread_chunk1() and is considered
|
|
deprecated:
|
|
|
|
herr_t H5Dread_chunk1(hid_t dset_id, hid_t dxpl_id,
|
|
const hsize_t *offset, uint32_t *filters,
|
|
void *buf);
|
|
|
|
The new signature is H5Dread_chunk2(). All code should be updated to use
|
|
this version:
|
|
|
|
herr_t H5Dread_chunk2(hid_t dset_id, hid_t dxpl_id,
|
|
const hsize_t *offset, uint32_t *filters,
|
|
void *buf, size_t *nalloc);
|
|
|
|
H5Dread_chunk() will map to the new signature unless the library is
|
|
explicitly configured to use an older version of the API.
|
|
|
|
- Replaced the ROS3 VFD's S3 backend based on libcurl with a new backend
|
|
based on the aws-c-s3 library (https://github.com/awslabs/aws-c-s3)
|
|
|
|
The ROS3 VFD now requires the aws-c-s3 library in order to be built.
|
|
This library offers several useful features for the VFD, including:
|
|
|
|
- Automatic retries of non-fatal failed requests (where the libcurl
|
|
backend would simply return an error)
|
|
- Built-in sourcing of credentials and other configuration information
|
|
from standard AWS configuration files and environment variables
|
|
- Automatic splitting and parallelization of large S3 requests
|
|
- Built-in handling of the formation of HTTP request authentication
|
|
headers (where the libcurl backend had to manage this manually)
|
|
|
|
and more. Additional benefits with the rewrite of this backend include:
|
|
|
|
- Support for s3:// object URIs
|
|
- The ability to enable debugging information printouts at runtime
|
|
by setting a new environment variable HDF5_ROS3_VFD_DEBUG (refer
|
|
to H5FDros3.h)
|
|
- The ability to capture logging information from the aws-c-s3 library
|
|
by setting a new environment variable HDF5_ROS3_VFD_LOG_LEVEL
|
|
(refer to H5FDros3.h)
|
|
|
|
With these changes, the behavior of the 'authenticate' field of the
|
|
ROS3 VFD's FAPL structure has changed slightly. If 'authenticate' is
|
|
true, the ROS3 VFD will ONLY use credentials specified in the FAPL
|
|
structure and will not attempt to load credentials from other places.
|
|
In this case, the 'secret_id' and 'secret_key' fields must still both
|
|
be non-empty strings, or an error will be returned when opening a file.
|
|
If a session token is to be used, it must be specified with
|
|
H5Pset_fapl_ros3_token().
|
|
|
|
If 'authenticate' is false, the ROS3 VFD will instead attempt to load
|
|
credentials from several different places, in this order:
|
|
|
|
- From the environment, by checking AWS environment variables such
|
|
as AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN
|
|
and AWS_ACCOUNT_ID
|
|
- From the AWS profile files, by reading from ~/.aws/config and
|
|
~/.aws/credentials, by default. The specific files read from can
|
|
be overridden with the AWS_CONFIG_FILE and AWS_SHARED_CREDENTIALS_FILE
|
|
environment variables.
|
|
- From STS, by using AssumeRoleWithWebIdentity
|
|
- From EC2 instance metadata
|
|
|
|
If the ROS3 VFD cannot source credentials from any of these locations,
|
|
it will fallback to using anonymous credentials.
|
|
|
|
This functionality effectively deprecates the --s3-cred option for the
|
|
h5dump, h5ls and h5stat tools. However, this option has been kept for
|
|
compatibility reasons and can still be used to force the specified
|
|
credentials to take precedence over any credentials that the VFD would
|
|
otherwise try to source.
|
|
|
|
Note that with these changes the AWS region to be used must always be
|
|
specified. However, the region can now be specified in several ways other
|
|
than just in the FAPL structure. The VFD will search for a specified AWS
|
|
region in the following order:
|
|
|
|
- The FAPL, if `aws_region` is not an empty string
|
|
- The AWS_REGION environment variable
|
|
- The AWS_DEFAULT_REGION environment variable
|
|
- The AWS configuration file (~/.aws/config by default)
|
|
- The 'default' profile from this file is used, unless a different
|
|
profile is specified with the AWS_PROFILE environment variable
|
|
|
|
If the ROS3 VFD cannot determine an AWS region from one of these locations,
|
|
an error will be returned when opening a file.
|
|
|
|
New API functions H5Pset_fapl_ros3_endpoint() and H5Pget_fapl_ros3_endpoint()
|
|
have been added for use with the ROS3 VFD. These functions set/get an
|
|
alternate endpoint URL to use when opening files with the ROS3 VFD. This
|
|
is useful in cases where the application needs to access files that are
|
|
in a location other than the standard s3.<region-code>.amazonaws.com, which
|
|
is what the ROS3 VFD uses when an alternate endpoint URL isn't specified.
|
|
The ROS3 VFD also checks the AWS_ENDPOINT_URL_S3 and AWS_ENDPOINT_URL
|
|
environment variables for an alternate endpoint URL if one isn't specified
|
|
with H5Pset_fapl_ros3_endpoint().
|
|
|
|
Instructions for building the ROS3 VFD with the aws-c-s3 library have been
|
|
added to release_docs/INSTALL_S3.txt. The ROS3 VFD and information about
|
|
usage of the driver is described in the HDF5 user's guide.
|
|
|
|
- Renamed some API decorations
|
|
|
|
Some API decorations (used to hide __declspec on Windows, among other
|
|
things) have been renamed:
|
|
|
|
H5_DLLCPP(VAR) --> H5CPP_DLL(VAR)
|
|
H5_HLDLL(VAR) --> H5HL_DLL(VAR)
|
|
H5_HLCPPDLL(VAR) --> H5CPP_HL_DLL(VAR)
|
|
H5_FCDLL(VAR) --> H5FC_DLL(VAR)
|
|
H5_FCTESTDLL(VAR) --> H5FC_TEST_DLL(VAR)
|
|
HDF5_HL_F90CSTUBDLL(VAR) --> H5FC_HL_DLL(VAR)
|
|
|
|
|
|
- The H5Iregister_type() signature has changed
|
|
|
|
The hash_size parameter has not been used since early versions of HDF5
|
|
1.8, so it has been removed and the API call has been versioned.
|
|
|
|
The old signature has been renamed to H5Iregister_type1() and is considered
|
|
deprecated:
|
|
|
|
H5I_type_t H5Iregister_type1(size_t hash_size, unsigned reserved, H5I_free_t free_func);
|
|
|
|
The new signature is H5Iregister_type2(). New code should use this
|
|
version:
|
|
|
|
H5I_type_t H5Iregister_type2(unsigned reserved, H5I_free_t free_func);
|
|
|
|
H5Iregister_type() will map to the new signature unless the library is
|
|
explicitly configured to use an older version of the API.
|
|
|
|
- The H5Tdecode() signature has changed
|
|
|
|
When provided malformed or too-small buffers, H5Tdecode() would crash.
|
|
The new buffer size parameter allows this to be reliably avoided.
|
|
|
|
The old signature has been renamed to H5Tdecode1() and is considered
|
|
deprecated:
|
|
|
|
hid_t H5Tdecode1(const void *buf);
|
|
|
|
The new signature is H5Tdecode2(). New code should use this version:
|
|
|
|
hid_t H5Tdecode2(const void *buf, size_t buf_size);
|
|
|
|
H5Tdecode() will map to the new signature unless the library is
|
|
explicitly configured to use an older version of the API.
|
|
|
|
- H5F_LIBVER_LATEST is now an enum value
|
|
|
|
This was previously #defined to the latest H5F_libver_t API version, but
|
|
is now an enum value with an integer value equal to the latest H5F_libver_t
|
|
API version's value. e.g.:
|
|
|
|
<snip>
|
|
H5F_LIBVER_V200 = 5,
|
|
H5F_LIBVER_LATEST = 5,
|
|
</snip>
|
|
|
|
- Added support for complex number datatypes
|
|
|
|
Support for the C99 "float _Complex", "double _Complex" and "long double _Complex"
|
|
(with MSVC, "_Fcomplex", "_Dcomplex" and "_Lcomplex") types has been added for
|
|
platforms/compilers that support them. These types have been implemented with a
|
|
new datatype class, H5T_COMPLEX. Note that any datatypes of class H5T_COMPLEX
|
|
will not be readable with previous versions of HDF5. If a file is accessed with
|
|
a library version bounds "high" setting less than H5F_LIBVER_V200, an error will
|
|
occur if the application tries to create an object with a complex number datatype.
|
|
If compatibility with previous versions of HDF5 is desired, applications should
|
|
instead consider adopting one of the existing conventions at
|
|
https://nc-complex.readthedocs.io/en/latest/#conventions-used-in-applications.
|
|
|
|
The following new macros have been added:
|
|
|
|
- H5_HAVE_COMPLEX_NUMBERS
|
|
|
|
This macro is defined in H5pubconf.h and will have the value 1 if native
|
|
support for complex numbers is available. It will not be defined otherwise.
|
|
|
|
- H5_HAVE_C99_COMPLEX_NUMBERS
|
|
|
|
This macro is defined in H5pubconf.h and will have the value 1 if native
|
|
support for C99 complex numbers is available. It will not be defined otherwise.
|
|
If this macro is not defined but H5_HAVE_COMPLEX_NUMBERS is defined, the
|
|
complex number types supported are the MSVC types.
|
|
|
|
- H5_SIZEOF_FLOAT_COMPLEX
|
|
|
|
This macro is defined in H5pubconf.h and will have a value corresponding
|
|
to the size of the native float complex datatype, as computed by sizeof().
|
|
If C99 complex number support is available, this will be the size of the
|
|
"float _Complex" type. Otherwise, it will be the size of the "_Fcomplex"
|
|
type. It will have the value 0 if support for a native float complex datatype
|
|
is not available.
|
|
|
|
- H5_SIZEOF_DOUBLE_COMPLEX
|
|
|
|
This macro is defined in H5pubconf.h and will have a value corresponding
|
|
to the size of the native double complex datatype, as computed by sizeof().
|
|
If C99 complex number support is available, this will be the size of the
|
|
"double _Complex" type. Otherwise, it will be the size of the "_Dcomplex"
|
|
type. It will have the value 0 if support for a native double complex datatype
|
|
is not available.
|
|
|
|
- H5_SIZEOF_LONG_DOUBLE_COMPLEX
|
|
|
|
This macro is defined in H5pubconf.h and will have a value corresponding
|
|
to the size of the native long double complex datatype, as computed by sizeof().
|
|
If C99 complex number support is available, this will be the size of the
|
|
"long double _Complex" type. Otherwise, it will be the size of the "_Lcomplex"
|
|
type. It will have the value 0 if support for a native long double complex
|
|
datatype is not available.
|
|
|
|
- H5T_NATIVE_FLOAT_COMPLEX
|
|
|
|
This macro maps to the ID of an HDF5 datatype representing the native C
|
|
float complex datatype (either "float _Complex" or "_Fcomplex") for the
|
|
platform. If support for a native float complex datatype is not available
|
|
(H5_HAVE_COMPLEX_NUMBERS is not defined), the macro will map to
|
|
H5I_INVALID_HID and should not be used.
|
|
|
|
- H5T_NATIVE_DOUBLE_COMPLEX
|
|
|
|
This macro maps to the ID of an HDF5 datatype representing the native C
|
|
double complex datatype (either "double _Complex" or "_Dcomplex") for the
|
|
platform. If support for a native double complex datatype is not available
|
|
(H5_HAVE_COMPLEX_NUMBERS is not defined), the macro will map to
|
|
H5I_INVALID_HID and should not be used.
|
|
|
|
- H5T_NATIVE_LDOUBLE_COMPLEX
|
|
|
|
This macro maps to the ID of an HDF5 datatype representing the native C
|
|
long double complex datatype (either "long double _Complex" or "_Lcomplex")
|
|
for the platform. If support for a native long double complex datatype is
|
|
not available (H5_HAVE_COMPLEX_NUMBERS is not defined), the macro will map
|
|
to H5I_INVALID_HID and should not be used.
|
|
|
|
- H5T_COMPLEX_IEEE_F16LE / H5T_COMPLEX_IEEE_F16BE
|
|
|
|
These macros map to IDs of HDF5 datatypes representing a complex number of
|
|
two parts, each of which is an IEEE 754 16-bit floating-point datatype in
|
|
little- or big-endian order. These datatypes are available regardless of
|
|
whether complex number support is available or not.
|
|
|
|
- H5T_COMPLEX_IEEE_F32LE / H5T_COMPLEX_IEEE_F32BE
|
|
|
|
These macros map to IDs of HDF5 datatypes representing a complex number of
|
|
two parts, each of which is an IEEE 754 32-bit floating-point datatype in
|
|
little- or big-endian order. These datatypes are available regardless of
|
|
whether complex number support is available or not.
|
|
|
|
- H5T_COMPLEX_IEEE_F64LE / H5T_COMPLEX_IEEE_F64BE
|
|
|
|
These macros map to IDs of HDF5 datatypes representing a complex number of
|
|
two parts, each of which is an IEEE 754 64-bit floating-point datatype in
|
|
little- or big-endian order. These datatypes are available regardless of
|
|
whether complex number support is available or not.
|
|
|
|
The following new API function has been added:
|
|
|
|
hid_t H5Tcomplex_create(hid_t base_type_id);
|
|
|
|
Creates a new complex number datatype from the base datatype specified
|
|
by the given HDF5 ID `base_type_id`. The base datatype must be a
|
|
floating-point datatype.
|
|
|
|
The following new hard datatype conversion paths have been added, but
|
|
will only be used when complex number support is available:
|
|
|
|
H5T_NATIVE_SCHAR <-> H5T_NATIVE_FLOAT_COMPLEX | H5T_NATIVE_UCHAR <-> H5T_NATIVE_FLOAT_COMPLEX
|
|
H5T_NATIVE_SHORT <-> H5T_NATIVE_FLOAT_COMPLEX | H5T_NATIVE_USHORT <-> H5T_NATIVE_FLOAT_COMPLEX
|
|
H5T_NATIVE_INT <-> H5T_NATIVE_FLOAT_COMPLEX | H5T_NATIVE_UINT <-> H5T_NATIVE_FLOAT_COMPLEX
|
|
H5T_NATIVE_LONG <-> H5T_NATIVE_FLOAT_COMPLEX | H5T_NATIVE_ULONG <-> H5T_NATIVE_FLOAT_COMPLEX
|
|
H5T_NATIVE_LLONG <-> H5T_NATIVE_FLOAT_COMPLEX | H5T_NATIVE_ULLONG <-> H5T_NATIVE_FLOAT_COMPLEX
|
|
H5T_NATIVE_FLOAT16 <-> H5T_NATIVE_FLOAT_COMPLEX | H5T_NATIVE_FLOAT <-> H5T_NATIVE_FLOAT_COMPLEX
|
|
H5T_NATIVE_DOUBLE <-> H5T_NATIVE_FLOAT_COMPLEX | H5T_NATIVE_LDOUBLE <-> H5T_NATIVE_FLOAT_COMPLEX
|
|
|
|
H5T_NATIVE_SCHAR <-> H5T_NATIVE_DOUBLE_COMPLEX | H5T_NATIVE_UCHAR <-> H5T_NATIVE_DOUBLE_COMPLEX
|
|
H5T_NATIVE_SHORT <-> H5T_NATIVE_DOUBLE_COMPLEX | H5T_NATIVE_USHORT <-> H5T_NATIVE_DOUBLE_COMPLEX
|
|
H5T_NATIVE_INT <-> H5T_NATIVE_DOUBLE_COMPLEX | H5T_NATIVE_UINT <-> H5T_NATIVE_DOUBLE_COMPLEX
|
|
H5T_NATIVE_LONG <-> H5T_NATIVE_DOUBLE_COMPLEX | H5T_NATIVE_ULONG <-> H5T_NATIVE_DOUBLE_COMPLEX
|
|
H5T_NATIVE_LLONG <-> H5T_NATIVE_DOUBLE_COMPLEX | H5T_NATIVE_ULLONG <-> H5T_NATIVE_DOUBLE_COMPLEX
|
|
H5T_NATIVE_FLOAT16 <-> H5T_NATIVE_DOUBLE_COMPLEX | H5T_NATIVE_FLOAT <-> H5T_NATIVE_DOUBLE_COMPLEX
|
|
H5T_NATIVE_DOUBLE <-> H5T_NATIVE_DOUBLE_COMPLEX | H5T_NATIVE_LDOUBLE <-> H5T_NATIVE_DOUBLE_COMPLEX
|
|
|
|
H5T_NATIVE_SCHAR <-> H5T_NATIVE_LDOUBLE_COMPLEX | H5T_NATIVE_UCHAR <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
H5T_NATIVE_SHORT <-> H5T_NATIVE_LDOUBLE_COMPLEX | H5T_NATIVE_USHORT <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
H5T_NATIVE_INT <-> H5T_NATIVE_LDOUBLE_COMPLEX | H5T_NATIVE_UINT <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
H5T_NATIVE_LONG <-> H5T_NATIVE_LDOUBLE_COMPLEX | H5T_NATIVE_ULONG <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
H5T_NATIVE_LLONG <-> H5T_NATIVE_LDOUBLE_COMPLEX | H5T_NATIVE_ULLONG <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
H5T_NATIVE_FLOAT16 <-> H5T_NATIVE_LDOUBLE_COMPLEX | H5T_NATIVE_FLOAT <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
H5T_NATIVE_DOUBLE <-> H5T_NATIVE_LDOUBLE_COMPLEX | H5T_NATIVE_LDOUBLE <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
|
|
H5T_NATIVE_FLOAT_COMPLEX <-> H5T_NATIVE_DOUBLE_COMPLEX
|
|
H5T_NATIVE_FLOAT_COMPLEX <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
H5T_NATIVE_DOUBLE_COMPLEX <-> H5T_NATIVE_LDOUBLE_COMPLEX
|
|
|
|
Alternative software implementation conversion paths have been added for all of
|
|
the above for use when native complex number support is not available. All of these
|
|
conversion paths follow the behavior outlined in the C standard for conversions of
|
|
complex number values.
|
|
|
|
Additionally, a special datatype conversion path has been added between complex number
|
|
datatypes and array or compound datatypes where the in-memory layout of data is the
|
|
same between the datatypes and data can be directly converted. This conversion path
|
|
is subject to the following rules:
|
|
|
|
- An array datatype must consist of exactly two elements where each element is of
|
|
the same floating-point datatype as the complex number datatype's base floating-point
|
|
datatype.
|
|
|
|
- A compound datatype must consist of exactly two fields where each field is of the
|
|
same floating-point datatype as the complex number datatype's base floating-point
|
|
datatype. The compound datatype must not have any leading or trailing structure
|
|
padding or any padding between its two fields. The fields must also have compatible
|
|
names, must have compatible offsets within the datatype and must be in the order
|
|
of "real" part -> "imaginary" part, such that the compound datatype matches the
|
|
following representation:
|
|
|
|
H5T_COMPOUND {
|
|
<float_type> "r(e)(a)(l)"; OFFSET 0
|
|
<float_type> "i(m)(a)(g)(i)(n)(a)(r)(y)"; OFFSET SIZEOF("r(e)(a)(l)")
|
|
}
|
|
|
|
where "r(e)(a)(l)" means the field may be named any substring of "real", such as
|
|
"r", or "re" and "i(m)(a)(g)(i)(n)(a)(r)(y)" means the field may be named any
|
|
substring of "imaginary", such as "im" or "imag".
|
|
|
|
Support for complex numbers has been added to the h5dump, h5ls and h5diff/ph5diff
|
|
tools. The h5dump command-line option '-m' can be used to change the floating-point
|
|
printing format for the float complex and double complex datatypes, as well as
|
|
the long double complex datatype if it has the same size as a double complex
|
|
datatype.
|
|
|
|
Support for the predefined complex number datatypes and the H5Tcomplex_create
|
|
function has been added to the Java wrappers. However, Java does not have
|
|
official types for complex numbers, so an application must be sure that
|
|
data is in an appropriate format in-memory when using these datatypes.
|
|
|
|
Support for the Fortran wrappers has not yet been added.
|
|
|
|
Support for the predefined complex number datatypes and the H5Tcomplex_create
|
|
function has been added to the high level library, allowing them to work
|
|
with the H5LTtext_to_dtype and H5LTdtype_to_text functions.
|
|
|
|
Simple example programs showing how to use complex number datatypes have
|
|
been added in the following files:
|
|
|
|
- HDF5Examples/C/H5T/200/h5ex_t_complex.c (Uses C99 complex number types)
|
|
- HDF5Examples/C/H5T/200/h5ex_t_complex_msvc.c (Uses MSVC complex number types)
|
|
- HDF5Examples/C/H5T/200/h5ex_t_complex_custom.c (Uses H5Tcomplex_create to create
|
|
a custom complex number type)
|
|
|
|
- FOR VOL DEVELOPERS: Renamed H5VLstart_lib_state and H5VLfinish_lib_state
|
|
|
|
The APIs H5VLstart_lib_state and H5VLfinish_lib_state have been renamed to
|
|
H5VLopen_lib_context and H5VLclose_lib_context, respectively, with the addition
|
|
of a "context" argument.
|
|
|
|
- Removed H5FDperform_init API routine. Virtual File Driver (VFD)
|
|
developers who wish to provide an ID for their driver should create
|
|
a routine specific to their individual implementation.
|
|
|
|
- H5Pset_external() now uses HDoff_t, which is always a 64-bit type
|
|
|
|
The H5Pset_external() call took an off_t parameter in HDF5 1.14.x and
|
|
earlier. On POSIX systems, off_t is specified as a 64-bit type via
|
|
POSIX large-file support (LFS). On Windows, however, off_t is defined
|
|
as a 32-bit type, even on 64-bit Windows.
|
|
|
|
HDoff_t has been added to H5public.h and is defined to be int64_t on
|
|
Windows and the library has been updated to use HDoff_t in place of
|
|
off_t throughout. The H5Pset_external() offset parameter has also been
|
|
updated to be HDoff_t.
|
|
|
|
There is no API compatibility wrapper for this change.
|
|
|
|
Fixes GitHub issue #3506
|
|
|
|
- H5Pset* routines now fail when used on default property lists
|
|
|
|
Modifying default property lists was never fully supported and could produce
|
|
inconsistent and unexpected behavior.
|
|
|
|
- H5Pset_vol() now fails when used on a non-file-access property list
|
|
|
|
Similar to the above. Setting the connector on a non-FAPL had no effect on
|
|
library behavior, and the connector ID and information could not be read back
|
|
from that plist later.
|
|
|
|
Parallel Library:
|
|
-----------------
|
|
-
|
|
|
|
|
|
Fortran Library:
|
|
----------------
|
|
-
|
|
|
|
|
|
C++ Library:
|
|
------------
|
|
-
|
|
|
|
|
|
Java Library:
|
|
-------------
|
|
-
|
|
|
|
|
|
Tools:
|
|
------
|
|
- Added AWS endpoint command option to allow specifying an alternate endpoint
|
|
URL when using the ROS3 VFD
|
|
|
|
The new option is --endpoint-url, which allows the user to set an alternate
|
|
endpoint URL than the standard "protocol://service-code.region-code.amazonaws.com".
|
|
If "--endpoint-url" is not specified, the ROS3 VFD will first check the
|
|
AWS_ENDPOINT_URL_S3 and AWS_ENDPOINT_URL environment variables for an alternate
|
|
endpoint URL before using a default one, with the region-code being supplied by
|
|
the FAPL or standard AWS locations/environment variables.
|
|
|
|
This option is supported by the following tools:
|
|
h5dump, h5ls, h5stat
|
|
|
|
- Specifying ROS3 VFD on the command line not required when using S3 URI
|
|
|
|
If using an S3 URI to reference HDF5 file in S3 (example:
|
|
s3://mybucket/myfile.h5) then ROS3 VFD will be automatically selected
|
|
unless the command-line option for the virtual file driver is used.
|
|
|
|
This feature applies to the following tools: h5dump, h5ls, h5stat.
|
|
|
|
- Deprecated h5dump XML option
|
|
|
|
The h5dump XML option is deprecated and will be removed in a future
|
|
release. The XML output format has not been maintained and is not
|
|
up-to-date with the latest features of HDF5.
|
|
|
|
- Added h5dump command option to set the floating point format for long double
|
|
|
|
The new option is --lformat, which allows the user to set the
|
|
floating point format for long double. The default format is %Lg.
|
|
There is already an option --format to set the floating point format
|
|
for double and float. The default format is %g.
|
|
|
|
- Remove the high-level GIF tools
|
|
|
|
The high-level GIF tools, h52gif and gif2h5, have unfixed CVE issues
|
|
(with no proof-of-concept files). They are not critical tools, are not
|
|
well maintained, and are an odd fit for building with the library.
|
|
Because of this, they have been removed. We may move them to a separate
|
|
repository in the future.
|
|
|
|
This also removes the following configure options:
|
|
CMake: HDF5_BUILD_HL_GIF_TOOLS
|
|
|
|
High-Level APIs:
|
|
----------------
|
|
-
|
|
|
|
|
|
C Packet Table API:
|
|
-------------------
|
|
-
|
|
|
|
|
|
Internal header file:
|
|
---------------------
|
|
-
|
|
|
|
|
|
Documentation:
|
|
--------------
|
|
- The COPYING file has been renamed to LICENSE
|
|
|
|
This is where most people will expect to find license information. The
|
|
COPYING_LBNL_HDF5 file has also been renamed to LICENSE_LBNL_HDF5.
|
|
The licenses are unchanged.
|
|
|
|
|
|
Support for new platforms, languages and compilers
|
|
==================================================
|
|
-
|
|
|
|
Bug Fixes since HDF5-2.0.0 release
|
|
===================================
|
|
Library
|
|
-------
|
|
- Revised handling of Unicode filenames on Windows
|
|
|
|
In the HDF5 1.14.4 release, a change was made to address some issues
|
|
with the library's handling of code pages and file paths on Windows.
|
|
This change introduced other issues with the handling of UTF-8 file
|
|
names that caused breakage for software using the 1.14.4 and 1.14.5
|
|
releases of HDF5. That change was reverted for the 1.14.6 release
|
|
and the behavior has been slightly modified for this release.
|
|
|
|
On Windows, the library once again assumes that filename strings will
|
|
be UTF-8 encoded strings and will attempt to convert them to UTF-16
|
|
before passing them to Windows API functions. However, if the library
|
|
fails to convert a filename string to UTF-16, it will now fallback to
|
|
the equivalent Windows "ANSI" API functions which will interpret the
|
|
string according to the active Windows code page.
|
|
|
|
Support for a new environment variable, HDF5_PREFER_WINDOWS_CODE_PAGE,
|
|
was added in order to instruct HDF5 to prefer interpreting filenames
|
|
according to the active Windows code page rather than assuming UTF-8
|
|
encoding. If this environment variable is set to "1" or "TRUE"
|
|
(case-insensitive), the active code page will be preferred. If it is
|
|
unset or set to "0" or "FALSE" (case-insensitive), UTF-8 will be
|
|
preferred.
|
|
|
|
- Fixed an issue with caching in the ROS3 VFD
|
|
|
|
The ROS3 VFD uses a very simple caching mechanism that caches the
|
|
first 16MiB of a file during file open and serves later reads from
|
|
that cache if the offset + length falls within the cached range of
|
|
bytes. Combinations of offset + length that extended exactly to the
|
|
end of the cached range of bytes (for example, offset=0 and
|
|
len=16777216) would end up not being served from the cache due to
|
|
an incorrect range check. This has now been fixed.
|
|
|
|
- Fixed an error with H5Fget_file_image() with the latest file format
|
|
|
|
When using H5Fget_file_image() on a file created with the latest file
|
|
format (or any format newer than the earliest), the library failed to
|
|
recalculate the superblock checksum after changing the access flags in the
|
|
superblock, causing any subsequent attempt to open the returned file image
|
|
to fail due to the checksum failing to verify. Fixed H5Fget_file_image()
|
|
to recalculate the checksum.
|
|
|
|
Fixes GitHub issue #1915
|
|
|
|
- Fixed an assertion failure in H5S__hyper_make_spans()
|
|
|
|
Calling H5Sselect_hyperslab() on dataspaces with invalid extents could
|
|
result in an assertion failure in debug builds of the library if the
|
|
dataspace has an extent with a rank value of 0. This has been fixed by
|
|
converting the assertion failure into a normal error check.
|
|
|
|
- Fixed an assertion failure in H5S__hyper_new_span_info()
|
|
|
|
Calling H5Scopy() on hyperslab selection dataspaces with invalid extents
|
|
could result in an assertion failure in debug builds of the library if
|
|
the dataspace has an extent with a rank value of 0. This has been fixed
|
|
by converting the assertion failure into a normal error check.
|
|
|
|
- Fixed a segfault in H5S__get_select_hyper_blocklist()
|
|
|
|
When attempting to retrieve the list of hyperslab blocks selected
|
|
within a dataspace, a segfault or bus error could occur when the
|
|
dataspace has an extent with a rank value of 0. This would cause
|
|
indexing into an array variable on the stack using a negative
|
|
value. An error check was added to return failure from the function
|
|
for such dataspaces.
|
|
|
|
- Fixed an error in H5Ddebug
|
|
|
|
H5Ddebug would fail for any chunked dataset with a chunk index, due to its
|
|
failure to tag the dataset before performing metadata operations. This
|
|
caused h5ls -va to silently fail to print chunk addresses. This has been
|
|
fixed.
|
|
|
|
- Fixed a bug in the H5Oexists and H5Oexists_by_name API routines that
|
|
would cause those routines to return FAIL instead of FALSE when checking
|
|
the existence of a non-existent object with a file ID instead of a
|
|
group ID.
|
|
|
|
- Fixed a segfault in h5dump when a B-tree node level is corrupted
|
|
|
|
h5dump produced a segfault on a mal-formed file because a B-tree node
|
|
level was corrupted.
|
|
|
|
An internal function was modified to help detecting when a decoded B-tree
|
|
node level has an unexpected value and an error will be produced.
|
|
|
|
Fixes GitHub issue #4432
|
|
|
|
- Fixed H5Ovisit2 to recursively visit all objects
|
|
|
|
H5Ovisit2 visited only the root group and not all the nested groups.
|
|
|
|
This behavior occurred when the fields are not H5O_INFO_BASIC or
|
|
H5O_INFO_ALL because an internal function did not obtain the basic
|
|
information needed by its caller. This problem is now fixed.
|
|
|
|
Fixes GitHub issue #4941
|
|
|
|
- Only clear FE_INVALID when that symbol is present on the system
|
|
|
|
When we initialize the floating-point types at library startup, it's
|
|
possible to raise floating-point exceptions when we check which things
|
|
are supported. Normally, we clear these floating-point exceptions via
|
|
feclearexcept(FE_INVALID), but FE_INVALID may not be present on all
|
|
systems. Specifically, this was reported as being a problem when using
|
|
Emscripten 3.1.68 to compile HDF5 1.14.5 to WebAssembly.
|
|
|
|
We've added an #ifdef FE_INVALID block around the exception clearing
|
|
code to correct this.
|
|
|
|
Fixes GitHub issue #4952
|
|
|
|
- Check for overflow in decoded heap block addresses
|
|
|
|
Currently, we do not check for overflow when decoding addresses from
|
|
the heap, which can cause overflow problems. We've added a check in
|
|
H5HL__fl_deserialize to ensure no overflow can occur.
|
|
|
|
Fixes GitHub issue #5382
|
|
|
|
Java Library
|
|
------------
|
|
- Renamed the Callbacks.java file to H5Callbacks.java
|
|
|
|
The Callbacks.java file was renamed to H5Callbacks.java to match the file
|
|
pattern used by doxygen. This change only affects the Java filenames and
|
|
does not change the classname or the package name.
|
|
|
|
|
|
Configuration
|
|
-------------
|
|
- Reorganized CMake HDF5 configuration options
|
|
|
|
The CMake configuration options have been reorganized to identify the
|
|
primary options that are relevant to the build. These options are now
|
|
in a separate file, CMakeBuildOptions.cmake, which is included by the
|
|
root CMakeLists.txt file. In addition some options have been converted
|
|
to cmake_dependent_option() calls, which allows the options to be hidden
|
|
from the CMake GUI when they are not relevant to the build.
|
|
|
|
- Remove default setting of CMAKE_DEBUG_POSTFIX
|
|
|
|
Move the default setting of CMAKE_DEBUG_POSTFIX to the cacheinit.cmake file
|
|
usually used by testing. If CMAKE_DEBUG_POSTFIX is not set with a -D option
|
|
then CMAKE_DEBUG_POSTFIX will be the default provided by CMake itself.
|
|
|
|
- The relative rpaths ($ORIGIN / @loader_path) are appended to the CMAKE_INSTALL_RPATH
|
|
|
|
The RPATH settings were removed by a pull-request #5271, but the settings are needed
|
|
under certain conditions. These settings have been restored by appending the necessary paths
|
|
and will not override/overwrite any existing settings.
|
|
|
|
- When using a system installed zlib library, the shared library is expected to
|
|
be found in the system library path.
|
|
|
|
Setting the HDF5_MODULE_MODE_ZLIB option to OFF will force find_package to
|
|
use config mode first. An installed zlib, or an alternate installed zlib
|
|
library, is expected to have a correct zlib-config.cmake file for config mode.
|
|
Current zlib installs usually do not have a zlib-config.cmake file, so the
|
|
option is set to ON by default.
|
|
|
|
- Use pre-installed libaec compression library
|
|
|
|
The CMake logic for finding the libaec compression library has been
|
|
modified for a system-installed version of the library. Two options
|
|
must be set;
|
|
HDF5_ALLOW_EXTERNAL_SUPPORT:STRING=NO
|
|
<LIB_PKG_NAME>_USE_EXTERNAL:BOOL=OFF
|
|
where <LIB_PKG_NAME> is one of ZLIB, ZLIBNG, SZIP, PLUGIN.
|
|
Note that HDF5_ALLOW_EXTERNAL_SUPPORT:STRING=NO disables building all plugins
|
|
and external libraries in-line with the HDF5 library.
|
|
|
|
In addition, the <LIB_PKG_NAME>_ROOT environment variables must be set,
|
|
where <LIB_PKG_NAME> is one of ZLIB, ZLIBNG, SZIP, libaec, PLUGIN.
|
|
Note that libaec is the expected name for using the libaec library in place of original szip.
|
|
|
|
See INSTALL_CMake.txt for more detailed information.
|
|
|
|
- Changed the zlib/szip compression find message to FATAL ERROR
|
|
|
|
The message was changed to indicate that zlib/szip compression was requested and
|
|
that it was not found. If an option is requested, not finding it should always
|
|
be an error.
|
|
|
|
- Removed the module search find_package for szip library
|
|
|
|
There is not a szip module file to use, so the find_package only uses
|
|
find_package in config mode. The choice then is to either build szip, with libaec,
|
|
inline or find a system installed szip library, built with CMake.
|
|
|
|
|
|
Tools
|
|
-----
|
|
- h5repack did not properly parse User Defined filters
|
|
|
|
The h5repack tool did not properly parse user-defined filter command-line
|
|
arguments when the number of elements value was 0 (zero).
|
|
Also, using a colon without a preceding object was enforced to behave the
|
|
same as not using a colon.
|
|
|
|
Fixes GitHub issue #5132
|
|
|
|
- Changed the default value for number of cd_values in filters.
|
|
|
|
The tools used an arbitrary value 0f 20 for the number of cd_values used in a filter.
|
|
Created a new define DEFAULT_CDELEMTS in H5tools.h for the default value, which
|
|
currently matches the library restriction of 256.
|
|
|
|
Fixes GitHub issue #5414
|
|
|
|
|
|
Performance
|
|
-------------
|
|
-
|
|
|
|
|
|
Fortran API
|
|
-----------
|
|
-
|
|
|
|
|
|
High-Level Library
|
|
------------------
|
|
- Fixed an issue with H5TB functions
|
|
|
|
The H5TB functions were not correctly creating the FILL_INFO attribute
|
|
for tables. This has been fixed by using the field offsets array from
|
|
the call to H5TBAget_fill instead of using the compound type member
|
|
offsets from the H5Tget_member_offset call for each compound type.
|
|
|
|
|
|
Fortran High-Level APIs
|
|
-----------------------
|
|
-
|
|
|
|
|
|
Documentation
|
|
-------------
|
|
-
|
|
|
|
|
|
F90 APIs
|
|
--------
|
|
-
|
|
|
|
|
|
C++ APIs
|
|
--------
|
|
-
|
|
|
|
|
|
Testing
|
|
-------
|
|
- Fixed an allocation in the t_bigio parallel test on 32-bit systems
|
|
|
|
A test in t_bigio.c attempts to allocate more than 4 GiB of memory,
|
|
which will overflow (and wrap) the size_t type on 32-bit systems,
|
|
creating a very small allocation instead of a very large allocation.
|
|
The test then segfaults when it accesses memory outside of the small
|
|
buffer.
|
|
|
|
The test has been fixed by limiting the buffer to 2 GiB on 32-bit
|
|
systems.
|
|
|
|
Fixes GitHub #2510
|
|
|
|
- Added skipping of a few parallel tests for OpenMPI 5.0.5
|
|
|
|
An issue in OpenMPI 5.0.5 causes a few parallel HDF5 tests
|
|
(mpiodup, props, fapl_preserve) to fail. These tests are
|
|
now skipped for that release of OpenMPI. The issue has
|
|
been fixed in the 5.0.6 release of OpenMPI.
|
|
|
|
|
|
Platforms Tested
|
|
===================
|
|
|
|
- HDF5 is tested with the two latest macOS versions that are available
|
|
on github runners. As new major macOS versions become available, HDF5
|
|
will discontinue support for the older version and add the new latest
|
|
version to its list of compatible systems, along with the previous
|
|
version.
|
|
|
|
Linux 6.8.0-1010-aws GNU gcc, gfortran, g++
|
|
#10-Ubuntu SMP 2024 x86_64 (Ubuntu 13.2.0-23ubuntu4) 13.2.0
|
|
GNU/Linux Ubuntu 24.04 Ubuntu clang version 18.1.3 (1ubuntu1)
|
|
Intel(R) oneAPI DPC++/C++ Compiler 2024.2.0
|
|
ifx (IFX) 2024.2.0 20240602
|
|
(cmake )
|
|
|
|
Linux 6.5.0-1018-aws GNU gcc, gfortran, g++
|
|
#18-Ubuntu SMP x86_64 GNU/Linux (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
|
|
Ubuntu 22.04 Ubuntu clang version 14.0.0-1ubuntu1
|
|
Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2
|
|
ifx (IFX) 2024.0.2 20231213
|
|
(cmake)
|
|
|
|
Linux 5.14.21-cray_shasta_c cray-mpich/8.1.28
|
|
#1 SMP x86_64 GNU/Linux cce/15.0.0
|
|
(frontier) gcc/13.2
|
|
(cmake)
|
|
|
|
Linux 5.14.0-427.24.1.el9_4 GNU gcc, gfortran, g++ (Red Hat 11.4.1-3)
|
|
#1 SMP x86_64 GNU/Linux clang version 17.0.6
|
|
Rocky 9 Intel(R) oneAPI DPC++/C++ Compiler 2024.2.0
|
|
ifx (IFX) 2024.2.0
|
|
(cmake)
|
|
|
|
Linux-4.18.0-553.16.1.1toss.t4 openmpi/4.1.2
|
|
#1 SMP x86_64 GNU/Linux clang 14.0.6
|
|
(corona, dane) GCC 12.1.1
|
|
Intel(R) oneAPI DPC++/C++ Compiler 2023.2.1
|
|
ifx (IFX) 2023.2.1
|
|
|
|
Linux-4.18.0-553.5.1.1toss.t4 openmpi/4.1/4.1.6
|
|
#1 SMP x86_64 GNU/Linux clang 16.0.6
|
|
(eclipse) GCC 12.3.0
|
|
Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2
|
|
ifx (IFX) 2024.0.2
|
|
(cmake)
|
|
|
|
Linux 4.14.0-115.35.1.3chaos spectrum-mpi/rolling-release
|
|
#1 SMP ppc64le GNU/Linux clang 17.0.6
|
|
(vortex) GCC 12.2.1
|
|
nvhpc 24.1
|
|
XL 2023.06.28
|
|
(cmake)
|
|
|
|
Linux-4.14.0-115.35.1 spectrum-mpi/rolling-release
|
|
#1 SMP ppc64le GNU/Linux clang 14.0.5, 15.0.6
|
|
(lassen) GCC 8.3.1
|
|
XL 2021.09.22, 2022.08.05
|
|
(cmake)
|
|
|
|
Linux 3.10.0-1160.36.2.el7.ppc64 gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
|
|
#1 SMP ppc64be GNU/Linux g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
|
|
Power8 (echidna) GNU Fortran (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
|
|
|
|
Linux 3.10.0-1160.80.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
Centos7 Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
(jelly/kituo/moohan) Version 4.9.3, Version 7.2.0, Version 8.3.0,
|
|
Version 9.1.0, Version 10.2.0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.0.098 Build 20160721
|
|
GNU C (gcc) and C++ (g++) 4.8.5 compilers
|
|
with NAG Fortran Compiler Release 7.1(Hanzomon)
|
|
Intel(R) C (icc) and C++ (icpc) 17.0.0.098 compilers
|
|
with NAG Fortran Compiler Release 7.1(Hanzomon)
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
MPICH 3.3 compiled with GCC 7.2.0
|
|
OpenMPI 3.1.3 compiled with GCC 7.2.0 and 4.1.2
|
|
compiled with GCC 9.1.0
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86_64;
|
|
Versions 18.4.0 and 19.10-0
|
|
NVIDIA nvc, nvfortran and nvc++ version 22.5-0
|
|
(cmake)
|
|
|
|
|
|
Linux-3.10.0-1160.119.1.1chaos openmpi/4.1.4
|
|
#1 SMP x86_64 GNU/Linux clang 16.0.6
|
|
(skybridge) Intel(R) oneAPI DPC++/C++ Compiler 2023.2.0
|
|
ifx (IFX) 2023.2.0
|
|
(cmake)
|
|
|
|
Linux-3.10.0-1160.90.1.1chaos openmpi/4.1
|
|
#1 SMP x86_64 GNU/Linux clang 16.0.6
|
|
(attaway) GCC 12.1.0
|
|
Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2
|
|
ifx (IFX) 2024.0.2
|
|
(cmake)
|
|
|
|
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
Centos6 Version 4.4.7 20120313
|
|
(platypus) Version 4.9.3, 5.3.0, 6.2.0
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86_64;
|
|
Version 19.10-0
|
|
|
|
Windows 10 x64 Visual Studio 2019 w/ clang 12.0.0
|
|
with MSVC-like command-line (C/C++ only - cmake)
|
|
Visual Studio 2019 w/ Intel (C/C++ only - cmake)
|
|
Visual Studio 2022 w/ clang 17.0.3
|
|
with MSVC-like command-line (C/C++ only - cmake)
|
|
Visual Studio 2022 w/ Intel C/C++ oneAPI 2023 (cmake)
|
|
Visual Studio 2019 w/ MSMPI 10.1 (C only - cmake)
|
|
|
|
|
|
Known Problems
|
|
==============
|
|
|
|
- When the library detects and builds in support for the _Float16 datatype, an
|
|
issue has been observed on at least one MacOS 14 system where the library
|
|
fails to initialize due to not being able to detect the byte order of the
|
|
_Float16 type (https://github.com/HDFGroup/hdf5/issues/4310):
|
|
|
|
#5: H5Tinit_float.c line 308 in H5T__fix_order(): failed to detect byte order
|
|
major: Datatype
|
|
minor: Unable to initialize object
|
|
|
|
If this issue is encountered, support for the _Float16 type can be disabled
|
|
with a configuration option:
|
|
|
|
CMake: HDF5_ENABLE_NONSTANDARD_FEATURE_FLOAT16=OFF
|
|
|
|
- When HDF5 is compiled with NVHPC versions 23.5 - 23.9 (additional versions may
|
|
also be applicable) and with -O2 (or higher) and -DNDEBUG, test failures occur
|
|
in the following tests:
|
|
|
|
H5PLUGIN-filter_plugin
|
|
H5TEST-flush2
|
|
H5TEST-testhdf5-base
|
|
MPI_TEST_t_filters_parallel
|
|
|
|
Sporadic failures (even with lower -O levels):
|
|
Java JUnit-TestH5Pfapl
|
|
Java JUnit-TestH5D
|
|
|
|
Also, NVHPC will fail to compile the test/tselect.c test file with a compiler
|
|
error of 'use of undefined value' when the optimization level is -O2 or higher.
|
|
|
|
This is confirmed to be a bug in the nvc compiler that has been fixed as of
|
|
23.11. If you are using an affected version of the NVidia compiler, the
|
|
work-around is to set the optimization level to -O1.
|
|
|
|
https://forums.developer.nvidia.com/t/hdf5-no-longer-compiles-with-nv-23-9/269045
|
|
|
|
- CMake files do not behave correctly with paths containing spaces.
|
|
Do not use spaces in paths because the required escaping for handling spaces
|
|
results in very complex and fragile build files.
|
|
|
|
- At present, metadata cache images may not be generated by parallel
|
|
applications. Parallel applications can read files with metadata cache
|
|
images, but since this is a collective operation, a deadlock is possible
|
|
if one or more processes do not participate.
|
|
|
|
- The subsetting option in ph5diff currently will fail and should be avoided.
|
|
The subsetting option works correctly in serial h5diff.
|
|
|
|
- Flang Fortran compilation will fail (last check version 17) due to not yet
|
|
implemented: (1) derived type argument passed by value (H5VLff.F90),
|
|
and (2) support for REAL with KIND = 2 in intrinsic SPACING used in testing.
|
|
|
|
- Fortran tests HDF5_1_8.F90 and HDF5_F03.F90 will fail with Cray compilers
|
|
greater than version 16.0 due to a compiler bug. The latest version verified
|
|
as failing was version 17.0.
|
|
|
|
- Several tests currently fail on certain platforms:
|
|
MPI_TEST-t_bigio fails with spectrum-mpi on ppc64le platforms.
|
|
|
|
MPI_TEST-t_subfiling_vfd and MPI_TEST_EXAMPLES-ph5_subfiling fail with
|
|
cray-mpich on theta and with XL compilers on ppc64le platforms.
|
|
|
|
MPI_TEST_testphdf5_tldsc fails with cray-mpich 7.7 on cori and theta.
|
|
|
|
- File space may not be released when overwriting or deleting certain nested
|
|
variable length or reference types.
|
|
|
|
- Known problems in previous releases can be found in the HISTORY*.txt files
|
|
in the HDF5 source. Please report any new problems found to
|
|
help@hdfgroup.org.
|
|
|
|
|
|
CMake Installations
|
|
=================================
|
|
See the AutotoolsToCMakeOptions.md file for CMake options for former Autotools options.
|
|
CMake produces the following set of folders; bin, include, lib and share. The
|
|
LICENSE and RELEASE.txt file are placed in the share folder.
|
|
|
|
The bin folder contains the tools and the build scripts. Additionally, CMake
|
|
creates dynamic versions of the tools with the suffix "-shared".
|
|
build scripts
|
|
-------------
|
|
CMake: h5c++, h5cc, h5hlc++, h5hlcc
|
|
|
|
The include folder holds the header files and the fortran mod files. CMake
|
|
places the fortran mod files into separate shared and static subfolders,
|
|
however the use of HDF5_INSTALL_MOD_FORTRAN places one set of mod files into
|
|
the include folder. Because CMake produces a tools library, the header files
|
|
for tools will appear in the include folder.
|
|
|
|
The lib folder contains the library files, and CMake adds the pkgconfig
|
|
subfolder with the hdf5*.pc files used by the bin/build scripts created by
|
|
the CMake build. CMake separates the C interface code from the fortran code by
|
|
creating C-stub libraries for each Fortran library. In addition, only CMake
|
|
installs the tools library. The names of the szip libraries are different
|
|
between the build systems.
|
|
|
|
CMake builds include a number of CMake specific files for support of CMake's
|
|
find_package and support for the HDF5 Examples CMake project.
|
|
|
|
The issues with the gif tool are:
|
|
HDFFV-10592 CVE-2018-17433
|
|
HDFFV-10593 CVE-2018-17436
|
|
HDFFV-11048 CVE-2020-10809
|
|
These CVE issues have not yet been addressed and are avoided by not building
|
|
the gif tool by default. Enable building the High-Level tools with this options:
|
|
cmake: HDF5_BUILD_HL_GIF_TOOLS=ON
|