mirror of
https://github.com/HDFGroup/hdf5.git
synced 2026-09-25 04:09:44 +03:00
Allen Byrne
b754dcb8f2
FFM build requires Java 25, Jextract 25. Generates FFM bindings during configure. JNI is default when the requirements are not met or can be forced. Presets added for maven and FFM - JNI is default selection. Enhanced Maven options will work with either JNI or FFM New Workflows for testing and maven uploads. Extensive documentation changes for java.
189 lines
8.7 KiB
Plaintext
189 lines
8.7 KiB
Plaintext
*************************************************************************
|
|
* Build and testing instructions for HDF5's ROS3 VFD *
|
|
*************************************************************************
|
|
|
|
NOTE: HDF5 2.0 and later requires CMake for building.
|
|
See INSTALL_CMake.txt for complete build instructions using CMake Presets.
|
|
|
|
Table of Contents
|
|
|
|
Section I: Preconditions
|
|
Section II: Obtaining the aws-c-s3 library
|
|
Section III: Building the aws-c-s3 library from source
|
|
Section IV: Building HDF5 with the ROS3 VFD enabled
|
|
Section V: Testing the ROS3 VFD
|
|
|
|
************************************************************************
|
|
|
|
========================================================================
|
|
I. Preconditions
|
|
========================================================================
|
|
|
|
- Refer to release_docs/INSTALL for preconditions and instructions for
|
|
building HDF5
|
|
|
|
========================================================================
|
|
II. Obtaining the aws-c-s3 library
|
|
========================================================================
|
|
|
|
As of the HDF5 2.0.0 release (or HDF5 commit 75fff3ed0663d3617e3c5cc3c57c1f0c83b38be4),
|
|
HDF5's ROS3 VFD requires the aws-c-s3 library (https://github.com/awslabs/aws-c-s3)
|
|
in order to be built. This library is distributed by several package
|
|
managers, usually as 'aws-c-s3', but consult your particular package
|
|
manager for the name of this package if it's available. HDF5 installs
|
|
this library in the following ways in CI testing:
|
|
|
|
Linux (Ubuntu):
|
|
|
|
- from source, as Ubuntu doesn't have a package for aws-c-s3 yet
|
|
|
|
Windows:
|
|
|
|
- vcpkg install aws-c-s3 (make sure to set CMAKE_TOOLCHAIN_FILE to
|
|
point to vcpkg.cmake when configuring HDF5)
|
|
|
|
MacOS:
|
|
|
|
- brew install aws-c-s3
|
|
|
|
========================================================================
|
|
III. Building the aws-c-s3 library from source
|
|
========================================================================
|
|
|
|
If you are unable to obtain the aws-c-s3 library from a package manager
|
|
or other distribution method, the library may also be built from source
|
|
code. The following instructions have been reproduced and slightly
|
|
modified from the aws-c-s3 repository and may be out of date; be sure to
|
|
check https://github.com/awslabs/aws-c-s3 for the latest build
|
|
instructions.
|
|
|
|
1. Obtain the source code for aws-c-s3 and its dependencies
|
|
|
|
Note that this build process intends for aws-c-s3 and all its
|
|
dependencies to be cloned into the same top-level directory.
|
|
|
|
Only if building on Linux:
|
|
- git clone https://github.com/aws/aws-lc.git
|
|
- git clone https://github.com/aws/s2n-tls.git
|
|
|
|
- git clone https://github.com/awslabs/aws-c-common.git
|
|
- git clone https://github.com/awslabs/aws-checksums.git
|
|
- git clone https://github.com/awslabs/aws-c-cal.git
|
|
- git clone https://github.com/awslabs/aws-c-io.git
|
|
- git clone https://github.com/awslabs/aws-c-compression.git
|
|
- git clone https://github.com/awslabs/aws-c-http.git
|
|
- git clone https://github.com/awslabs/aws-c-sdkutils.git
|
|
- git clone https://github.com/awslabs/aws-c-auth.git
|
|
- git clone https://github.com/awslabs/aws-c-s3.git
|
|
|
|
2. Build aws-c-s3 and its dependencies
|
|
|
|
Note that this build process intends for <install-path> to be the
|
|
same directory for all commands. '--parallel <num-jobs>' is used
|
|
to speed up the build process by building with <num-jobs> concurrent
|
|
processes, and can be omitted if desired.
|
|
|
|
Only if building on Linux:
|
|
Build aws-lc:
|
|
- cmake -S aws-lc -B aws-lc/build -DCMAKE_INSTALL_PREFIX=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-lc/build --target install --parallel <num-jobs>
|
|
|
|
Build s2n-tls:
|
|
- cmake -S s2n-tls -B s2n-tls/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build s2n-tls/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-c-common:
|
|
- cmake -S aws-c-common -B aws-c-common/build -DCMAKE_INSTALL_PREFIX=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-c-common/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-checksums:
|
|
- cmake -S aws-checksums -B aws-checksums/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-checksums/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-c-cal:
|
|
- cmake -S aws-c-cal -B aws-c-cal/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-c-cal/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-c-io:
|
|
- cmake -S aws-c-io -B aws-c-io/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-c-io/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-c-compression:
|
|
- cmake -S aws-c-compression -B aws-c-compression/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-c-compression/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-c-http:
|
|
- cmake -S aws-c-http -B aws-c-http/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-c-http/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-c-sdkutils:
|
|
- cmake -S aws-c-sdkutils -B aws-c-sdkutils/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-c-sdkutils/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-c-auth:
|
|
- cmake -S aws-c-auth -B aws-c-auth/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-c-auth/build --target install --parallel <num-jobs>
|
|
|
|
Build aws-c-s3:
|
|
- cmake -S aws-c-s3 -B aws-c-s3/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
|
|
- cmake --build aws-c-s3/build --target install --parallel <num-jobs>
|
|
|
|
Note:
|
|
As shown in the above commands, it is suggested to pass the CMake
|
|
option BUILD_SHARED_LIBS=1 when building aws-c-s3 and each of its
|
|
dependencies if shared libraries will be enabled when building HDF5.
|
|
Otherwise, HDF5's CMake configuration logic might find and use a
|
|
libcrypto library from the system, which could result in linking errors
|
|
depending on the version of that library. If only static libraries will
|
|
be enabled when building HDF5, this option can be omitted.
|
|
|
|
Note:
|
|
The instructions above install aws-c-s3 and all of its dependencies
|
|
into the same directory with the use of CMAKE_INSTALL_PREFIX. This is
|
|
the suggested way of building aws-c-s3 as it simplifies the process of
|
|
locating aws-c-s3 and its dependencies with CMAKE_PREFIX_PATH, both
|
|
when building aws-c-s3 and HDF5, especially if installing aws-c-s3 to
|
|
a non-standard location.
|
|
|
|
========================================================================
|
|
IV. Building HDF5 with the ROS3 VFD enabled
|
|
========================================================================
|
|
|
|
HDF5 uses CMake's find_package() mechanism to locate the aws-c-library,
|
|
so as long as it is installed to a standard location, the CMake option
|
|
HDF5_ENABLE_ROS3_VFD=ON simply needs to be passed when configuring HDF5
|
|
in order to enable the ROS3 VFD, as in:
|
|
|
|
cmake -DHDF5_ENABLE_ROS3_VFD=ON ..
|
|
|
|
If the aws-c-s3 library is installed to a non-standard location, the
|
|
environment variable CMAKE_PREFIX_PATH should be set to that path when
|
|
configuring HDF5, as in:
|
|
|
|
CMAKE_PREFIX_PATH=<install-path> cmake -DHDF5_ENABLE_ROS3_VFD=ON ..
|
|
|
|
Refer to release_docs/INSTALL_CMake.txt for more general instructions on
|
|
building HDF5 with CMake.
|
|
|
|
========================================================================
|
|
V. Testing the ROS3 VFD
|
|
========================================================================
|
|
|
|
The ROS3 VFD is tested using a combination of docker, s3proxy and the
|
|
AWS CLI. If the 'docker' and 'aws' executables are available in standard
|
|
locations which CMake can find and the docker daemon is running, the
|
|
ROS3 VFD tests can be enabled by passing the CMake option
|
|
HDF5_ENABLE_ROS3_VFD_DOCKER_PROXY=ON, as in:
|
|
|
|
cmake -DHDF5_ENABLE_ROS3_VFD=ON -DHDF5_ENABLE_ROS3_VFD_DOCKER_PROXY=ON ..
|
|
|
|
If everything configured correctly, the ROS3 VFD tests can be run with:
|
|
|
|
ctest -R "S3TEST" -VV .
|
|
|
|
These tests are separated into different groups based on the particular
|
|
functionality being tested. For each group of tests, the CMake logic
|
|
will bring up s3proxy, create a testing bucket and populate that bucket
|
|
with any testing files needed, run all of the tests for the testing
|
|
group and then bring down s3proxy.
|