Commit Graph
118 Commits
Author SHA1 Message Date
0743df227f Add documentation and s3proxy testing for ROS3 VFD (#5560)
Adds workflow to build ROS3 VFD and optionally build aws-c-s3 library
from source or use package managers

Adds testing of ROS3 VFD with s3proxy and docker

Adds new H5Pset_fapl_ros3_endpoint()/H5Pget_fapl_ros3_endpoint() API
functions to set/get an alternative endpoint URL to use when opening
files with the ROS3 VFD

Cleans up warnings in tools and tests related to ROS3 VFD structure size

Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
2025-06-19 16:22:49 -05:00
Allen Byrne 5191c2b38a Use stream variables in tools (#5469) 2025-04-23 18:45:39 -05:00
Dana Robinson 7f1e49206d Renamed COPYING to LICENSE (#4978)
This is where most people will expect to find license information. The
COPYING_LBNL_HDF5 file has also been renamed to LICENSE_LBNL_HDF5.
The licenses are unchanged.
2024-10-18 21:13:04 -07:00
Dana Robinson 4dbaec7991 Remove HD from last non-win32 C89 functions (#4348)
Removes the HD prefix from the last C89 functions with no special
Windows equilvalent:

* rename
* rewind
* setbuf
* signal
* tmpfile
2024-04-08 07:58:13 -07:00
jhendersonHDF ce53bc020d Fixes for file format security issues (#4283) 2024-03-31 22:44:18 -05:00
Scot Breitenfeld a87ca572fe Replaces HDgetenv with getenv (#3599) 2023-09-26 13:11:22 -07:00
Dana Robinson 44a00ef876 Strip HD prefix from string/char C API calls (#3540)
* Strip HD prefix from string/char C API calls

* HD(f)(put|get)(s|c)
* HDstr*
* HDv*printf
* HD(s)(print|scan)f
* HDperror

But NOT:

* HDstrcase*
* HDvasprintf
* HDstrtok_r
* HDstrndup

As those are not C99 and have portability work-around
implementations. They will be handled later.

* Fix th5_system.c screwup
2023-09-15 15:13:18 -07:00
Dana Robinson ae1379094b hbool_t/TRUE/FALSE --> bool/true/false in tools (#3491) 2023-09-05 11:50:06 -07:00
305ac88865 ROS3: (feature) Temporary security credentials (#3030)
- Implemented support for AWS temporary security credentials.
  For this kind of credentials also a session/security token
  should be included in the request by adding the
  x-amz-security-token header.

Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Co-authored-by: Jordan Henderson <jhenderson@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2023-07-14 15:15:11 -05:00
Dana Robinson 7196d1e4c2 Remove HD from HDis* (e.g., isalpha) (#3212)
* HDisalnum
* HDisalpha
* HDiscntrl
* HDisdigit
* HDisgraph
* HDislower
* HDisprint
* HDispunct
* HDisspace
* HDisupper
* HDisxdigit
2023-06-29 12:47:16 -07:00
Dana Robinson 39e6bf48c9 Remove HD from HDmem* calls (#3211) 2023-06-29 12:33:46 -07:00
Dana Robinson fd933f30b1 Remove programmer/date from comments (#3210)
* Removes Programmer: and Date: fields
* Fixes a few Modifications: fields leftover from previous work
2023-06-29 12:13:29 -07:00
Dana Robinson 8aef67f0ae Remove HD from strto* calls (#3204)
* HDstrtod
* HDstrtol
* HDstrtoll
* HDstrtoul
* HDstrtoull
* HDstrtoumax
2023-06-29 08:18:49 -07:00
Dana Robinson 9f430d15b0 Rename HDexit() and related to exit(), etc. (#3202)
* HDatexit
* HDexit
* HD_exit
2023-06-29 08:18:01 -07:00
Dana Robinson aebac33a1f Remove HD from memory allocate/free calls (#3195)
* HDcalloc
* HDfree
* HDmalloc
* HDrealloc
2023-06-28 15:48:12 -07:00
Dana Robinson 187ea8a9ae Rename HD(f)printf() to (f)printf() (#3194) 2023-06-28 08:31:32 -07:00
Dana Robinson d81e751d41 Remove HDva_(arg|copy|end|start) (#3184) 2023-06-27 17:43:23 -07:00
Allen Byrne 445fcab52f Add no subsets option to h5diff like h5dump #2688 (#2756) 2023-04-18 11:21:18 -07:00
Larry Knox 11dfa25910 Update copyright headers (#2184)
* Updated source file copyright headers to remove "Copyright by the Board of Trustees
of the University of Illinois", which is kept in the top-level COPYING file.
2022-11-01 16:02:27 -05:00
luzpaz 174505ca3c Fix typos found in the rest of the hdf5 code-base (#1985)
* Fix typos found in the rest of the hdf5 code-base

* Typo in tool.cmake

* Revert "Typo in tool.cmake"

This reverts commit 06b8403d99.

* Fix clang-format styling
2022-08-10 13:57:26 -07:00
Allen Byrne ae414872f5 Develop clang 13 format (#1933)
* Update format source to clang 13

* More format changes
2022-07-26 14:45:46 -07:00
f859cb732b Fixed Spelling Errors (#1166)
* fixed missed closing of a dataset

* fixed missed closing of a dataset

* fixed typo in error return

* Committing clang-format changes

* minor edits

* code format

* Committing clang-format changes

* code format

* minor edit

* switched from using MPI_count, to actual bytes written for H5FD_mpio_debug rw debugging

* Committing clang-format changes

* changed size_i in printf to reflect the I/O.

* Committing clang-format changes

* Fixed seg fault with xlf on BE with -qintsize=8

* fixed error function string

* spelling corrections via codespell, added new spell check github actions

* Committing clang-format changes

* misc

* misc

* misc

* misc

* misc

* misc

* misc

* misc

* misc

* misc

* misc

* misc

* misc

* misc

* Committing clang-format changes

* misc

* misc

* misc

* misc

* misc

* misc

* Committing clang-format changes

* misc

* work around for https://github.com/codespell-project/codespell/issues/2137

* misc

* added missing file

* misc

* misc.

* misc

* switch to using Codespell with GitHub Actions

* misc.

* misc.

* fixed more sp errors

* Fix new typos found by codespell.

* fixed proceed with precede

* fixed variable in fortran test

* fixed minnum

* updated spelling list

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
2021-12-07 08:27:29 -06:00
Allen Byrne eca8d5b767 Fix ASAN issue in h5dump error path (#1051)
* Fix ASAN issue in h5dump error path

* Rework error allocation free.
2021-10-03 08:06:08 -05:00
Dana Robinson 2090a527c1 Brings the tools getopt(3) replacement into the main library (#803)
* Moves get_option from the tools library to the C library

* Adds H5 prefix to get_option call and variables

* Renames the H5_get_option long options struct and enum
2021-06-29 15:06:48 -05:00
Allen Byrneandgithub-actions e21f7aaac4 Tools long double updates (#522)
* OESS-98 convert plugin option to FetchContent, add tests

* Fixes for pkcfg files because of plugin option

* OESS-98 fix tools test for plugins

* Keep doxygen comments under 100 chars long - format hint

* Whitespace

* HDFFV-11144 - Reclassify CMake messages

* HDFFV-11099/11100 added help text

* Reworked switch statement to compare string instead

* Fix typo

* Update CDash mode

* Correct name of threadsafe

* Correct option name

* Undo accidental commit

* Note LLVM 10 to 11 format default changes

* Update format plugin

* Undo clang-format version 11 changes

* One more correction

* Update supported platforms

* Revert whitespace changes

* Correct whitespace

* Changes from PR#3

* HDFFV-11213 added option to control gcc10 warnings diagnostics

* HDFFV-11212 Use the new references correctly in JNI utility and tests

* format source

* Fix typo

* Add new test file

* HDFFV-11212 - update test and remove unused arg

* Minor non-space formatting changes

* Use H5I_INVALID_ID instead of "-1"

* source formatting

* add missing testfile, update jni function

* Undo commit of debug code

* remove mislocated file

* Fix h5repack test for handling of fapls and id close

* Update h5diff test files usage text

* HDFFV-11212 add new ref tests for JNI export dataset

* src format update

* Remove blank line typo

* src format typo

* long double requires %Lg

* Another long double foramt specifer S.B. %Lg

* issue with t128bit test

* Windows issue with h5dump and type.

* Fix review issues

* refactor function nesting and fix error checks

* format fixes

* Remove untested functions and javadoc quiet comments

* Restore TRY block.

* Change string append errors to memory exception

* revert to H5_JNI_FATAL_ERROR - support functions need work

* Add assertion error for h5util functions

* remove duplicate function

* format fix

* Revert HD function error handling

* Update copyright comments

* GH #386 java folder copyright corrections

* Whitespace

* GH #359 implement and fix tools 1.6 API usage

* remove excessive comments

* Flip inits to correct ifdef section

* rework ifdef to be simpler

* format issue

* Reformat ifdef inits

* remove static attribute

* format compliance

* Update names

* Revert because logic relies on float not being int

* Changes noticed from creating merge of #412

* Double underscore change

* Correct compiler version variable used

* Remove header guard underscores

* Whitespace cleanup

* Split format source and commit changes on repo push

* remove pre-split setting

* Change windows TS to use older VS.

* correct window os name

* HDFFV-11212 JNI export util and Javadoc

* Suggested review changes

* Another change found

* Committing clang-format changes

* HDFFV-11113 long double in tools

* HDFFV-11113 add note

* Disable long double tests for now

* HDFFV-11228 remove arbitrary CMake warning groups.

* Make each flag a string

* Some Javadoc warning fixes

* Updated javadoc fixes

* # WARNING: head commit changed in the meantime

HDFFV-11229 Fix long double usage in tools and java

Changed h5dump and h5ls to just print 128-bit for long double type.
Added test and file for dataset and attributes with all three float
types.

* Committing clang-format changes

* HDFFV-11229 Add release note

* HDFFV-11229 windows testfile needed

* fix typo

* Remove non supported message text

* HDFFV-11229 - change ldouble test to check both native and general

* HDFFV-11229 add second file compare

* HDFFV-11229 fix reference file

* HDFFV-11229 autotools check two refs

* HDFFV-11229 revert back to removal of NATIVE_LDOUBLE in tools output

* Committing clang-format changes

* Update release note

* Update attribute type of ref file

* Change source of ninja for macs

* try port instead of brew

* Recommended is to use brew.

* Undo non long double changes

* remove unneeded file

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2021-04-15 07:59:01 -05:00
Allen Byrne c7ffe683e5 Primary change is HDFFV-11212 - new refs and JNI (#372)
* OESS-98 convert plugin option to FetchContent, add tests

* Fixes for pkcfg files because of plugin option

* OESS-98 fix tools test for plugins

* Keep doxygen comments under 100 chars long - format hint

* Whitespace

* HDFFV-11144 - Reclassify CMake messages

* HDFFV-11099/11100 added help text

* Reworked switch statement to compare string instead

* Fix typo

* Update CDash mode

* Correct name of threadsafe

* Correct option name

* Undo accidental commit

* Note LLVM 10 to 11 format default changes

* Update format plugin

* Undo clang-format version 11 changes

* One more correction

* Update supported platforms

* Revert whitespace changes

* Correct whitespace

* Changes from PR#3

* HDFFV-11213 added option to control gcc10 warnings diagnostics

* HDFFV-11212 Use the new references correctly in JNI utility and tests

* format source

* Fix typo

* Add new test file

* HDFFV-11212 - update test and remove unused arg

* Minor non-space formatting changes

* Use H5I_INVALID_ID instead of "-1"

* source formatting

* add missing testfile, update jni function

* Undo commit of debug code

* remove mislocated file

* Fix h5repack test for handling of fapls and id close

* Update h5diff test files usage text

* HDFFV-11212 add new ref tests for JNI export dataset

* src format update

* Remove blank line typo

* src format typo

* long double requires %Lg

* Another long double foramt specifer S.B. %Lg

* issue with t128bit test

* Windows issue with h5dump and type.

* Fix review issues

* refactor function nesting and fix error checks

* format fixes

* Remove untested functions and javadoc quiet comments

* Restore TRY block.

* Change string append errors to memory exception

* revert to H5_JNI_FATAL_ERROR - support functions need work

* Add assertion error for h5util functions

* remove duplicate function

* format fix

* Revert HD function error handling

* Update copyright comments
2021-02-25 15:12:57 -06:00
Sean McBrideandLarry Knox c29e1b9fdf Patches from vtk (#358)
* Drop the export attribute on forward declaration

They don't mean anything on forward declarations anyways.

* Fixed -Wunused-parameter warnings in some C++ files

Allowed all the attribute #defines to work in C++ also.

* Fixed -Wunused-variable warnings by removing dead code

* Fixed all -Wshorten-64-to-32 warnings by adjusting casts

There was no truncation in fact, since the unsigned result was upcast to size_t then implicitly downcast to uint32_t.

* Fixed all -Wcomma warnings by spliting statements onto own lines

* Fixed all -Wself-assign warnings with different unused param suppression

* Fixed all -Wextra-semi warnings

* Fixed all -Wswitch-enum warnings

Just added new explicit cases with the existing default case.

* Fixed all -Wmissing-prototypes warnings

Just made functions static. Moved them into an extern "C" block.

* Reformatted source with bin/format_source using clang version 10.0.1

Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
2021-02-25 12:15:02 -06:00
Sean McBride 5ed255a607 Fixed all -Wincompatible-pointer-types-discards-qualifiers warnings (#341)
* Fixed various -Wincompatible-pointer-types-discards-qualifiers warnings by adding const

* Fixed various -Wincompatible-pointer-types-discards-qualifiers warning by removing extraneous consts

There were casts with const, but the function parameter doesn't actaully take const, so just modified the casts.

In the other case, a local variable was const that should not have been, becuase its source wasn't const either.

* Fixed a -Wincompatible-pointer-types-discards-qualifiers warning by strdup-ing a string

Create a duplicate string instead of mutating a supposedly const one.
2021-02-22 11:37:12 -06:00
Larry Knox 2ea165efd0 Update license url part2 (#333)
* Modify temporary rpath for testing in java example scripts.

* Update URL in source file Copyright headers for web copy of COPYING
    file - files not in src or test.
2021-02-17 08:52:36 -06:00
Allen Byrne 799ec0fde4 Small fixes (#285)
* OESS-98 convert plugin option to FetchContent, add tests

* Fixes for pkcfg files because of plugin option

* OESS-98 fix tools test for plugins

* Keep doxygen comments under 100 chars long - format hint

* Whitespace

* HDFFV-11144 - Reclassify CMake messages

* HDFFV-11099/11100 added help text

* Reworked switch statement to compare string instead

* Fix typo

* Update CDash mode

* Correct name of threadsafe

* Correct option name

* Undo accidental commit

* Small changes plus merge of tools arg parse from 1.12
2021-01-27 07:56:28 -06:00
Allen Byrne b2d661b508 Clang-format of source files 2020-09-30 09:27:10 -05:00
Allen Byrne 500d87fd1f HDFFV-9984 Add options to merge/prune external links during repack 2020-08-14 12:16:58 -05:00
Dana Robinson 36e8e2be4b Misc changes for h5dump VOL changes. 2020-03-30 01:51:36 -07:00
Jordan Henderson fa087c3919 Update Tools library to be better compatible with VOL connectors
Modify h5repack to integrate with VOL connectors

Update tools library to accomodate VOL connectors

Update logic in h5tools_fopen for VOL connectors

Add command-line options to h5repack for specifying in/out VOL
connectors

Implement h5tools_set_vol_fapl

Fix library shutdown issue

Integrate ROS3 and HDFS VFDs into new h5tools_get_fapl() scheme

Avoid H5Ocopy in h5repack when using different VOL connectors

Update h5tools_test_utils.c for ROS3 and HDFS integration
2020-03-17 14:25:36 -05:00
Dana Robinson a92c735c9b Squashed commit of the token_refactoring branch: 2020-01-16 13:29:34 -08:00
Allen Byrne 341600b4a6 Update tools hid_t declarations with H5I_INVALID_HID 2020-01-13 17:10:09 -06:00
Jordan Henderson 1795aa660a Clean up tools warnings introduced by H5TOOLS_ERR_INIT macro 2019-12-30 18:02:06 -06:00
Jordan Henderson 2cbf31cb3a Refactor tools library error handling macros 2019-12-28 14:08:04 -06:00
Allen Byrne 628c63f125 HDFFV-10980 - h5diff uses new ref APIs 2019-12-19 16:29:43 -06:00
Allen Byrne 7e92c2e3e5 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '0772b975d1d2bfa15aedeb4b6e2c2aac78c61a2f':
  Fix missing free in H5T__ref_mem_read()
  Fix bugs in H5VL file comparison code.  Add short circuit success to H5VL_cmp_connector_cls().
  Implement file comparison VOL callback.  Other changes to allow references to work with non-native connectors.  There is a bug somewhere.
  Add support for point selections to H5S_select_project_intersection.
  Remove H5VL_NATIVE_FILE_GET_FILE_ID and add H5VL_OBJECT_GET_FILE
  Revert "Make a squash commit of 'Quiet some warnings by adjusting warnings level and fixing some code.' (commit 5c911d8baf)"
  Revert "Oops, remove more C99 designated initializers for VS 2010 compatibility."
  H5R: fix H5Tconv to check for null references
  Trivial parameter renaming in VOL API calls.
  Move checking for zero offset in selection adjust calls to the selection callbacks.  This makes the procedure for checking it consistent across selection types and between _s and _u, ensures it is always is performed even when called within the H5S package, and removes the redundant check that would occur when callins H5S_select_adjust_s() from outside the H5S package.
  Replace H5Sselect_adjust_u() and H5Shyper_adjust_s() with H5Sselect_adjust.  Implement "adjust_s" callback for all selection types.  Add range checking to H5Sselect_adjust().
2019-12-02 08:00:20 -06:00
David Young 0108072f7a Revert "Make a squash commit of 'Quiet some warnings by adjusting warnings level and fixing some code.' (commit 5c911d8baf)" 2019-11-27 11:26:02 -06:00
Allen Byrne dcae4544a3 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit 'a8892bb42d6f6e4fbc30fae0eb2b957f81c938b9': (45 commits)
  Oops, remove more C99 designated initializers for VS 2010 compatibility.
  Add an #include to get a function declaration.
  Don't use C99 designated initializers, they're not compatible with Visual Studio 2010.
  Quiet some more maybe-uninitialized warnings---each is a false positive, *sigh*.  This is more code that may not compile with VS2010, *sigh sigh*.
  Always warn on maybe-uninitialized.  -Wincompatible-pointer-types was not available until GCC 5, so enable it only if that's the GCC version we're using.
  Only promote maybe-uninitialized warnings to errors on GCC 8.  Even on GCC 8, there may be false positives at low optimization levels?  I need to check.
  Only use -Werror=cast-function-type with GCC 8 and later.
  Put all of the -W options back into the order I found them in so that it's easier to compare old and new config/gnu-flags.
  Add new source files to CMakeLists.txt.
  Mention the -Werror= flags in libhdf5.settings.in.
  free -> HDfree
  Promote decleration-after-statement warnings to errors.
  Quiet decleration-after-statement warnings.
  Move a statement under some declarations since some vintages of Visual Studio don't like declarations after statements.
  Document H5D__chunk_mem_xfree_wrapper().
  Undo accidental test deletion.
  Oops, delete a debug printf that snuck in here.
  Undo my changes to the HD macros, hadn't really intended those to be on this branch....
  Make errors of some more warnings.  Move disabled warnings to DEVELOPER_WARNING_CFLAGS.  Put just one warning option on a line, and sort some of the options.
  Cast to the parameter type, H5VL_token_t *, instead of to unsigned char *.
  ...
2019-11-25 18:21:14 -06:00
David Young 0cb24f8ad4 Mark an unused parameter H5_ATTR_UNUSED. 2019-11-13 11:25:27 -06:00
Allen Byrne 6b927a773c HDFFV-10876 Update h5dump and h5ls for new ref api. 2019-11-07 09:49:11 -06:00
Dana Robinson 8c40dab59e Untangle #ifdef mess in ROS3 code. 2019-08-19 18:02:47 -07:00
Dana Robinson 5b6bd7c922 Added the map (H5M) API 2019-08-13 01:36:42 -07:00
Larry Knox 8008294578 Squashed commit of the following:
Merge changes from update_merged_S3_HDFS branch into develop.

commit d5034315aea88629929ac0c9c59ebfafd5f21a31
Merge: 9c48823 d3fdcd8
Author: Larry Knox <lrknox@hdfgroup.org>
Date:   Thu Jul 25 08:24:53 2019 -0500

    Merge branch 'develop' into update_merged_S3_HDFS
2019-07-25 11:47:12 -05:00
Binh-Minh Ribler 90d13bef33 Fixed typo
Platforms tested:
    Darwin (osx1010test)
2019-01-07 20:46:55 -06:00
Binh-Minh Ribler 78d0564c2a Refixed HDFFV-10578
Description:
    Applied Neil's fix for this issue after removing previous
    attempt.  The resources are now released in init_objs() when
    failure occurs there.
    Neil will fix HDFFV-10676 separately.
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1010test)
2019-01-07 20:36:34 -06:00
Vailin Choi b178d80be0 Changes made based on feedback from pull request #1039. 2018-05-14 12:26:48 -05:00