Added infrastructure support for multithreaded concurrency by adding an optional way to switch to using a non-recursive R/W lock for the global API lock. This is enabled with a new 'concurrency' configuration flag for the autotools & CMake builds, which is disabled by default.
When the 'concurrency' build option is chosen, the global API lock will use the R/W lock and all API calls currently will acquire a write lock, ensuring exclusive access by one thread. Over time, the API routines that are converted to support multithreaded concurrency will switch to acquiring a read lock instead.
Reentering the library from application callbacks is managed by the 'disable locking for this thread' (DLFTT) threadsafety protocol. This is internally handled within the H5_API_LOCK / H5_API_UNLOCK macros in H5private.h (as before), which invoke the 'dlftt' routines in H5TSint.c.
To support this change, the threadsafety configuration macros for the library have been updated:
- --enable-threadsafe now defines the H5_HAVE_THREADSAFE macro
- --enable-concurrency defines the H5_HAVE_CONCURRENCY macro
The new H5_HAVE_THREADSAFE_API macro is set if either H5_HAVE_THREADSAFE or H5_HAVE_CONCURRENCY is enabled.
New Github actions are added to include the concurrency configuration in the CI for the develop branch.
To support the new non-recursive R/W locking for API routines, some other changes are necessary:
Added macro wrappers around all callback invocations that could call an
application function, and therefore re-enter the library:
H5_BEFORE_USER_CB* / H5_AFTER_USER_CB*
Added H5_user_cb_prepare / H5_user_cb_restore routines that save the
state of the library when callback leaves the library. Includes error
stack and threadsafe reentry state currently.
There's also some small cleanups to various places in the library:
Moved the H5E_mpi_error_str / H5E_mpi_error_str_len globals to be local for
pushing MPI errors, so that multiple threads can't interfere with each
other.
Added H5TS_rwlock_trywrlock() routine to R/W lock interface.
Emulate R/W locks on MacOS because its implementation of
pthread_rwlock_wrlock() does not conform to the POSIX standard.
Don't acquire the global API lock in H5close, since it's acquired in H5_term_library, which is necessary because H5_term_library is invoked via other code paths that don't hold the global API lock.
Don't call H5Eget_auto2 API routine within H5_term_library.
Switched 'return NULL' in H5allocate_memory to HGOTO_DONE(NULL).
Switched H5Pget_file_space_strategy / H5Pset_file_space_strategy to use
internal routines instead of API routines.
Switched H5Oopen_by_addr & H5Ovisit1 to use internal routines instead of
API routines.
Fixed a few places in src/H5Odeprec.c where a major
error ID was passed as a minor ID.
This is where most people will expect to find license information. The
COPYING_LBNL_HDF5 file has also been renamed to LICENSE_LBNL_HDF5.
The licenses are unchanged.
* Strip HD prefix from string/char C API calls
* HD(f)(put|get)(s|c)
* HDstr*
* HDv*printf
* HD(s)(print|scan)f
* HDperror
But NOT:
* HDstrcase*
* HDvasprintf
* HDstrtok_r
* HDstrndup
As those are not C99 and have portability work-around
implementations. They will be handled later.
* Fix th5_system.c screwup
* Updated source file copyright headers to remove "Copyright by the Board of Trustees
of the University of Illinois", which is kept in the top-level COPYING file.
* Committing clang-format changes
* Converted BEGIN_FUNC, etc. macros to FUNC_ENTER
* Rips out the BEGIN_FUNC, etc. macros
* Removes 'end if', etc. comments from H5HL package
* Committing clang-format changes
* Fixes an H5EA iterate issue
* Fixes an issue in the H5FA iterator code
* Further cleanup in bin/trace after macro removal
* Iterator changes in H5EA and H5FA
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* OESS-98 convert plugin option to FetchContent, add tests
* Fixes for pkcfg files because of plugin option
* OESS-98 fix tools test for plugins
* Keep doxygen comments under 100 chars long - format hint
* Whitespace
* HDFFV-11144 - Reclassify CMake messages
* HDFFV-11099/11100 added help text
* Reworked switch statement to compare string instead
* Fix typo
* Update CDash mode
* Correct name of threadsafe
* Correct option name
* Undo accidental commit
* Note LLVM 10 to 11 format default changes
* Update format plugin
* Undo clang-format version 11 changes
* One more correction
* Update supported platforms
* Revert whitespace changes
* Correct whitespace
* Changes from PR#3
* HDFFV-11213 added option to control gcc10 warnings diagnostics
* HDFFV-11212 Use the new references correctly in JNI utility and tests
* format source
* Fix typo
* Add new test file
* HDFFV-11212 - update test and remove unused arg
* Minor non-space formatting changes
* Use H5I_INVALID_ID instead of "-1"
* source formatting
* add missing testfile, update jni function
* Undo commit of debug code
* remove mislocated file
* Fix h5repack test for handling of fapls and id close
* Update h5diff test files usage text
* HDFFV-11212 add new ref tests for JNI export dataset
* src format update
* Remove blank line typo
* src format typo
* long double requires %Lg
* Another long double foramt specifer S.B. %Lg
* issue with t128bit test
* Windows issue with h5dump and type.
* Fix review issues
* refactor function nesting and fix error checks
* format fixes
* Remove untested functions and javadoc quiet comments
* Restore TRY block.
* Change string append errors to memory exception
* revert to H5_JNI_FATAL_ERROR - support functions need work
* Add assertion error for h5util functions
* remove duplicate function
* format fix
* Revert HD function error handling
* Update copyright comments
* GH #386 java folder copyright corrections
* Whitespace
* GH #359 implement and fix tools 1.6 API usage
* remove excessive comments
* Flip inits to correct ifdef section
* rework ifdef to be simpler
* format issue
* Reformat ifdef inits
* remove static attribute
* format compliance
* Update names
* Revert because logic relies on float not being int
* Changes noticed from creating merge of #412
* Double underscore change
* Correct compiler version variable used
* Remove header guard underscores
* Whitespace cleanup
* Modify temporary rpath for testing in java example scripts.
* Update URL in source file Copyright headers for web copy of COPYING
file - files not in src or test.
Modify h5repack to integrate with VOL connectors
Update tools library to accomodate VOL connectors
Update logic in h5tools_fopen for VOL connectors
Add command-line options to h5repack for specifying in/out VOL
connectors
Implement h5tools_set_vol_fapl
Fix library shutdown issue
Integrate ROS3 and HDFS VFDs into new h5tools_get_fapl() scheme
Avoid H5Ocopy in h5repack when using different VOL connectors
Update h5tools_test_utils.c for ROS3 and HDFS integration
* commit '54957d37f5aa73912763dbb6e308555e863c43f4':
Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes.
Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh.
Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1
Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
Enable tools lib to be built as a dll on windows. Added two get/set functions for progname and d_status.
Also add windows import/export declarations to functions.
Updated error_mesg() and warn_mesg() to remove progname argument and use get functions
Tested:
Windows, linux
Bring r18076-18096 from hdf5_1_8_coverity branch to trunk:
r18076:
Correct Coverity issue #1 by removing dead code
r18077:
Fix coverity item 142. When an error occurred while copying a linked list in
H5S_point_copy, the library would not free the partially allocated list. Added
code to free the list in this case.
r18078:
Correct Coverity issue #2 by removing impossible to reach code.
r18079:
Correct #3 by removing impossible to reach code.
r18080:
Correct Coverity issue #4 by removing impossible to reach code.
r18081:
fix coverity 26 , check (dblik->parent) before calls H5HF_man_iblock_detach().
r18082:
Fixed coverity issues 321 and 316.
321: freed sm_buf in error handling to remove resource leak.
Also set sm_buf to NULL after other instances in which
it is freed to prevent double free.
316: initialized nmembs to 0.
r18083:
Correct Coverity issue #6 by removing debugging knob from error reporting code.
r18084:
Fix coverity item 269 + others. When a error occurred in a function using the
h5tools error framework, the "past_catch" variable would not be set to true
because that statement was before the label that goto jumped to. This could
cause a failure in the cleanup section to go back to the start of the section,
freeing variables twice, etc.
Moved the label infront of past_catch=TRUE.
r18085:
fixed coverity #27, check if (heap) before use heap->obj....
r18086:
fixed coverity #28, check curr_span not null before use it at if(curr_span &&
(io_bytes_left==0 || curr_seq>=maxseq))
r18087:
Correct Coverity issue #7 by cleaning up correctly on error
r18088:
Correct Coverity #8 by removing unchanged variable checking code.
r18089:
Correct Coverity issue #9 - remove impossible to reach code.
r18090:
Correct Coverity issue #11 by removing impossible to reach code. Also clean
up some minor style issues.
r18091:
Fix coverity items 314 and 318. Changed the improper assertion of the return
value of a library function to a check, and a return(void) on failure.
r18092:
Fix coverity item 70. Changed the improper assertion of the return value of a
library function to a check, and a return(void) on failure.
r18093:
Correct Coverity issue #12 by removing dead code.
r18094:
Correct Coverity issue #16 by removing debugging code.
r18095:
Fixed coverity issue # 271.
Removed redundant checking and freeing of sm_buf1 and sm_buf2.
r18096:
Correct Coverity issue #17 by refactoring test to remove dead code.
Also, removed previous "coverity" statements in comments, we'll review
those issues again and see if we can figure them out, now that we have more
experience with Coverity.
Tested on:
Mac OS X/32 10.6.2 (amazon)
Bring Coverity changes into the trunk: (also other minor cleanups)
r17991:
Fix Coverity items 175 and 176. Fixed memory leak on error in print_enum in
H5LT.c.
r17993: (r17992 was not a Coverity change)
Close Coverity issue #206: inconsistently checking whether dt->shared was
non-NULL after H5T_alloc() returned a valid 'dt' value (which should guarantee
that dt->shared is valid).
r17994:
Fix Coverity item 149. Fixed file handle leak on error in H5FD_stdio_open.
r17995:
Fixed Coverity issues 154 to 161:
Added H5MP_close routine to error handling in the event *mp has not been
freed before error.
r17996:
Close Coverity issue #126: potentially leaking merged_spans on routine failure.
r17997:
Fix Coverity items 147 and 148. Fixed resource leaks on error in H5FDloc.c.
r17998:
Coverity issue 269-272:
Added integer result variable to functions that could return negative.
Assigned to unsigned after checking.
Added H5E_BEGIN_TRY block around H5Tclose and removed H5E_THROW in the
catch block.
Checked buffer is NULL before free. Changed HGOTO_ERROR outside of the if
block to H5E_THROW.
r17999:
Close Coverity issue #127: release temporary spans in more generic manner.
(Also add error checking to previous fix)
r18000:
Resolved Coverity issues 211 and 212 in H5T.c. Added comments to ignore
Coverity warning regarding not checking pointer for NULL, as we are using
an assert which catches the issue.
r18001:
Fix Coverity item 146. Fixed resource leak on error in H5O_layout_copy.
r18002:
Fix Coverity items 143 and 145. Fixed resource leaks on error in
H5D_compact_copy and H5D_contig_copy.
r18003:
Close Coverity issue #192: close file on error
r18004:
Fix Coverity issue #125: release temporary spans on error
r18005:
Resolved Coverity issues 5, 25, and 83 (in H5T.c):
Separated embedded functions in order to check for NULL on return of
H5I_object before passing into H5T_copy.
Check to see if new_dt is NULL within error handling before
dereferencing it.
Ignore Coverity's dead code warnings as the checks that lead to the code
are machine dependent.
r18006:
Coverity 63,70,73: Checked result of function before assigning to an unsigned
variable.
r18007:
Coverity 78,79: added continue statement if H5Pget_filter2 returns negative.
r18008:
Fixed Coverity issue # 138:
Added support in error handling to free dst pointer (if allocated) on error.
r18009:
Whitespace & coding style cleanup
Quincey Koziol
Dana Robinson
Sean McBrideandgithub-actions
Larry Knox
Allen Byrne
Allen Byrne
Jordan Henderson
Quincey Koziol