Issue #6616: Group Info and Link Info are labeled Optional, but
new-style group creation always writes both, and the C library errors
reading a new-style group missing either one.
- Group Info / Link Info Status: Optional -> Required for "new style"
groups; may not be repeated (matches existing Symbol Table wording
for old-style groups).
- Documents the decoder's fallback values (8, 6) for Link Phase
Change: Maximum Compact/Minimum Dense Value when absent.
Issue #6610: the Enumeration datatype's Names field claimed member
names are ASCII, though the reference implementation doesn't enforce
encoding; and the Compound datatype's Name field description was
copy-pasted from the Opaque datatype section.
- Compound Datatype Name field (Versions 1, 2, 3): describes the
member name instead of the opaque-type description.
- Enumeration Datatype Names field (Versions 1/2, 3): drops the ASCII
claim; documents the actual constraint (no NUL byte within the
name) and that other encodings, such as UTF-8, aren't rejected.
* Document the Metadata Cache Image block and header message
Adds docs/doxygen/dox/H5.format.4.0.dox coverage for:
- Level 1J: the Metadata Cache Image Block (subsec_fmt4_infra_mdci)
- IV.A.3.y: the Metadata Cache Image header message (0x0018)
- TOC/cross-reference entries and the superblock-extension list entry for both
Also clarifies the 'Size of Header Message #n Data' field description
for v1 and v2 object header messages to note it includes flags/reserved
(v1) or flags/creation-order (v2) fields, not just type and length.
* Address review feedback on MDCI message docs
- Clarify that the Metadata Cache Image message and its image block
are only removed/reclaimed on the next file open when the file is
opened for read/write access; read-only opens leave them unchanged.
- Fix a malformed table row (missing closing </td>) in the Status
field of the IV.A.3.y message table.
- The layout of the shared message encoding was actually decribed at the beginning of the section IV.A.2.
For clarification, the section was split into two with one section purely for the shared message encoding
and another section for the catalog of message types.
- Cross-linked shareable messages to the shared message encoding section.
- Misc. cleanup
* Fix for issue #6430: add decription for H5Tencode/H5Tdecode to Appendix.
* Fix for issue #6392:
The fix addressed part 1 of the issue.
No change is needed for part 2 of the issue: verify that the info is already there in the Driver Info Message.
* Fix for issue #6360
* Fix for issue #6416 in section III.I. Disk Format: Level 1I - Shared Object Header Message Table.
---------
Co-authored-by: Matt L <124107509+mattjala@users.noreply.github.com>
* Fix for issue #6430: add decription for H5Tencode/H5Tdecode to Appendix.
* Fix for issue #6392:
The fix addressed part 1 of the issue.
No change is needed for part 2 of the issue: verify that the info is already there in the Driver Info Message.
* Fix for issue #6360
---------
Co-authored-by: Matt L <124107509+mattjala@users.noreply.github.com>
* Fix for issue #6430: add decription for H5Tencode/H5Tdecode to Appendix.
* Fix for issue #6392:
The fix addressed part 1 of the issue.
No change is needed for part 2 of the issue: verify that the info is already there in the Driver Info Message.
* Fixes for issues: #6448, #6449, #6444
* Fix for issue #6443.
* Fix for issue #6365.
* Fix spelling error.
* Modifications based on PR review comments.
* Correct spelling error.
* Replaced 3 duplicate field descriptions with cross-reference to the version 3 layout message.
* Refactor the description for the layout message regarding the dimension related fields.
---------
Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org>
* Fix for issue #4849 that settings in fapl libver bounds causes unexpected H5Fopen failures.
File with non-SWMR-write access can now be opened without regard for superblock version.
Due to the fix, H5Fstart_swmr_write() also needs to be modified as well as the tests for libver bounds.
The "RFC: Setting Bounds for Object Creation in HDF5 1.10.0" is also updated to reflect the changes.
* Fix c++ libver bound test failure.
* Fix for github issue #1388: can't delete renamed dense attribute with corder tracking enabled
The problem occurs in step 3(b) below which will delete the attribute with corder x
from the creation order index v2 B-tree.
The rename sequence in H5A__dense_rename() occurs in the following order:
1) The old attribute with corder x was removed from the creation order index v2 B-tree
2) The new renamed attribute was inserted via H5A__dense_insert():
(a) insert the attribute with new name j into the name index v2 B-tree
(b) insert the attribute with corder x into the creation order index v2 B-tree
3) The old attribute was removed via H5A__dense_remove():
(a) remove the attribute with old name k from the name index v2 B-tree
(b) remove the attribute with coorder x from the creation order index v2 B-tree
Fix: deactivate the "corder_bt2_addr" field so that H5A__dense_remove()
won't delete the attribute with corder x from the creation order index v2 B-tree.
Document the limitation in the Passthrough Conncector section of the VOL Connector Author Guide.
The limitation is posted by Neil in the github issue on Dec 22, 2022.
* Fix for github issue #3790: infinite loop closing library
Cause of the problem:
When h5dump tries to open the user provided test file, the metadata cache will
call the "get_final_load_size" callback to find out the actual size of the
the root object header. The callback function will call
H5O__prefix_deserialize() to allocate space for the object header
data structure (via H5FL_CALLOC) and to deserialize the object header prefix
in order to find the actual size of the object header.
The metadata cache will then check whether the actual size obtained
will exceed the file's EOA.
Since the actual size obtained from the test file exceeds the EOA,
the metadata cache throws an error and return.
However, the oh structure that was allocated in H5O__prefix_deserialize()
was not freed and hence causing the problem described in this issue.
Fix:
1) Deallocate the oh structure after obtaining and saving the needed
information in udata which will be used later on in the "verify_chksum" callback.
2) Deserialize the object header prefix in the "object header's
"deserialize" callback regardless. The original coding intends to keep the
deserialized prefix so that the object header's "deserialize" callback
does not need to deserialize the prefix again if the object header is coming
through the "get_final_load_size" callback.
* Fix for github issue #2414: segfault when copying dataset with attributes.
This also fixes github issue #3241: segfault when copying dataset.
Need to set the location via H5T_set_loc() of the src datatype
when copying dense attributes.
Otherwise the vlen callbacks are not set up therefore causing seg fault
when doing H5T_convert() -> H5T__conv_vlen().
* Changes for ECP-344: Implement selection vector I/O with collective chunk filling.
Also fix a bug in H5FD__mpio_write_vector() to account for fixed size optimization
when computing max address.
* Fixes based on PR review comments:
For H5Dchunk.c: fix H5MM_xfree()
For H5FDmpio.c:
1) Revert the fix to H5FD__mpio_write_vector()
2) Apply the patch from Neil on the proper length of s_sizes reported by H5FD__mpio_vector_build_types()
* Put back the logic of dividing up the work among all the mpi ranks similar to the
original H5D__chunk_collective_fill() routine.
* Add a test to verify the fix for the illegal reference problem in H5FD__mpio_write_vector().
It is because the object in the testfile points back to the root group.
When the tool tries to traverse the object, it goes back to the root group and then back again.
* Fix for the bug exposed from running test/set_extent.c when selection I/O is enabled.
This is a fix from Neil.
The test/set_extent.c is modified to test for selection I/O enabled.
* Fix for github issue #2599:
As indicated in the description, memory leak is detected when running "./h5dump pov".
The problem is: when calling H5O__add_cont_msg() from H5O__chunk_deserialize(),
memory is allocated for cont_msg_info->msgs. Eventually, when the library tries to load
the continuation message via H5AC_protect() in H5O_protect(), error is
encountered due to illegal info in the continuation message.
Due to the error, H5O_protect() exits but the memory allocated for cont_msg_info->msgs is not freed.
When we figure out how to handle fuzzed files that we didn't generate,
a test needs to be added to run h5dump with the provided "pov" file.
* Add message to release notes for the fix to github issue #2599.
* Fix up the H5Pencode2 test for virtual layout.
* Committing clang-format changes
* Remove VERIFY and use TEST_ERROR.
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fix for HDFFV-11052: h5debug fails on a corrupted file (h5_nrefs_POC) producing a core dump.
When h5debug closes the corrupted file, the library calls H5F__dest() which performs all the
closing operations for the file "f" (H5F_t *) but just keeping note of errors in "ret_value"
all the way till the end of the routine. The user-provided corrupted file has an illegal
file size causing failure when reading the image during the closing process.
At the end of this routine it sets f->shared to NULL and then frees "f".
This is done whether there is error or not in "ret_value".
Due to the failure in reading the file earlier, the routine then returns error.
The error return from H5F__dest() causes the file object "f" not being removed from the
ID node table. When the library finally exits, it will try to close the
file objects in the table. This causes assert failure when H5F_ID_EXISTS(f) or H5F_NREFS(f).
Fix:
a) H5F_dest(): free the f only when there is no error in "ret_value" at the end of the routine.
b) H5VL__native_file_close(): if f->shared is NULL, free "f"; otherwise, perform closing on "f" as before.
c) h5debug.c main(): track error return from H5Fclose().
* Committing clang-format changes
* Add test and release note info for fix to HDFFV-11052 which is merged via PR#2291.
* Committing clang-format changes
* Add the test file to Cmake.
* Skip test_misc37() for drivers that is not default compatible as it is using a pre-generated file.
* Committing clang-format changes
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fix for HDFFV-11052: h5debug fails on a corrupted file (h5_nrefs_POC) producing a core dump.
When h5debug closes the corrupted file, the library calls H5F__dest() which performs all the
closing operations for the file "f" (H5F_t *) but just keeping note of errors in "ret_value"
all the way till the end of the routine. The user-provided corrupted file has an illegal
file size causing failure when reading the image during the closing process.
At the end of this routine it sets f->shared to NULL and then frees "f".
This is done whether there is error or not in "ret_value".
Due to the failure in reading the file earlier, the routine then returns error.
The error return from H5F__dest() causes the file object "f" not being removed from the
ID node table. When the library finally exits, it will try to close the
file objects in the table. This causes assert failure when H5F_ID_EXISTS(f) or H5F_NREFS(f).
Fix:
a) H5F_dest(): free the f only when there is no error in "ret_value" at the end of the routine.
b) H5VL__native_file_close(): if f->shared is NULL, free "f"; otherwise, perform closing on "f" as before.
c) h5debug.c main(): track error return from H5Fclose().
* Committing clang-format changes
* Add test and release note info for fix to HDFFV-11052 which is merged via PR#2291.
* Committing clang-format changes
* Add the test file to Cmake.
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fix for HDFFV-11052: h5debug fails on a corrupted file (h5_nrefs_POC) producing a core dump.
When h5debug closes the corrupted file, the library calls H5F__dest() which performs all the
closing operations for the file "f" (H5F_t *) but just keeping note of errors in "ret_value"
all the way till the end of the routine. The user-provided corrupted file has an illegal
file size causing failure when reading the image during the closing process.
At the end of this routine it sets f->shared to NULL and then frees "f".
This is done whether there is error or not in "ret_value".
Due to the failure in reading the file earlier, the routine then returns error.
The error return from H5F__dest() causes the file object "f" not being removed from the
ID node table. When the library finally exits, it will try to close the
file objects in the table. This causes assert failure when H5F_ID_EXISTS(f) or H5F_NREFS(f).
Fix:
a) H5F_dest(): free the f only when there is no error in "ret_value" at the end of the routine.
b) H5VL__native_file_close(): if f->shared is NULL, free "f"; otherwise, perform closing on "f" as before.
c) h5debug.c main(): track error return from H5Fclose().
* Committing clang-format changes
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fix for HDFFV-10840: Instead of using fill->buf for datatype conversion
if it is large enough, a buffer is allocated regardless so that the element
in fill->buf can later be reclaimed.
Valgrind is run on test/set_extent.c and there is no memory leak.
* Add information of this fix to release notes.
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
vchoi-hdfgroup
